Speaker: Trish Khoo, Google (@hogfish – trishkhoo.com)
Geo at Google Scale
- 1 billion active monthly users for Google Maps Services
- 1 billion downloads of Google Maps on Android
- 2 million active we sties & apps use the API every week
- 10,000s of user edits to local data and map data every day
Satellite images cover 45% of the world’s land surface and 80% of the world’s population in sub-meter high resolution
“We often find that when doing due dliigence, that the team got the thing up and running as fast as possible, and that’s fine for getting to 1-5M euros in revenue, but they’ve created scaling issues for themselves after that. ”
– Simon Menashy, MMC Ventures
Typical Solution #1: Hire a Q/A team to fix the quality problem
- Don’t do this
- Manual testting is slow
- Q/A is accountable, but has little power to change anything
- How many Q/A testers does it take to change a lightbulb?
- None. They’ll just tell you the room is dark.
Typical Solution #2: Hire automation team to fix the testing problem
- Write a program to run the tests faster than humans
- Automated test = good. Automated testing team = bad.
- Automated testers tend to be worse programmers
- Technology for doing automated UI testing still poor
7 years ago Google teams released to production once or twice a month, with the majority of the time taken by testing. Today Google teams release to production daily or sub-daily, with no decrease in quality.
Correct Solution: The Whole Team Commits To Quality
The “Testing” Phase – Bad
- No idea when the product will actually launch
- Massive amount of human hours to manage effectively
- Huge wast in expensive engineering hours
What is “Testing”?
- Testing provides expert-driven feedback on the state of business requirement gaps, user impact and overall project quality.
- One method of doing this is checking things. But everyone can check things. It’s easy. Walk through a checklist.
Better: Developer Checks & Tests Own Code
- Writes own unit and functional tests
- Runs through test scenarios
Checking Is Easy. Testing Is Hard.
- Testing experts on your team can help:
- Make sure people get info about your product at the optimal time to make timely decisions
- Make your development process more efficient
- Design tools and infrastructure to make your developers more productive
Change Your Company Culture
- Typical pushback:
- No time to write tests
- Don’t know how to write tests
- Not how we do things at this company (reward system focused on speed, not reliability)
- Fast developers are not necessarily good developers.
- Evaluate developers not on how many features they get done, but how bug free those features are?
Hire the Right Skills
Google hires two type of test engineers:
- Software Engineer, Tools & Infrastructure: same standard as regular engineers. Create new tools & infrastructure and helps train developers.
- Test Engineer: Higher in test experience. Look at test processes and make team more efficient. Not as high on technical ablity, but high ability in testing.
Empower Your Testing Expert
- First class citizen within the team
- Similar job title and salary to developers
- Needs management support
- NOT the same level as a manual tester
Learn, Don’t Blame
- Cultivate a postmortem culture
- Postmortem is a document created when process fails, written as a factual document by the person who has the most experience with what went wrong. Includes ideas how to avoid the problem in the future.
- Creates a culture that accepts failure, which allows people to take risks
- A culture that accepts failure helps you identify problems earlier, since people aren’t afraid of reporting issues
- Trust your people
Fast Releases, Fast Feedback
- Increases feedback cycle to improve faster
Experiments & Dogfood
- Run experiments on small percentage of the users
- Try out your product as a user yourself before you release it
- Great for finding usability issues
- Helps employees & developers get closer to the product and the customer
Why Do This?
- Rock solid software
- Ablity quickly change production direction
- Freedom to innovate
Questions
- Do developers do the regression testing?
- Yes. Regression testing is the main thing we want developers to do. It’s the main “checking” we want developers to do.
- One developer said he sees automated regression testing as protection for his code against other developers. Helps you avoid issues with code that you don’t know about.
- Acts as documentation to tell you what the code was supposed to do in the first place.
- When you have a large application with a large team, how do you handle testing the application when a developer’s code affects a large application?
- End-to-end testing is a pain. Unit testing is great. Having intermediate tests help.
- Break the application down into smaller concerns. Test in these smaller concerns.
- If developers don’t understand the requirements, they’ll test it incorrectly. How do you handle that?
- Need to write out user scenarios better
- What percentage code coverage do you strive for on your team?
- Depends on the type of application you’re building and how well it works with unit testing.
- If you’re not doing unit testing yet, do an incremental code coverage: what percentage of new code has unit tests
- What are the differences to your process for testing when releasing to the cloud where you can release multiple times per day versus releasing desktop or mobile apps where you have limited release frequency?
- Even if you aren’t releasing to production as frequently, it’s important to have “releasable” code as often as possible.
- How do you train developers to have a tester mentality?
- Excellent book: Explore It! by Elizabeth Hendrickson
- What are the other benefits of test-driven development?
- Helps developers think about design.
- Forces developers to build their product in a testable way, which avoids refactoring headaches later on.
- Also serves as documentation,
Leave a Reply