How Google TestsSoftware - Part Two

In order for the “you buildit, you break it” motto to be real, there are roles beyond the traditionaldeveloper that are necessary. Specifically, engineering roles that enable developersto do testing efficiently and effectively have to exist. At Google we havecreated roles in which some engineers are responsible for making others moreproductive. These engineers often identify themselves as testers but theiractual mission is one of productivity. They exist to make developers moreproductive and quality is a large part of that productivity. Here's a summaryof those roles:

The SWE or Software Engineeris the traditional developer role. SWEs write functional code that ships to users.They create design documentation, design data structures and overallarchitecture and spend the vast majority of their time writing and reviewingcode. SWEs write a lot of test code including test driven design, unit testsand, as we explain in future posts, participate in the construction of small,medium and large tests. SWEs own quality for everything they touch whether theywrote it, fixed it or modified it.

The SET or Software Engineerin Test is also a developer role except their focus is on testability. Theyreview designs and look closely at code quality and risk. They refactor code tomake it more testable. SETs write unit testing frameworks and automation. Theyare a partner in the SWE code base but are more concerned with increasingquality and test coverage than adding new features or increasing performance.

The TE or Test Engineer isthe exact reverse of the SET. It is a a role that puts testing first anddevelopment second. Many Google TEs spend a good deal of their time writingcode in the form of automation scripts and code that drives usage scenarios andeven mimics a user. They also organize the testing work of SWEs and SETs,interpret test results and drive test execution, particular in the late stagesof a project as the push toward release intensifies. TEs are product experts,quality advisers and analyzers of risk.

From a quality standpoint,SWEs own features and the quality of those features in isolation. They areresponsible for fault tolerant designs, failure recovery, TDD, unit tests andin working with the SET to write tests that exercise the code for theirfeature.

SETs are developers thatprovide testing features. A framework that can isolate newly developed code bysimulating its dependencies with stubs, mocks and fakes and submit queues formanaging code check-ins. In other words, SETs write code that allows SWEs totest their features. Much of the actual testing is performed by the SWEs, SETsare there to ensure that features are testable and that the SWEs are actively involvedin writing test cases.

Clearly SETs primary focusis on the developer. Individual feature quality is the target and enablingdevelopers to easily test the code they write is the primary focus of the SET.This development focus leaves one large hole which I am sure is already evidentto the reader: what about the user?

User focused testing is thejob of the Google TE. Assuming that the SWEs and SETs performed module andfeature level testing adequately, the next task is to understand how well thiscollection of executable code and data works together to satisfy the needs ofthe user. TEs act as a double-check on the diligence of the developers. Anyobvious bugs are an indication that early cycle developer testing wasinadequate or sloppy. When such bugs are rare, TEs can turn to their primarytask of ensuring that the software runs common user scenarios, is performantand secure, is internationalized and so forth. TEs perform a lot of testing andtest coordination tasks among TEs, contract testers, crowd sourced testers, dogfooders, beta users, early adopters. They communicate among all parties therisks inherent in the basic design, feature complexity and failure avoidancemethods. Once TEs get engaged, there is no end to their mission.

Ok, now that the roles arebetter understood, I'll dig into more details on how we choreograph the workitems among them. Until next time...thanks for your interest.