Some days ago I found a very interesting series of posts in the google testing blog, from James Whittaker.

The links, plus some excerpts and comments:

In Part I he begins by showing Google’s organizational structure and how testing can fit in there.

In Part II, he describes the main roles for software developers and testers at Google:

“The SWE or Software Engineer is the traditional developer role. (…) SWEs own quality for everything they touch whether they wrote it, fixed it or modified it”

“The SET or Software Engineer in Test is also a developer role except their focus is on testability (…) they are more concerned with increasing quality and test coverage than adding new features or increasing performance.”. James published an entire post only about the life of a SET in part VI, but I also found this about SETs, which is just amazing:

“Who are these rare creatures who keep our code bases healthy and make our development process run so smoothly?”

Concerning the two above roles:

“Much of the actual testing is performed by the SWEs, SETs are there to ensure that features are testable and that the SWEs are actively involved in writing test cases. (…) Clearly SETs primary focus is on the developer.”

“The TE or Test Engineers spend a good deal of their time writing code in the form of automation scripts and code that drives usage scenarios and even mimics a user. (…) TEs are product experts, quality advisers and analyzers of risk.”

About TEs:

“User focused testing is the job of the Google TE (…) When bugs are rare, TEs can turn to their primary task of ensuring that the software runs common user scenarios, is performant and secure, is internationalized and so forth.”. An in-depth explanation of this role is given in Part VII. From there:

“Ultimately, test engineers are paid to protect users and the business from bad design, confusing UX, functional bugs, security and privacy issues and so forth. At Google, TEs are the only people on a team whose full-time job is to look at the product or service holistically for weak points. As such, the life of a Test Engineer is much less prescriptive and formalized than that of an SET. ”

In Part III, James talks about quality:

“While it is true that quality cannot be tested in, it is equally evident that without testing it is impossible to develop anything of quality. How does one decide if what you built is high quality without testing it? (…) Quality is not equal to test; it is achieved by putting development and testing into a blender and mixing them until one is indistinguishable from the other. ”

and more about the process:

“Who better to do all that testing than the people doing the actual coding? (…) The reason Google can get by with so few dedicated testers is because developers own quality (…) Having too large a test team is a very strong sign that the code/test mix is out of balance. Adding more testers is not going to solve anything. (… At google…) Manifestations of this blending of development and testing are all over the place from code review notes asking ‘where are your tests?’ to posters in the bathrooms reminding developers about best testing practices, our infamous Testing On The Toilet guides.”

and finally:

“This means that quality is more an act of prevention than it is detection. Quality is a development issue, not a testing issue.”

In Part IV, James talks a little bit about their release process, which follows what he calls the “crawl, walk, run approach”. Very similar to the Lean Startup process:

“One of the key ways Google achieves good results with fewer testers than many companies is that we rarely attempt to ship a large set of features at once. In fact, the exact opposite is often the goal: build the core of a product and release it the moment it is useful to as large a crowd as feasible, then get their feedback and iterate”

In Part V, the kinds of tests are explained:

“Instead of distinguishing between code, integration and system testing, Google uses the language of small, medium and large tests emphasizing scope over form. Small tests cover small amounts of code and so on.”

“For small tests the focus is on typical functional issues such as data corruption, error conditions and off by one errors. The question a small test attempts to answer is does this code do what it is supposed to do?”

‘Medium Tests can be automated or manual and involve two or more features and specifically cover the interaction between those features. The question a medium test attempts to answer is does a set of near neighbor functions interoperate with each other the way they are supposed to?”

“Large Tests cover three or more (usually more) features and represent real user scenarios to the extent possible. (…) The question a large test attempts to answer is does the product operate the way a user would expect?”

about automation:

“the mix between automated and manual testing definitely favors the former for all three sizes of tests. If it can be automated and the problem doesn’t require human cleverness and intuition, then it should be automated. (…) The ongoing effort to automate to within the “last inch of the human mind” is currently the design spec for the next generation of test engineering tools Google is building. ”

but

“it is important to note that Google performs a great deal of manual testing, both scripted and exploratory, but even this testing is done under the watchful eye of automation.”

These are the seven posts James has published so far. There’s also a short Q&A post, in which he talks more about the roles and their career paths.