S12-503: A to Z Testing in Production: TiP Methodologies, Techniques, and Examples

Track: Performance Testing

Testing in production (TiP) is a set of software testing methodologies that utilizes real users and production environments in a way that both leverages the diversity of production, while mitigating risks to end users. By leveraging the diversity of production we are able to exercise code paths and use cases that we were unable to achieve in our test lab, or did not anticipate in our test planning.

This session introduces participants to TiP and gives testers methodologies they can use to TiP like Controlled Test Flights, Synthetic Test in Production, Load/Capacity Test in Production, Data Mining, Destructive Testing and more. These are illustrated with examples from Microsoft, Netflix, Amazon and others. Along the way we will cover techniques that enable TiP like Exposure Control, Test Data Handling, Production Test Hooks, and Crowd Sourcing.

Test labs try to approximate the production environment – TiP is the production environment.
Test plans try to anticipate user behavior and workflows – TiP is user behavior.

Download Presentation
Please Note: The presentations are intended for attendees only. The presentations page is password protected – contact info@softwaretestpro.com for verification of attendance and the password to access the presentation.

Session Speaker:

Seth EliotSeth Eliot – Senior Knowledge Engineer – Test, Microsoft Corp.
Seth Eliot is Senior Knowledge Engineer for Microsoft Test Excellence focusing on driving best practices for services and cloud development and testing across the company. He previously was Senior Test Manager, most recently for the team solving exabyte storage and data processing challenges for Bing, and before that enabling developers to innovate by testing new ideas quickly with users “in production” with the Microsoft Experimentation Platform (http://exp-platform.com). Testing in production (TiP), software processes, cloud computing, and other topics are ruminated upon at Seth’s blog at http://bit.ly/seth_qa and on Twitter (@setheliot). Prior to Microsoft, Seth applied his experience at delivering high quality software services at Amazon.com where he led the Digital QA team to release Amazon MP3 download, Amazon Instant Video Streaming, and Kindle Services.