F12 – 604: Non-Regression Test Automation (Part 2)

Track: Test Automation

Automated software testing has historically meant having the computer run individually crafted test cases that perform the same exercises as manual tests, only run by a machine. The principle advantages for these automated tests are repeatability and speed. The principle disadvantages are that they are relatively more expensive to create than manual tests, require more maintenance than manual tests, and are more limited in the specificity of things they can compare compared with manual tests.

Part 1 of this presentation describes another way to approach test automation: to test things that cannot be tested manually. Extending the scope of testing in this way allows checking for errors that might not be found otherwise or even conceived of. These tests enable us to focus on learning about the software, can go behind the UI to extend our reach, are not limited to doing the same thing each time (although even random sequences can be repeated), and can perform huge numbers of iterations and combinations unthinkable using manual testing or automated regression tests. These tests are usually high-volume, quick-hit, or abstracted one or two levels from the user interface, which substantially reduces maintenance costs. This approach also encourages checking broader classes of test outcomes, thus improving the types of errors that can be discovered.

Part 2 describes oracle mechanisms that enable testers to take advantage of non-regression automation. The oracles determine whether the software’s behavior appears to be normal or erroneous. The oracles allow non-regression tests to vary their behavior and still have predictable, checkable outcomes. This session presents over a dozen different types of oracle mechanisms.

Session Takeaways:

  • How to use automated tests to extend testers’ capabilities
  • The virtues and drawbacks to automated regression tests
  • Design of automated tests for exploration
  • A model for understanding outside influences and hidden outcomes during testing
  • Test oracle mechanisms for automated tests

Download Presentation
Please Note: The presentations are intended for attendees only. The presentations page is password protected – contact info@softwaretestpro.com for verification of attendance and the password to access the presentation.


Session Speaker:

Doug HoffmanDoug Hoffman – Consultant, Software Quality Methods
Doug Hoffman is a management consultant for Software Quality Methods and a trainer in strategies and tactics for quality assurance with over 30 years experience. His technical focus is on test automation and test oracles. His management focus is on evaluating, recommending, and leading of quality improvement programs. He is the President of the Association for Software Testing (AST); a Fellow of the ASQ; and holds MBA, MSEE, and BACS degrees. He is certified by ASQ as a Software Quality Engineer and as a Manager of Quality/Organizational Excellence. Doug is past Chair of the Silicon Valley Section of ASQ, a founding member, past Chair, and current Treasurer of SSQA (Silicon Valley Software Quality Association), a founding member of AST, and a Senior member of ACM and IEEE. He has spoken at dozens of conferences and chaired several international conferences on software quality.