Notes from 2018 Day 1: Process
with thanks to Nicholas Dunning
Resources available for learning how to do automated testing
Book: The art of unit testing
Mostly self-taught it seems
Automate at all levels, not just integration testing
Dive deep into the documentation
Book: Testing legacy systems
How to deal with fast iteration, especially in relation to black-box testing
functional testing might not be suitable while the feature is still in early development or you know it will change a lot in the near future
some tests don’t need to focus on exact values, so make a distinction between these types of tests
Make a unit test to focus on the exact change you made, ranter than the whole feature
Use pre-validation when creating objects to ensure they start in the most valid state possible
Communication is key, which may lead to a false-negative in the test
This relates a lot to hand-off
Check for valid outputs, not exact values
Reduce the dependancy of the test by putting the system into a state a near to the target state-change as possible
How to go from manual to automated testing, large scale
Reduce the duplicated tests to stop wasting time
Someone who is experienced with QA testing working with an experienced developer can help jump-start the system
Assume un-tested code is broken, so focus on that stuff first (Uber)
Local CI is a good place to start, as well as smoke tests
Build the feature tests from a customer-perspective
Get the dev team into the testing, not all handed off to the QA engineers
Show how the user is impacted if the bug is not fixed, to help prioritise them
How do maintain quality no matter who is testing?
TDD, do it. (Google)
Make sure you pin-point where things have changed before asking someone else to test it
System-map could help here
You need input from both development and testing specialists, as they have a different outlook on how the feature needs to be tested
Keep the testing framework as simple as possible so that everyone can write and run test
If people don’t trust the testing framework, they lose confidence in it, which actually leads to writing bad tests
Put a test plan in place (earlier the better)
This is a good point to start working out where and what to automate
How much should we automate, and where should we draw the line?
More valuable tests are integration and end-to-end (Google)
The more manual effort, the better of an idea it is to convert it to an automated test
If the overhead is low, there no such thing as too much testing (Google)
Unit tests should be the highest amount, with the amount decreasing until you reach end-to-end (Google)
If you don’t know what test to write, write the tests when the system or api is going to change (Apple)
Run the heavy stuff over-night to reduce the feedback time of builds
If the code you are changing doesn’t have a test, write one first