Reliability

When it comes to automation, the reliability of tests is the key. If your tests are not reliable, they will not be trusted, which can have far-reaching consequences. I'm sure you have all worked in environments where test reliability has been hard for one of many reasons; let's have a look at a couple of scenarios.

The test automation team that works in isolation

One of the more common reasons that tests are not reliable is having a dedicated test automation team that works in isolation from the team that develops the application. This should really be avoided if possible, as the test automation team is always playing catch up. The development team rolls out new features that the test automation teams have to automate, but they are never sure what is coming next. They usually find out that existing features have changed when their tests break. As well as fixing their tests, they need to work out what the new functionality is and whether this new functionality is behaving as expected...