Benchmark iterations
In the previous section, we prepared all we needed to start a benchmark in an efficient way. Now we need to see how we will work during the benchmark.
The first important thing is to establish that we only deal with optimization iterations here and not evaluation ones, which are straightforward—you run the scenarios and gather the reports.
Iterating
It is probably natural, but you will iterate with your optimizations. This means that you will run the same test again and again to measure the result of a change—for instance, increasing the pool size.
The direct implication of such a work structure is that you need to prepare yourself to store lots of reports in an organized way. There are many solutions for that and it mainly depends on the tools you are used to relying on. But at a very high level, you need to, at least, store:
- The benchmark report.
- The benchmark date (to be able to sort them, it is often useful to replay the iterations done afterwards).
- The benchmark configuration...
