Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Status
colourGreen
titleupdate 25 nov 2024

Once you have measured your application, you should verify the consistency of the results (stability, potential errors...).

...

Table of Contents
minLevel1
maxLevel6
outlinefalse
typelist
printablefalse

Step-by-step guide

In the Overall view of your application tab on Greenspector App:Studio, an indication is given of the level of reliability of the measured version.

...

In addition :

  • Check that at least 3 iterations have been run (full journey)

  • In the Performance section, check that the steps durations are greater than 0.5 sec.

Expand
titleTo go further in the Workshop...
  1. Verify the minimum of iterations for each of your test cases (at least 3 iterations).

  2. On the Test Results view, check that the functional tests are all passed. If not, you can have measurements but you should not be confident about them (it might be that you were not measuring what you thought)!

Image Modified
  1. On the Test Results view, check thatall your measures are present.

  2. On the Meter view, control the standard deviation of the measures. If the standard deviation is too high (in that case, the average of the measures is displayed in red or orange), it means that the measures of the iterations are not stable.

Image Modified
  1. To improve the standard deviation, you can add iterations and/or deactivate some iterations (in case you think some should be considered as errors). Measures outside confidence range are displayed in color (from yellow to red).

  2. Select the Test Duration metric and verify that you have durations higher than 0.5 second. If not, it means that you have too few measurement points. In that case, you might want to modify your automated tests in order to increase the test time.

Image Modified

...

Link between measure instability and backend instability

...