Measure on the Testbench

update 25 nov 2024

2 types of measure are possible on the Testbench :

Run a functional journey

Personalized analysis with an automated test execution

Run a benchmark

fastest way to get result with an automated test execution
Does not give a personalized analysis
Possible to run a benchmark on Greenspector Studio directly
Go to the section lower

Run a functional journey

Initialization of configuration

Windows/Linux/Mac:

gspt init --greenspectorApi "https://core-saas-prod.greenspector.com/api" --privateToken [MYTOKEN]

Regardless of your OS, please replace [MYTOKEN] with a personal token.

It can be found in your account.

gspt ping

Link Greenspector CLI with the application on Greenspector App.

gspt initprojectconfiguration --application [APP-NAME] --version [VERSION]

Set the environment

To use the testbench command, you first need to set the environment for your project.
To do so, you should use this command:

Enter the number of your choice:

Launch of a web measure on Test bench

Use the command line below to start a measurement.

Launch of an application measure on Test bench

For web measurements, the monitoredPackage option needs to be modified with the name of the application package.

You also need to add the app option as follows:

If the application is available on Play Store :

If the application is present with a local apk :

If the application is present with a public apk :

Use the command line below to start a measurement:

Mandatory option*

--apkFile: APK(s) to measure. The apk file could be a local path, a public URL or a package name to download from the PlayStore. Prefer separate APKs for your app and for your tests.
--testsSuite* : Gdsl script name and lists of test files writen with Greenspector DSL. All test files will be launched in the specified order.
--monitoredPackage* : Name of the android package. Probes can monitor multi package at a time. In this case, metrics are agreggated. For website, choose com.android.chrome.
--iterations: Number of iterations of measure wanted. (Default: 1).
--report: Automatically generate the test report in the .greenspector folder and wait for the job to end.(Default: false). This command will send a measure request to the server. After this measure will ran on the Power Test Bench/Cloud, you will find the results on the Web Interface.
--extra: Use to transmit extras parameters with the format name=value. (exemple --e PARAMETERNAME="value" will replace all ${PARAMETERNAME} in the GDSL file by value).
--networkMode: The network mode for the test. Values can be "WIFI", "4G", "3G" or "2G". (Default: WIFI).
--disable-dumpsys: Disable Android System Monitoring. By default, this is enabled and will add one iteration to your job for more detailed metrics (Views, Activity, OpenSSL Sockets, ...). If no need these metrics, disable this iteration, it will reduce testing time.
--only-dumpsys: Run only a Android System Monitoring iteration. This iteration will only be able to retrieve Android System metrics (Views, Activity, OpenSSL Sockets, ...).

Results

Results are available on :

The work folder with greenspector/dd-mm-yyyy XXhXXm/tests-results/results.txt
On Greenspector Studio if online is true in job.yml.

Run a benchmark

Benchmark is the fastest way to get result with an automated test execution. However, it does not give a personalized analysis with a precise user journey.

All the benchmark procedure takes place on the Workshop with the detailed procedure below or via the CLI after the previous commands.

First, click on Benchmark in the side menu:

Choose + Add an application, enter name and validate. Description field is optional.

Choose + Add a version, and give a version name:

Choose a test environment. The list shows the available devices on our testbench :

Specify the type of the item to measure: Website or Mobile app
For website, enter the URL for a website
For application, enter:
- A link to the .apk file
- A package name which will be downloaded on Google Playstore

Enter the amount of times the benchmark has to be executed:
- The higher the number, the better the precision, but also the longer the execution time.
For a website, specify the browser to use

Follow the measurement process

The Tests tracking tab allows to verify if the test is taking into account by the testbench:

When Pending, the test is waiting for an available device in testbench
When Running, the test is currently running on a device
When Finished, the results are available

Waiting time depends of the availability of testbench devices.

At the end, click on the test to get more information, as logs, screenshots or dump. They stay available for 7 days.

Go in Analysis portfolio
Click on New Analysis
Select Benchmark
Enter needed information
Validate

Results

After the test is complete, the results are available in the dashboard tab, with a global Ecoscore from network and client resources grades.

Launch with the CLI

It is possible to launch benchmark measures via the CLI. For that, the first configuration steps presented at the beginning of this page must be followed until the terminal selection.

This feature allows you to run a measure of a website on the TestBench.
To use this feature, you have to first configure the application name, version and the environment to run the measure on.
The command to run is:

Options

Mandatory option*

--url* : URL to benchmark
--iterations: Number of iterations of measure (Default: 1).
--browser: Browser to run the measure on. (Default: Google Chrome).
--networkMode: Network mode for the test. Values can be "WIFI", "4G", "3G" or "2G". (Default: WIFI).
--report: Automatically generate the test report in the .greenspector folder and wait for the job to end. (Default: false).

This command will send a measure request to the server. After this measure has run on the Power Test Bench/Cloud, results are available on the web interface.

Examples