Versions Compared

Version Old Version 1 New Version Current
Changes made by

Maeva Rondeau

Maeva Rondeau

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Status
colourGreen
titleupdate 25 nov 2024

2 types of measure are possible on the Testbench :

Panel
bgColor#E3FCEF

Run a functional journey

  • Personalized analysis with an automated test execution

Panel
bgColor#DEEBFF

Run a benchmark

  • fastest way to get result with an automated test execution

  • Does not give a personalized analysis

  • Possible to run a benchmark on Greenspector App directly

Panel
bgColor#EAE6FF

Run a functional journey

Run a functional journey

Initialization of configuration

Expand
titleReminder : Initialize connection to Greenspector server

  • Windows/Linux/Mac:

Code Block
languagebash
gspt init --greenspectorApi "https://appcore-saas-prod.greenspector.com/api" --privateToken [MYTOKEN]
Info

Regardless of your OS, please replace [MYTOKEN] with a personal token.

Choose Preferences

It can be found in app.greenspector.com:

  • Click on the account name in the upper right

    • your account.

    Code Block
    gspt ping

    Link Greenspector CLI with the application on Greenspector App.

    Code Block
    gspt initprojectconfiguration --application [APP-NAME] --version [VERSION]
    Info

    Please replace [APP-NAME] and [VERSION] with the information used in the previous section.

    Set the environment

    To use the testbench command, you first need to set the environment for your project.
    To do so, you should use this command:

    Code Block
    gspt testbench set-environment

    Enter the number of your choice:

    Code Block
    Available environments: 
1 - mobile: Samsung - Galaxy S7 with android 8 
2 - mobile: pixelc with android 8.1.0 
3 - mobile: Samsung - Galaxy S9 with android 10 
4 - mobile: Samsung - Galaxy S9 - with sim with android 10 
5 - mobile: Samsung - Galaxy S9 - Maintenance with android 10 
6 - mobile: Samsung - Galaxy S22 with android 13 
7 - mobile: Samsung - Galaxy Tab S7 FE with android 13 
8 - mobile: Samsung - Galaxy S10 with android 12 
9 - mobile: Samsung - Galaxy S10 - with sim with android 12 
Please choose an environment:

    Launch of a web measure on Test bench

    Use the command line below to start a measurement.

    Code Block
    breakoutModefull-width
    gspt testbench custom-tests --testsSuite launch:./script.testgb --monitoredPackage "com.android.chrome" --iterations=3 -e config-skipsetupphone=false --networkMode=Wifi -e PAUSEDURATION=30000 -e PAUSEAFTERLOAD=1000

    Launch of an application measure on Test bench

    For web measurements, the monitoredPackage option needs to be modified with the name of the application package.

    You also need to add the app option as follows:

    • If the application is available on Play Store :

    Code Block
    --app [package name of app] # ex : --app com.google.android.youtube

    • If the application is present with a local apk :

    Code Block
    -app [path to the apk] # ex : --app ./my_apk.apk

    • If the application is present with a public apk :

    Code Block
    --app [path to the apk] # ex : --app https://my-website.com/my-apk.apk

    Use the command line below to start a measurement:

    Code Block
    breakoutModefull-width
    gspt testbench custom-tests --testsSuite launch:./script.testgb --monitoredPackage [appPackageName] --iterations=3 -e config-skipsetupphone=false --networkMode=Wifi -e PAUSEDURATION=30000 -e PAUSEAFTERLOAD=1000 --app [appPackageName]

    Expand
    titleCommand options

    Mandatory option*

    • --app: Precise the APKs you want to measure and the APKs which contain your functional tests to launch. Prefer separate APKs for your app and for your testsapkFile: APK(s) to measure. The apk file could be a local path, a public URL or a package name to download from the PlayStore. Prefer separate APKs for your app and for your tests.

    • --testsSuite* : Gdsl script name and lists of test files writen with Greenspector DSL. All test files will be launched in the specified order.

    • --monitoredPackage* : Name of the android package. Probes can monitor multi package at a time. In this case, metrics are agreggated. For website, choose the browser where the tests runs (i.e. com.android.chrome).

    • --iterations: Number of iterations of measure you wantwanted. (Default: 1).

    • --report: Automatically generate the test report in the .greenspector folder and wait for the job to end.(Default: false). This command will send a measure request to the server. After this measure will ran on the Power Test Bench/Cloud, you will find the results on the Web Interface.

    • --extra: Use to transmit extras parameters with the format name=value. (exemple --extra e PARAMETERNAME="value" will replace all ${PARAMETERNAME} in the GDSL file by value).

    • --networkMode: The network mode for the test. Values can be "WIFI", "4G", "3G" or "2G". (Default: WIFI).

    • --disable-dumpsys: Disable Android System Monitoring. By default, this is enabled and will add one iteration to your job for more detailed metrics (Views, Activity, OpenSSL Sockets, ...). If no need these metrics, disable this iteration, it will reduce testing time.

    • --only-dumpsys: Run only a Android System Monitoring iteration. This iteration will only be able to retrieve Android System metrics (Views, Activity, OpenSSL Sockets, ...).

    Note

    Please verify theses parameters to have relevant measurements:

    • PAUSEDURATION: 30000

    • PAUSEAFTERLOAD: 1000

    • config-skipsetupphone: false

    • iterations: 3

    Info

    Note that extras in job.yml which contains text, as PSWD for example, can’t contain a “!” because it’s interpreted by the bash. To know more about it go to the Troubleshooting tool.

    Results

    Results are available on app.greenspector.com.For a better understanding, screenshots of steps are available at :

    • The work folder with greenspector/dd-mm-yyyy XXhXXm/tests-results/

    MYAPP/parcours/iteration 1/screenshots. The final state is called success.png or failed.png regarding the result.
    Warning

    If the test doesn’t succeed, refer to Measure Troubleshooting tool

    Tip

    If the test succeeds, congratulations! (smile)

    Check the consistency of measures with Verification of measures consistency

    Run a benchmark

    Benchmark is the fastest way to get result with an automated test execution. However, it does not give a personalized analysis with a precise user journey.

    All the benchmark procedure takes place on app.greenspector.com/testbench/jobs/new the Workshop with the detailed procedure below or via the CLI after the previous commands.

    Expand
    titleHere the detailed procedure on Greenspector AppStudio

    • First, click on Benchmark in the side menu:

    • Choose + Add an application, enter name and validate. Description field is optional.

    Info

    Versions allow to compare different updates of the same application:

    • For an application, version number is often used (ex: 1.02.03)

    • For a website, the measurement date is often used (ex: 06_12_2023 with format dd_mm_yyyy)

    • Choose + Add a version, and give a version name:

    • Choose a test environment. The list shows the available devices on our testbench :

    • Specify the type of the item to measure: Website or Mobile app

    • For website, enter the URL for a website

    • For application, enter:

      • A link to the .apk file

      • A package name which will be downloaded on Google Playstore

    image-20240208-122547.png

    • Enter the amount of times the benchmark has to be executed:

      • The higher the number, the better the precision, but also the longer the execution time.

    • For a website, specify the browser to use

    Warning

    Firefox is not available anymore for the moment. Sorry for the inconvenience.

    Follow the measurement process

    The Tests tracking tab allows to verify if the test is taking into account by the testbench:

    • When Pending, the test is waiting for an available device in testbench

    • When Running, the test is currently running on a device

    • When Finished, the results are available

    Waiting time depends of the availability of testbench devices.

    At the end, click on the test to get more information, as logs, screenshots or dump. They stay available for 7 days.

    Expand
    titleHere the detailed procedure on the SaaS

    • Go in Analysis portfolio

    • Click on New Analysis

    • Select Benchmark

    • Enter needed information

    • Validate

    Results

    After the test is complete, the results are available in the dashboard tab, with a global Ecoscore from network and client resources grades.

    Note

    Verify that all rules have been evaluated : 24 rules should be verified.

    image-20240716-141308.png

    If it is not the case, re launch benchmark.

    Launch with the CLI

    It is possible to launch benchmark measures via the CLI. For that, the first configuration steps presented at the beginning of this page must be followed until the terminal selection.

    Expand
    titleBenchmark a website via the CLI

    This feature allows you to run a measure of a website on the TestBench.
    To use this feature, you have to first configure the application name, version and the environment to run the measure on.
    The command to run is:

    Code Block
    gspt testbench measure-url --url [URL]

    Options

    Mandatory option*

    • --url* : URL to benchmark

    • --iterations: Number of iterations of measure (Default: 1).

    • --browser: Browser to run the measure on. (Default: Google Chrome).

    • --networkMode: Network mode for the test. Values can be "WIFI", "4G", "3G" or "2G". (Default: WIFI).

    • --report: Automatically generate the test report in the .greenspector folder and wait for the job to end. (Default: false).

    This command will send a measure request to the server. After this measure has run on the Power Test Bench/Cloud, results are available on the web interface.

    Examples

    Code Block
    gspt testbench measure-url --url https://greenspector.com/ 
gspt testbench measure-url --url https://greenspector.com/ --iterations 5 --browser chrome --networkMode 4G 
gspt testbench measure-url --url https://www.mozilla.org/fr/ --browser firefox --report
    Expand
    titleBenchmark one or more APK(s) via the CLI

    This feature allows to benchmark an Android application on the TestBench.
    To use this feature, you have to first configure the application name, version and the environment to run the measure on.
    The command to run is:

    Code Block
    gspt testbench benchmark-apk --app [APK-PATH]

    Options

    Mandatory option*

    • --app* : APK(s) to benchmark. (If set multiple times, all the APK will be installed but only the first one will be launched and monitored). Can be a local path, a public URL or a package name to download from the PlayStore. Optional if the applications to benchmark are already installed on phone

    • --iterations: Number of iterations of measure you want. (Default: 1).

    • --report: Automatically generate the test report in the .greenspector folder and wait for the job to end. (Default: false).

    • --networkMode: The network mode for the test. Values can be "WIFI", "4G", "3G" or "2G". (Default: WIFI).

    This command will send a measure request to the server. After this measure has run on the Test Bench, you will find the results on the Web Interface.

    Example

    Code Block
    gspt testbench benchmark-apk --app target/final.apk # local path to apkgspt testbench benchmark-apk --app https://demo/demo.apk # apk from public urlgspt testbench benchmark-apk --app com.microsoft.office.outlook # package name to download the apk from the PlayStoregspt testbench benchmark-apk --app target/main.apk --app target/app-module.apk --app target/commons.apk --networkMode 3G # three apk to install and launch benchmark in 3Ggspt testbench benchmark-apk --app target/final.apk --iterations 5 --report # Launch 5 iterations of the benchmark and wait for the report