Skip to content

32. Launching a test suite

32.1 Launching a test suite from a project

  1. From your project, click the Test Management tab.

  2. Optionally, search for an existing test suite by entering a string in the Search field.

    The test plans list is filtered to match the string.

  3. Optionally, select Show only mine to see only your test plans.

  4. Select a test plan and click Launch Test Suite on the top right.

  5. Optionally, select the required labels.

    See more details on how to create labels.

  6. Verify Definition details.

    See Test plan fields for descriptions of the fields.

  7. Click Launch.

    A message indicating a successful or failed launch is displayed.

  8. Optionally, navigate to the Results > Test Suite Results tab to monitor the test suite progress.

32.2 Command-line scripts for launching

Foretify Manager offers two utility scripts for launching a test-suite from the command-line:

Both scripts allow passing environment variables into the container running Foretify in order to control various run-time configurations.

32.3 Launch a test suite with FRun

If your environment is not set up to run with Foretify Manager Dispatcher, you can launch a test suite with FRun, the front-end of Dispatcher. You can also use FRun if you want to examine the test files it generates before executing them in order to make sure that they will compile and execute correctly.

FRun creates tests based on a test template and on the test parameters you specify in the run table. FRun is useful for many reasons:

  • It allows simplified test creation without learning a GUI.

  • It performs filtering of impossible, illegal and uninteresting scenarios and demonstrates how a constraints-driven approach is better than variations.

  • It lets you ask for cross value combinations to fill desired scenario spaces.

  • It allows refining error messages into warnings.

FRun has several use modes:

Create mode: Create the test files and run directories, without executing the tests.

Batch mode: Execute many runs (a test suite) for later analysis with Foretify Manager.

Interactive mode: Execute a specific test from a table using the --line command option.

To see a complete list of FRun invocation options: Invoke FRun with the --help option.

To invoke a test suite for the Interceptor scenario: Invoke FRun with the batch and csv options:

Shell command: invoke FRun in batch
$ frun --batch \
--csv $FTX_PACKAGES/base_scenarios/test_suites/test_suite_definitions/junction_family/sut_approach_junction_main.csv \
--csv $FTX_PACKAGES/base_scenarios/test_suites/test_suite_definitions/junction_family/sut_approach_junction.csv \
--csv $FTX/packages/common/csv/carla_config.csv \
--csv $FTX/packages/common/csv/sumo_config.csv

For more information on using FRun, see the Using FRun.