Skip to content

54. Attributes

54.1 Default run attributes

Foretify Manager uses attributes for displaying information, creating filters and performing triage. The following are attributes that are defined by default. You can also create custom attributes for Triage.

Attribute ID Attribute display name Where used Description Type Unit
assignee Assignee Triage Lets you assign a test suite result to a user for triage. Select among users defined for the workspace. User
comparedRunId Compared run ID Triage Specifies the ID of the comparison test suite result. The "Compared Run ID" attribute cannot be reassigned. String
comparisonStatus Matching Status Triage Specifies whether the comparison test suite result matches the triaged test suite result. Valid values include New, Same, Different, and Not Compared. See Compare test suite results in a Triage view for definitions of these values. The "Matching Status" attribute cannot be reassigned. Enum
duration Duration Workspace Runs table, Workspace filters Specifies the wall clock duration of the entire run, including load, generation and simulation (End time - Start time) in seconds. This is equal to Load duration + Run duration. Float Seconds
endTime End time Workspace Runs table, Workspace filters Specifies the wall clock at the end of the run. Time (UTC)
firstRunLoadDuration First run load duration Workspace Runs table, Workspace filters Specifies the wall clock duration of the load phase of the test (equal to load_duration for crun's first seed, and copied over from that to all subsequent seeds). Float Seconds
foretifyRunCommand Foretify run command Workspace Runs table, Workspace filters Specifies the command for this run. String
foretifyRunUid Foretify ID Workspace Runs table, Workspace filters Specifies the Foretify-assigned ID for this run. String
hasRecordedVideo Video Available Workspace Runs table, Workspace filters Specifies whether a recorded video is available for the run. String
iD ID Workspace Runs table, Workspace filters Specifies the ID that Foretify Manager has assigned for this run. String
Job directory Dispatch Job Directory Workspace Runs table, Workspace filters Specifies the directory of the dispatch job. Only available when run through Dispatcher. String
Job ID Dispatcher Job ID Workspace Runs table, Workspace filters Specifies the Dispatcher-assigned ID for each job that it dispatches. Only available when run through Dispatcher. String
loadDuration Load Duration Workspace Runs table, Workspace filters Specifies the wall clock duration of the load phase of this particular execution in seconds. This is 0 for all subsequent runs in a consecutive run after the first one. Float Seconds
mainIssueCategory Main issue category Workspace Runs table, Workspace filters Specifies the category of the main issue reported during the run. String
mainIssueClusterString Cluster Workspace Runs table, Workspace filters Specifies the cluster of the main issue reported during the run. String
mainIssueDetails Main issue details Workspace Runs table, Workspace filters Specifies the details of the main issue reported during the run (details provided as part of the main issue). String
mainIssueFullMessage Main issue full message Workspace Runs table, Workspace filters Specifies the full message of the main issue reported during the run.
This is built by combining the fields such as results, kind, details, and modification_reason.
String
mainIssueKind Main issue kind Workspace Runs table, Workspace filters Specifies the kind of the main issue reported during the run. For more information, see detailed list of issue kinds. String
mainIssueNormalizedDetails Main issue normalized Workspace Runs table, Workspace filters Specifies the main issue reported during the run. This is the details' field normalized (during normalization process, we replace all numbers with "<>"). This means that if you have the same issue but with different numbers (for example, exceeding speed limit), the text will be identical after normalization. String
mainIssueReason Main issue reason Workspace Runs table, Workspace filters Modification reason (as provided by the user). It will be empty if the issue was not modified. String
mainIssueResult Main issue result Workspace Runs table, Workspace filters Specifies the result for the main issue reported during the run.
This is built by combining the fields such as category and severity.
String
mainIssueSeverity Main issue severity Workspace Runs table, Workspace filters Severity of the issue which is considered the main issue (with the highest severity), including the following:
  • None
  • Info
  • Warning
  • Error
  • Passed
  • Ignore

For more information, see the detailed list of issue severities.
Enum
mainOriginalIssueSeverity Main original issue severity Workspace Runs table, Workspace filters Captures all Foretify runtime error types, including error_stop_now and error_continue. Displayed in the run summary in the single run debugger. Also available in the run filter and column chooser. For more information, see the detailed list of issue severities. Enum
mainIssueTime Main issue time Workspace Runs table, Workspace filters Specifies the time, in ms, relative to the run start, when the main issue (if one exists) occurred. All passed runs have a main issue time of 0. A failed run may have a main issue time of 0 if it failed immediately as the run started. String Milliseconds
metricModelId Metric Model ID Workspace Runs table, Workspace filters Specifies the ID of the metric model. String
name Name Workspace Runs table, Workspace filters Specifies the name of the test. String
numericSeed Seed Workspace Runs table, Workspace filters Specifies the Foretify seed used to start generation in the first run. Random-seed runs are also useful to investigate further parameter combinations. Number
objectListPath Object list path Workspace Runs table, Workspace filters For a ‘Road data’ run, specifies the path from which the real world logs were processed. String
osUser OS User Workspace Runs table, Workspace filters Specifies the OS user. String
Regression ID Regression ID Workspace Runs table, Workspace filters Specifies the ID of the regression. String
runDir Dir Workspace Runs table, Workspace filters Specifies the directory where the test was executed. String
runDuration Run duration Workspace Runs table, Workspace filters Specifies the wall clock duration of the run phase of this execution. Float Seconds
runType Run Type Workspace Runs table, Workspace filters Specifies the type of the run. Can be one of:
  • Generative: A generative/simulated run.
  • Road data: The run is based on real world logs.
  • Smart Replay: The run was extracted from another Foretify run using the Smart Replay (export scenario) feature.
  • Unknown: No known source for the run.
Enum
seed Seed (Legacy) Workspace Runs table, Workspace filters Specifies the Foretify seed used to start generation in the first run. Random-seed runs are also useful to investigate further parameter combinations. String
serialNumber Serial Number Workspace Runs table, Workspace filters Specifies the serialized number of the run. Serialized numbers are unique per test suite and start from 0, 1, 2 … Long
simulationTime Simulation time Workspace Runs table, Workspace filters Specifies the duration of the run from the Ego/simulation perspective. This is the number you see in Visualizer. Float Seconds
startTime Start time Workspace Runs table, Workspace filters Specifies the wall clock time at the start of the run. Time (UTC)
status Status Workspace Runs table, Workspace filters Specifies the run status. Enum
tags Tags Workspace Runs table, Workspace filters Specifies the user-specified tags from the Filter menu. String
testFile Test File Workspace Runs table, Workspace filters Specifies the pathname of the test file for the run. String
testMap Test Map Workspace Runs table, Workspace filters Specifies the map used in this run. String
testName Test Name Workspace Runs table, Workspace filters Specifies the name of the top-level OSC2 test. String
testRunGroupId Test Suite Result ID Workspace Runs table, Workspace filters Specifies the ID that Foretify Manager assigned to the Test Suite Result of the run. String
topLevelScenarios Scenarios Workspace Runs table, Workspace filters Specifies the scenario(s) that were executed in the run. String
triage_status triage status Triage Specifies the triage status of the test suite result. Values include:
  • Open: Issue needs further assessment. Default status of all new issues.
  • Known issue: After assessment, the issue has been determined to be a known issue that does not need any further progress within the context of this triage.
  • Non-Issue: The issue has been determined to be a non-issue and can be ignored.
  • Confirmed: The issue has been confirmed to be a problem that needs to go through resolution steps.
  • Root-Cause Identified: The root cause for the issue has been identified but no fix has been implemented.
  • Fix Implemented: Fix has been implemented but still needs to be validated.
  • Fix Validated: The fix has been validated.
  • Will Not Fix: Issue will not be fixed.
Enum
triageVerdict Triage Verdict Triage Specifies the triage verdict of the test suite result. Values include:
  • Passed: There are no issues that require attention in the run.
  • Failed SUT: The SUT behaved differently than expected and the issue needs to be investigated.
  • Failed Others: Simulator / Foretify / other issues (server/connection issues, incomplete scenarios, etc.)
Enum
verdictAnalysisComment Verdict justification Workspace Runs table, Workspace filters Specifies the justification for a triage verdict. String

54.2 Test Name attribute

Foretify Manager’s Test Name attribute represents the name of the high-level test that runs an OSC2 scenario. Note that the display name is "Test Name" and the attribute name is test_name.

The Test Name is obtained as follows:

  1. If the test is generated by FRun, the Test Name is the name of the main FRun table from which the test was created.

    For example, for the following FRun table (in CSV format), the Test Name of the test generated from the table is vehicle_cut_in.

    // Cut_in scenario combinations,,,,,
    *vehicle_cut_in,template $FTX_PACKAGES/base_scenarios/test_suites/templates/vehicle_cut_in/vehicle_cut_in.tosc,,,,
    runs,seed,sim_config,map_config,vehicle_cut_in,user_config
    5,3,sim_config[1],map_config[4],vehicle_cut_in_sub[1],user_config
    5,8,sim_config[1],map_config[5],vehicle_cut_in_sub[2],user_config
    5,13,sim_config[1],map_config[7],vehicle_cut_in_sub[3],user_config
    
  2. You can also name a test file yourself by setting a value for the Foretify test_name attribute, for example:

    extend test_config:
    
        set test_name = “my_test”
    

    In this case, when the relevant test result is uploaded to Foretify Manager for the corresponding test run, the value of the Test Name attribute is my_test.

  3. If #1 and #2 are not applicable, the default Test Name value for a given run is based on the name of the test file that was loaded and run by Foretify to generate the test result, in the following format:

    If the full path of the test file is : /<path>/<to>/<test_file>/<test_file_name>.osc, the value of the Test Name attribute is <test_file_name>.