Defining legacy checkers

You can define checkers to capture problematic behavior or conditions specific to your SUT.

Important

This section explains how to use legacy checkers; however, using watcher-based checkers provides a more robust mechanism for adding checkers to scenarios. See Defining watchers and checkers to learn how.

OSC2 supports the following checker-related tasks:

• Defining a response to a checker failure with sut_error(), sut_issue() and so on.
• Defining Boolean checkers with the on modifier.
• Defining collect checkers with the collect() modifier.

You can also define checkers in C++.

After you create your checkers and run your test, you analyze detected issues. See Understanding and resolving issues to learn how to handle issues. You can also analyze issues in Foretify Manager using Triage. See Triaging test suite results for details.

Define failure response

To define a response to a checker failure, you must use the external methods and scenarios shown below. You can use the zero-time scenario call to execute the external method, or you can invoke the equivalent scenario directly with similar parameters.

Notes

• With the on modifier, you must call the external method.
• These methods/scenarios require that you specify the kind of issue. Thus, you must use one of the predefined vales of the enumerated type issue_kind or extend the type to include a custom value.

Builtin methods/scenarios for issue definition

External method	Equivalent scenario	issue severity	issue category
top.sut_issue(severity, kind, details)	NA	<severity>	sut
top.sut_error(kind, details)	top.sut_error()	error	sut
top.sut_warning(kind, details)	top.sut_warning()	warning	sut
top.scenario_completion_ error(kind, details)	top.scenario_completion_error()	error	scenario_completion
top.scenario_completion_ warning(kind, details)	top.scenario_completion_warning()	warning	scenario_completion
top.other_issue(severity, kind, details)	NA	<severity>	other
top.other_error(kind, details)	top.other_error()	error	other
top.other_warning(kind, details)	top.other_warning()	warning	other

Create Boolean checkers using on

The simplest checkers define a failure response if the SUT fails to meet a Boolean condition. For example, if the SUT should never go faster than 115 kph, you can write a checker like this:

OSC2 code: define Boolean checker

on (sut.car.state.speed > 115kph):
    call sut_error(sut_too_fast,
        "SUT speed too high: $(sut.car.state.speed)")

If you want to check the SUT's speed only upon specific events, you can write the checker like this:

OSC2 code: trigger checker on event

on @end if (sut.car.state.speed > 70kph):
    call sut_error(assertion,
        "SUT speed too high at end: $(sut.car.state.speed)")

You can use the sut.car.state or sut.car.physical structs or some of the predefined methods such as get_ttc_to_object() to access the data you need for the checker. Other actors provide similar access to their state and physical properties. See Vehicle actor for more information.

Example: SUT does not brake for pedestrian

This example creates a check to determine whether a vehicle slows down if there is a pedestrian on the path.

extend issue_kind: [ no_brake_for_pedestrian]

scenario sut.brake_for_pedestrian:
  person: person

  !car_should_brake := sample(sut.car.state.collision_info == null ? false:
    ((sut.car.state.collision_info.other_car.plain_object == person)
    and (sut.car.state.collision_info.TTC < 1.5second)), @top.clk)

  !sut_to_pedestrian_distance := sample((person.state.global_position != null ?
    map.distance_between_global_positions(
    sut.car.state.msp_pos.global_position,
    person.state.global_position) :
    10km), # A large value when person is not created yet
    @top.clk)

  on (car_should_brake and sut_to_pedestrian_distance in [0..2]m and
    sut.car.state.acceleration >= 0mpsps and sut.car.state.speed > 10kph):

    call sut_error(no_brake_for_pedestrian,
      "SUT should brake. It is on a collision course with $(person).")

• Line 1 extends the issue_kind enumerated type to include an issue called no_brake_for_pedestrian.
• Lines 6 - 8 check whether the car should brake. car_should_brake is set to false if there is no collision information. Otherwise, it is set to true if a person is in front of the vehicle and time to collision (TTC) is less than 1.5s.
• Lines 10 - 15 check whether a person was created. If created, sut_to_pedestrian_distance is set to the relative distance between the SUT and the person. If not created, sut_to_pedestrian_distance is set to 10km.

• Lines 17 - 21 activate sut_error() if all the following conditions are true:

  • car_should_brake is true.
  • sut_to_pedestrian_distance is less than 2m and more than 0m.
  • acceleration is non-negative (a negative value means braking is occurring).
  • SUT speed is above 10kph.