87. Configuring the ref_kit_config.yaml file

The ref_kit_config.yaml file is structured according to tasks. When the configuration file is edited with a supported IDE such as VSCode or PyCharm, users get syntax assistance and hints inline.

As an example, please see: flow/config/suburban_sanity.yaml. The full configuration example below helps you understand how to create your own config yaml file.

87.1 Task Ordering and Dependencies

The evaluation pipeline follows a specific task sequence with built-in dependencies:

Correct Task Order: parse → denoise → roi → ingest → match → post-match

Key Features

• Automatic Sorting: Tasks are automatically reordered according to the correct pipeline sequence, regardless of how they appear in your configuration.
• Gap Tolerance: You can skip tasks in the sequence (e.g., ["match"] or ["parse", "ingest"] is valid).
• Dependency Validation: Each task can be executed independently using its own configuration YAML file, as long as all required dependencies are specified as configuration options.
• Error Prevention: No need to manually ensure correct ordering - the system handles this automatically.

Examples

• Input: ["match", "roi", "ingest"] → Auto-sorted to: ["roi", "ingest", "match"] ✅
• Input: ["parse", "ingest"] → Remains: ["parse", "ingest"] ✅ (gap allowed)
• Input: ["ingest"] → Remains: ["ingest"] ✅ (single task)

Each task has a main processing script that wraps the FTX core tool, except for the parse task, which wraps a custom converter script.

87.1.1 Full configuration example

The following YAML file provides a full configuration example.

# yaml-language-server: $schema=./config_schema.json  ---> Do not change this 

fmanager_info:
  host: localhost
  port:
  security_protocol: http

dataset: suburban

tasks:
  - parse
  - denoise
  - roi
  - ingest
  - match

raw_input_dataset: /path/to/raw_input_dataset
ftlx_drive_logs: /path/to/ftlx_drive_logs

max_results: 100
max_failures: 3

map_file:
  path: $ESR_KIT_DRIVE_LOGS/object_lists/suburban_odd/M499_FTX_suburban/M499_FTX_suburban.xodr

parse_config:
  parse_script: $FTX_ESR_KIT_HOME/flow/0_convert/convert_generative_runs.sh
  parse_script_flags: null

denoise_config:
  denoise_config: $FTX_ESR_KIT_HOME/flow/1_preprocess/config/denoiser_config_full.yaml

roi_config:
  roi_config: $FTX_ESR_KIT_HOME/flow/1_preprocess/config/roi_config.yaml

ingest_config:
  ingest_top_file:
    path: $FTX_ESR_KIT_HOME/flow/2_ingestion/config/ingest_extend.osc
  ingest_script_flags: "--videos /path/to/videos"

match_config:
  eval_top_file:
    path: $FTX_ESR_KIT_HOME/evaluators/eval_scenarios/scenarios.osc
  ingestion_runs: $ESR_KIT_WORK_DIR/ingestion

post_match_config:
  post_match_type: "cpp"
  eval_lib_path: $FTX/logiq/src/osc/post_match_adapter.osc
  eval_lib_extend_path: $FTX_ESR_KIT_HOME/evaluators/eval_scenarios/scenarios_cpp_non_osc.osc
  eval_lib_flags: null
  matches: $ESR_KIT_WORK_DIR/matches

87.1.2 YAML file configuration parameters