98. Sensor Sim Plan Simulation environments
When you create a Sensor Sim Plan, you configure environment-specific settings that control how sensor data is generated. Your choice of simulation environment determines which sensors, Ego vehicles, and maps are available throughout the pipeline. Sensors and the Ego created in one environment cannot be used in another.
The SDG pipeline supports three simulation environments, each with its own sensor models, capabilities, and configuration options.
98.1 NVIDIA Cosmos
Cosmos uses generative AI to produce realistic sensor data. Rather than physically simulating sensors, it generates video output from prompts and control signals, making it well-suited for creating diverse visual scenarios.
98.1.1 NVIDIA Cosmos environment settings
| Setting | Type | Description |
|---|---|---|
| Prompt | String | Text description that guides the visual generation |
| Transfer version | String | Cosmos model version to use for generation |
| Run start time | Float | Start offset in seconds (negative values supported) |
| End time | Float | End offset in seconds |
| Guidance | Float | Controls generation quality (default: 3.0) |
| Control weight | Float | How strongly the generation follows the control signal (default: 1.0) |
| Seed | Integer | Random seed for reproducible output |
| Num steps | Integer | Number of generation steps (default: 35) |
| Chunk overlap | Integer | Overlap for autoregressive chunking (default: 1) |
| Separate views | Boolean | When enabled, saves one file per camera instead of a combined view |
98.1.2 NVIDIA Cosmos supported sensor types
- FTheta Camera
98.2 NVIDIA NuRec (Neural Reconstruction)
NuRec uses neural rendering to reconstruct sensor data from real-world recordings. It produces photorealistic output by combining neural reconstruction with optional post-processing.
98.2.1 NVIDIA NuRec environment settings
| Setting | Type | Description |
|---|---|---|
| Fixer checkpoint | String | Path to the neural fixer checkpoint file. Leave empty to disable the fixer. |
98.2.2 NVIDIA NuRec supported sensor types
- FTheta Camera
- Lidar
98.3 NVIDIA Omniverse SensorRTX
NVIDIA Omniverse SensorRTX is a physics-based sensor simulation environment powered by NVIDIA Omniverse. Unlike Cosmos (generative AI) and NuRec (neural reconstruction), SensorRTX uses ray tracing to physically simulate sensor data.
98.3.1 NVIDIA Omniverse SensorRTX environment settings
| Setting | Type | Description |
|---|---|---|
| Environment | String | Sky and weather preset for environment lighting |
| Time of day | Time | Time of day for lighting simulation |
| FPS | Integer | Frames per second |
| Duration | Float | Duration of the simulation in seconds |
98.3.2 NVIDIA Omniverse SensorRTX supported sensor types
Supported sensor types:
- Pinhole Camera
- Fisheye Camera
- ISP Camera
- Rotary Lidar
- Solid State Lidar
- Generic Radar
98.4 Choosing a simulation environment
Use this environment comparison to choose the best fit for your scenario.
| Environment | Approach | Sensor variety | Best for | Configuration complexity |
|---|---|---|---|---|
| SensorRTX | Physics-based ray tracing | Camera, lidar, and radar | Physically accurate multi-sensor simulation | Low |
| Cosmos | Generative AI | Camera only | Diverse visual scenarios from text prompts | High |
| NuRec | Neural reconstruction | Camera and lidar | Photorealistic output from real-world data | Low |