Autonomous Driving Systems: The Future of Transportation

Autonomous vehicles are no longer a research curiosity. They are shipping software, running on public roads, and generating real liability. Here is what the engineering actually looks like, where the hard problems still sit, and what a technical team needs to think through before building or integrating any part of this stack.

How Does an Autonomous Driving System Actually Work?

At its core, an ADS (Autonomous Driving System) is a perception-planning-control loop running in real time. The loop has to close in under 100 milliseconds for highway driving. In dense urban environments, some teams target sub-50ms end-to-end latency.

Perception

Perception fuses data from multiple sensor modalities. The dominant combination in production systems today is:

• LiDAR (typically 64-channel or 128-channel spinning units, or newer solid-state units like Innoviz One)
• Cameras (usually 8–12 cameras covering 360°, running at 30fps or higher)
• Radar (short and long range, essential for adverse weather where LiDAR degrades)
• IMU + GNSS (for ego-motion and localisation)

Sensor fusion is done either at the raw data level (early fusion) or at the object detection level (late fusion). Early fusion gives you richer signal but is computationally heavier. Most production teams use a hybrid. Tesla's camera-only approach (no LiDAR) is an outlier, and it remains genuinely controversial in the AV engineering community. The argument for it is cost and redundancy simplicity. The argument against is that cameras lose depth information that LiDAR gives you directly.

Planning

Planning splits into three layers: route planning (where am I going), behavioural planning (should I change lanes, yield, turn), and motion planning (what exact trajectory do I follow). The motion planner has to produce a path that is kinematically feasible, collision-free, and comfortable for occupants. Most teams use a combination of lattice-based planners and optimisation-based methods (e.g., model predictive control). Learned planners are being integrated but are not yet dominant in safety-critical production systems.

Control

The control layer executes the planned trajectory by commanding throttle, braking, and steering. This is where hardware-in-the-loop (HIL) testing matters. A poorly tuned controller that overshoots at 80 km/h is a safety issue, not a UX issue.

What Are the Real Engineering Challenges in 2024?

The hard problems are not perception of clear objects in good weather. That is mostly solved. The hard problems are:

Long-tail scenarios. A child chasing a ball between parked cars, a construction zone with no lane markings, a hand signal from a police officer. These scenarios are rare in training data and catastrophic when handled wrong. Waymo estimates it would need to drive billions of real-world miles to cover the tail naturally. Simulation is the only practical answer, but simulation-to-real transfer (the sim2real gap) is an unsolved problem.

HD map dependency. Most ADS systems rely on centimetre-accurate HD maps for localisation. Maintaining those maps at scale is expensive. HERE Technologies and TomTom charge per km for coverage. If the map is stale and the road has changed, the localisation prior is wrong.

Compute budget. The NVIDIA DRIVE Orin SoC delivers 254 TOPS (tera operations per second) and is the current standard in production-intent platforms. Running a full perception + planning stack in real time still consumes a large fraction of that budget. Power envelope matters too: an always-on ADS system in a consumer vehicle has to live within the thermal limits of a car, not a data centre.

Functional safety certification. ISO 26262 (road vehicles) and ISO 21448 (SOTIF, safety of the intended functionality) are the relevant standards. Achieving ASIL-D (the highest automotive safety integrity level) for the full ADS stack is not straightforward. Most teams decompose the system so that each component can be certified independently.

How Should You Think About Building vs. Integrating?

If you are a team evaluating whether to build ADS capability in-house or integrate an existing stack, the honest answer is: almost no team should build the full stack from scratch.

The full stack includes sensor drivers, a perception pipeline, an HD map system, a simulation environment, a data labelling pipeline, a planning stack, a control layer, and a safety monitoring layer. Each of those is a multi-year engineering investment.

Component	Build in-house	Integrate (examples)
Sensor drivers	Usually yes	Vendor SDKs (Ouster, Velodyne)
Perception	Selective (domain-specific)	NVIDIA DriveWorks, Mobileye
HD maps	Almost never	HERE, TomTom, Mapbox
Simulation	Sometimes	CARLA, LGSVL, Applied Intuition
Planning	Often, for domain fit	Autoware (open source)
Safety monitor	Always	Custom, ISO 26262 compliant

The decision usually comes down to your specific operational design domain (ODD). If you are building a last-mile delivery robot for a controlled campus, your ODD is narrow enough that a custom planning layer on top of an existing perception stack is reasonable. If you are building a Level 4 highway system, you are competing with Waymo and Cruise. That is a different conversation.

The Regulatory Situation Is Fragmented and Moving Fast

In the US, NHTSA's AV TEST Initiative replaced the earlier AV Guidance documents. California, Arizona, and Texas have different permitting regimes for driverless testing. In Europe, the UNECE WP.29 framework (specifically UN Regulation 157, covering ALKS systems up to 130 km/h) is the operative standard, but member states still have discretion in implementation. China has the most permissive testing environment by surface area.

What this means for engineering teams: your data logging, event reconstruction, and OTA update processes are now regulatory artefacts, not just engineering choices. GDPR and similar frameworks also apply to the sensor data you collect from public spaces.

Simulation Is the Real Differentiator

Teams that win in AV engineering are not the ones with the most road miles. They are the ones with the best simulation pipelines.

A well-run simulation environment lets you run millions of scenario variations overnight. CARLA (open source, based on Unreal Engine 4) is the most widely used research platform. Applied Intuition and AVxcelerate (from ANSYS) are the enterprise options. Scenario description languages matter here: OpenSCENARIO 2.0 and OpenDRIVE are the standards for portability.

The gap between a team that does ad-hoc simulation and one that has a proper scenario database, a CI/CD pipeline that gates on simulation regression, and a coverage metric for ODD scenarios is measured in years of development time.

Conclusion

ADS engineering is genuinely hard, and most of the difficulty sits in the long tail, not the core loop. If you are scoping a project in this space, start with a precise ODD definition. That single decision shapes every downstream engineering and regulatory choice. If your ODD is narrow and well-defined, a custom stack built on open-source components (Autoware, CARLA, ROS 2) is achievable. If it is broad, you need a partner with existing infrastructure. Either way, get your safety case written early, because retrofitting ISO 26262 compliance into an existing system is far more expensive than designing for it from the start.

If you want to talk through your specific ODD and what a realistic build scope looks like, reach out to the team at Sodio.

---

FAQ

What SAE level should I target for my ADS project? SAE Level 2 (partial automation, driver monitors) is achievable with off-the-shelf ADAS components. Level 4 (no driver needed within a defined ODD) requires a full perception-planning-control stack, extensive simulation, and a safety case. Most commercial deployments today sit at Level 2+ or Level 4 in a very restricted ODD.

Is camera-only perception viable without LiDAR? For highway ADAS, yes, with significant investment in camera resolution, redundancy, and a well-trained depth estimation model. For Level 4 in urban environments, most safety engineers still require LiDAR as a separate sensing modality. The cost of LiDAR has dropped from over $75,000 per unit to under $500 for solid-state units, which changes the trade-off considerably.

How much data do you need to train a perception model? A minimum viable urban object detection model typically needs several hundred thousand annotated frames across a range of lighting, weather, and geographic conditions. Manual annotation costs between $0.05 and $0.50 per object depending on complexity. Semi-automated labelling pipelines (using model-assisted labelling) can cut that cost by 60–80%.

What is the difference between ISO 26262 and ISO 21448? ISO 26262 covers failures caused by hardware or software faults (e.g., a sensor short-circuits). ISO 21448 (SOTIF) covers failures caused by the system working as designed but encountering a scenario outside its intended operating conditions (e.g., perception failing on an unusual road marking). Both apply to a production ADS.

Can I use ROS 2 in a production ADS? ROS 2 (especially with the Autoware stack) is used in production-intent systems, primarily in robotics and low-speed AV deployments. For high-speed automotive applications, you need deterministic scheduling guarantees that vanilla ROS 2 does not provide out of the box. Teams typically use a real-time OS (e.g., QNX, ROS 2 with a PREEMPT_RT kernel patch) for the control layer and ROS 2 for perception and planning.

Have a project in mind? Contact Sodio Technologies to discuss your requirements and explore the right technology solution for your business.