Sim-to-Real in 2026: Closing the Gap Without Endless Real-World Data

Every robot learning team eventually hits the same wall: data. Language models can be trained on a large share of the public internet, but nobody has an internet-sized archive of robots grasping, walking, falling and recovering. Collecting that experience on physical hardware is slow, expensive and sometimes dangerous. Simulation offers an obvious way out: train in a virtual world where data is cheap, then deploy on the real machine. The catch has a name, the reality gap, and closing it is what sim-to-real robotics is about.

This guide explains where the gap comes from, the techniques that have narrowed it (domain randomization, fast GPU simulation, digital twins) and how synthetic data for robots, increasingly produced by generative world models, fits into modern robot learning and autonomy.

What is the reality gap?

In their 2017 paper on domain randomization, researchers from OpenAI and UC Berkeley described the problem clearly. Physics simulators differ from reality in ways that are hard to remove. System identification, meaning tuning a simulator’s parameters to match a physical robot, is “time-consuming and error-prone”. Even a well-tuned simulator misses effects such as non-rigid materials, gear backlash, wear-and-tear and fluid dynamics. Rendered camera images also lack the richness and noise of real sensors [1]. Together these differences make up the reality gap, and they explain why a policy that looks perfect in simulation can fail on a real robot.

The same paper sets out why simulation is still worth the trouble. Deep reinforcement learning (RL) relies on random exploration, which can be dangerous on hardware. It also often needs hundreds of thousands or millions of samples, which could take thousands of hours to collect on a physical robot [1].

Technique 1: Domain randomization, making reality “just another variation”

The first widely adopted answer was counter-intuitive. Instead of making the simulator perfectly realistic, make it wildly varied. If a model sees enough random variation during training, the authors argued, “the real world may appear to the model as just another variation” [1].

Their experiment trained an object detector only on simulated images rendered with deliberately unrealistic random textures. The textures were flat colours, gradients and checker patterns, shown under randomised lighting, camera poses and distractor objects and rendered with MuJoCo’s basic built-in renderer [1]. The detector located objects in the real world to within about 1.5 cm on average, and a Fetch robot using it picked up the target object in 38 out of 40 trials in cluttered scenes [1]. The authors described it as the first successful transfer to real-world robot control of a deep network trained only on simulated RGB images, with no pre-training on real images [1].

The ablations are still useful today. Performance dropped significantly with fewer than about 1,000 unique textures. Leaving distractor objects out of training raised the average error in cluttered scenes from about 1.8 cm to 7.2 cm [1]. Variety itself was the training signal.

Automatic domain randomization

Hand-picking randomization ranges is hard: too much randomization makes learning difficult, while too little hinders transfer [2]. OpenAI addressed this in its 2019 Rubik’s Cube project with Automatic Domain Randomization (ADR). Training starts in a single, non-randomised environment. Each time the policy reaches a performance threshold, the randomization ranges widen automatically, covering parameters such as cube size and mass, finger friction and the hand’s visual appearance [2].

The neural networks were trained entirely in simulation and then controlled a real human-like robot hand to solve the cube [2]. OpenAI was open about the limits: the system succeeded 60% of the time on scrambles needing 15 rotations, and 20% on maximally difficult scrambles needing 26 [2]. On a simpler block-flipping task, ADR eventually doubled real-robot transfer performance over manually tuned randomization, without human tuning [2]. The policy also coped with perturbations it never saw in training [2].

Technique 2: Speed, with GPU simulation and fast iteration

Domain randomization multiplies the amount of experience a policy needs. That makes simulator speed a central engineering concern rather than a convenience.

MuJoCo Playground, released in 2025 by researchers from UC Berkeley, Google DeepMind and others, shows how far this has come. It is a fully open-source framework built on MJX, a JAX-based version of the MuJoCo physics engine that runs on GPUs. Researchers can install it with a single pip install playground and train many policies in minutes on one GPU [3]. The authors describe the standard sim-to-real recipe in four steps: build a simulated environment that matches the real world (the “digital twin”), encode the desired behaviour as a reward, train in simulation, and deploy [3]. They argue that speed matters because reward design is iterative. RL is good at maximising reward but often produces odd behaviour, so the time from asking for a behaviour to seeing it on the robot becomes the bottleneck [3].

Some of their reported results:

• Locomotion: joystick-controlled walking policies for the Berkeley Humanoid trained on flat ground in under 15 minutes. Policies for the Unitree G1 and Booster T1 humanoids took under 30 minutes on two RTX 4090 GPUs and transferred zero-shot to stable walking and turning [3].
• Manipulation from pixels: with visual properties randomised (lighting, shadows, camera pose, object colours), a Franka arm learned to pick up a small block from 64×64 RGB images after ten minutes of training on a single RTX 4090. It then succeeded in all 12 real-world trials [3].
• Contact-rich control: a torque-controlled policy for pushing and reorienting a yoga block, trained with randomised delays and a curriculum, reached a mean real-world success rate of about 86% over 35 hardware trials [3].
• Iteration pace: the team deployed state- and vision-based policies on six robot platforms in under eight weeks [3].

The paper also gives an honest view of the remaining gap. Its LEAP-hand cube reorientation policy managed a median of only 3.5 consecutive rotations in real trials, with failures such as the cube getting wedged between fingers and palm [3]. Dexterous, contact-rich manipulation is still where sim-to-real is weakest.

NVIDIA’s competing stack runs on its PhysX GPU engine through Isaac Lab and Isaac Sim. The MuJoCo Playground authors acknowledge that this ecosystem has produced a wide range of robotic sim-to-real results, while noting that the PhysX GPU implementation is closed-source [3].

Technique 3: Layered sim-to-real for humanoids

By 2026, sim-to-real is less a single trick than an architecture. NVIDIA’s January 2026 technical walkthrough of its Isaac GR00T N1.6 humanoid model shows the pattern [4]:

• Low level: a whole-body controller trained with RL in Isaac Lab produces dynamically stable motion primitives for locomotion, manipulation and multi-contact behaviour. These policies are stress-tested at scale in Isaac Lab and Isaac Sim, then transferred zero-shot to physical humanoids [4].
• Mid level: a navigation policy trained on large synthetic datasets generated with NVIDIA’s COMPASS workflow sends velocity commands to that controller. NVIDIA reports zero-shot sim-to-real transfer, including deployment in new physical environments, without extra task-specific data collection [4].
• High level: the GR00T N1.6 vision-language-action model reasons about the task and sequences skills. Its training mix combines simulated environments (BEHAVIOR, RoboCasa and a custom GR-1 environment) with real demonstrations from several robot platforms [4].

The design choice matters. The hardest physics, balance and contact, is handled by a policy trained heavily in simulation, where falls cost nothing. The large model can then focus on what to do rather than how to stay upright [4]. Before deployment, the full stack is validated in simulation [4].

Technique 4: Synthetic long-tail data from world models

Classic simulators are physics engines with renderers attached. In 2026 a second kind of simulator is maturing: generative world models that produce physically plausible video and sensor data.

At CES in January 2026, NVIDIA released Cosmos Transfer 2.5 and Cosmos Predict 2.5, which it describes as open, fully customisable world models for physically based synthetic data generation and for evaluating robot policies in simulation. It also released Cosmos Reason 2, an open reasoning vision-language model, with all of them available on Hugging Face [5]. The attraction is the long tail: rare lighting conditions, unusual objects and edge-case scenes that are expensive or unsafe to stage physically but that a deployed robot will eventually meet.

The same announcement shows how the tooling around simulation is becoming industrial infrastructure [5]:

• Evaluation: Isaac Lab-Arena, an open-source framework for large-scale robot policy evaluation and benchmarking in simulation, connects to benchmarks such as Libero and RoboCasa.
• Orchestration: NVIDIA OSMO coordinates synthetic data generation, training and software-in-the-loop testing across workstations and cloud.
• Healthcare: LEM Surgical is using Isaac for Healthcare and Cosmos Transfer to train the autonomous arms of its Dynamis surgical robot.
• Partners: AGIBOT introduced Genie Sim 3.0, a simulation platform integrated with Isaac Sim, and Franka Robotics, NEURA Robotics and Humanoid are using GR00T-enabled workflows to simulate, train and validate new behaviours.

Why the edge matters: from simulation to on-device autonomy

Sim-to-real does not end when training stops. A policy trained in the cloud still has to run in real time on the robot, often with limited power. The MuJoCo Playground locomotion policies, for example, run at 50 Hz through ONNX Runtime inside a real-time C++ control stack [3]. The Franka torque policy issues commands every 5 ms [3].

Hardware vendors are building for this. NVIDIA’s Jetson T4000 module, announced in January 2026, brings its Blackwell architecture to autonomous machines. NVIDIA quotes 1,200 FP4 TFLOPS and 64 GB of memory within a configurable 70-watt envelope, at $1,999 per unit at 1,000-unit volume [5]. For industrial robotics the takeaway is that the train-in-simulation, deploy-at-the-edge pipeline now has purpose-built pieces at both ends.

What still doesn’t transfer

Simulation has not made real-world data unnecessary. The sources above point to recurring weak spots:

• Contact-rich dexterity. OpenAI found domain randomization alone was “not enough” for the physics of a Rubik’s Cube in a robot hand [2], and in-hand reorientation remains fragile in more recent open frameworks [3].
• Unmodelled physics. Backlash, wear, deformable materials and fluids remain hard to simulate faithfully [1].
• Sensor and latency mismatch. Successful transfers often depend on deliberately modelling delays and noise. MuJoCo Playground randomises action and observation delays and gripper lag, and its authors report that roughly calibrating the action scale on the real setup “drastically” improved transfer [3].
• Reward design. Training is now fast, but specifying what the robot should do is still a human, iterative process [3].

The bottom line

The question has changed. It used to be “can a robot trained in simulation work in the real world?”, and by 2026 the answer is clearly yes for locomotion and a growing range of manipulation tasks. The question now is how to divide the work between simulation, synthetic data and real experience. The working answer has several parts:

• Train the physics-heavy low-level skills in heavily randomised GPU simulation.
• Fill the long tail with synthetic data from world models.
• Validate whole stacks in digital twins before they reach hardware.
• Keep a smaller but still essential stream of real-world data for the contact, sensing and edge cases that simulators still miss.

For anyone building robot learning and autonomy, sim-to-real has become core infrastructure.

Related reading

• Vision-Language-Action Models: The New Foundation Layer for Robots
• Physical AI Data Factories: How Synthetic Data Accelerates Industrial Robots
• AI Cobots and Vision: Where Factories See ROI First

Image credits: Both illustrations (sim2real-hero.jpg, sim2real-domain-randomization.jpg) are AI-generated images, compliant with BP §5.2. Text is original; figures are paraphrased and attributed per §5.1.

Images: original illustrations generated for AndBeyond.Tech; no logos or real people depicted.