The missing ground truth for physical AI
The evaluation and data infrastructure for industrial physical AI. We capture aligned intent, action and outcome from expert human operators in physics-grounded environments, and use it to evaluate and improve models.
AI has learned from text and video.
The next bottleneck is learning from action.
The world-model bottleneck is not compute — it is grounded action data
Every warehouse robot deployed today was trained without knowing why a human made the decisions it is trying to imitate.
AI is moving from prediction to decision-making, but synthetic data alone misses the critical edge cases and the nuanced texture of real human decision-making.
Industrial AI fails when it doesn't understand how humans actually operate under safety and workflow constraints.
One industrial loop for Physical AI
From human ground truth to model evaluation, improvement and deployment. Humans define ground truth, models are evaluated against it, failures generate targeted improvement, and every deployment produces new evidence.
We don't care how your model was trained: imitation learning, RL or agentic robotics. We measure whether it works.
01Human-in-the-loop simulationReal material-handling tasks with OEM controls and physical interfaces.
02Deployed and validated in industry500+ systems, ~300 customers, 17 countries: real operating context.
03 · where the loops meetIntent, action and outcome captured togetherStructured ground truth: telemetry, task hierarchy, xAPI intent, safety events.
- Edge cases
- Task stitching
- Customization
- Human intervention
- Transferable skills
- Multi-agent workflows
Live in use today · In development being built now · Example illustrative values
Real-world ground truth. We are harvesting human mastery.
Over the past decade we have built a proprietary data platform on real OEM hardware, collecting high-quality, multimodal behavioral data that captures the operational reality of logistics environments.
Our simulators train operators at
“Every action is captured with its intent and its consequence for step-level causality.”
Causality, action and outcome — on one deterministic clock
Most datasets show behavior. We make the reward function and intent explicit. Our data is instrumented at every layer and locked to a single, deterministic clock.
The “why”
Explicit task objectives, gaze rays, anticipatory head tracking and step-level curriculum context.
The “how”
High-fidelity 50 Hz stream of continuous control inputs, vehicle kinematics and environmental interactions.
The “what happened”
Recorded success, failure, collisions and safety violations, scored against OSHA standards.
Built for policy learning
The first publicly available action dataset for warehouse physical AI that captures operator intent at the task level. Distributed as JSONL and a pre-built trajectory.parquet, aligned to a deterministic 50 Hz physics grid.
Scale278,000+ expert operator exercises across 71,000+ training sessions.
Rule breaksRecorded mistakes, safety violations and edge cases too dangerous to collect in the real world.
Observation vectorVehicle body state, mast/carriage pose, operator HMD pose and gaze rays.
Action vectorNormalized throttle, steer, lift, tilt and sideshift inputs.
Built-in reward signalsCollision penalties, step completion, task success and time efficiency.
from datasets import load_dataset ds = load_dataset( "fl-simulators/Human-Action-and-Intent" ) # one step on the 50 Hz grid step = ds["train"][0] step["observation"] # float[44] step["action"] # float[7] step["intent"] # task / step / gaze step["reward"] # collision, step, # success, time
Evaluate your model against expert human operators
The FL Warehouse Benchmark is a physical-world industrial environment being built for OpenEnv and Isaac Lab-Arena. It will score your policy against expert human baselines across three difficulty levels.
- Task completionDid the policy finish the assigned material-handling task?
- Task stitchingCan it chain sub-tasks (pick, travel, place) into one workflow?
- SafetyCollisions, near-misses and rule breaks, scored against OSHA criteria.
- EfficiencyTime and path compared with expert human operators.
- SmoothnessControl stability: abrupt steering, lifting and braking.
Expert human baselines from 278,000+ exercises on real OEM controls.
Works withAny training approach: imitation learning, RL or agentic robotics.
Join the early-access list →Grounding AI in how humans actually operate
Frontier AI & world models
World models need causally grounded experience. Use our explicit task structure, intent labels and reward signals for goal-conditioned agents, RLHF reward modeling, and grounding world models in human behavior.
- goal-conditioned RL
- reward modeling
- world models
- imitation learning
Industrial robotics & automation
Overcome the sim-to-real gap. Reduce deployment risk by giving your autonomous mobile robots the behavioral priors and OSHA-compliant safety context they need to navigate complex, social warehouse environments.
- AMRs
- autonomous forklifts
- safety validation
- behavioral priors
Evaluation & certification
Validate deployment readiness before real-world risk. Score autonomous material-handling systems against expert human baselines and OSHA safety standards, and generate the safety evidence that EU Machinery Regulation compliance and enterprise deployment approvals require.
- deployment readiness
- human baselines
- OSHA criteria
- EU Machinery Regulation
Published, open, and built with the community
RL: A Structured Human Action & Intent Dataset for Physical AI and World Models
- Partner to structure, comment on and publish datasets
- Opening the environment as a gym for a community model contest
- Thought partner on use of data for general AI development
Unlocking the Power of Immersive Learning: The FAIRI Instructional Design Proposition
- 4+ year research partnership, new publications on indicators upcoming
- Ongoing research on rewards, RL agents and synthetic data modeling
- World-class xAPI contributors
If you want robots to understand the world, you need data that captures human intent.
Licensing, partnerships & custom data
Tell us what you're building. We'll get back to you within two business days.
- Commercial licensing of the full dataset
- Custom data collection on specific tasks or vehicles
- Research collaborations & benchmarks
- Early access to the FL Warehouse Benchmark
- Access to the simulation environment as a gym