The missing data layer for physical AI
The foundational dataset for industrial embodied AI. We capture aligned intent, action and outcome trajectories from human operators in physics-grounded environments.
AI has learned from text and video.
The next bottleneck is learning from action.
The world-model bottleneck is not compute — it is grounded action data
Every warehouse robot deployed today was trained without knowing why a human made the decisions it is trying to imitate.
AI is moving from prediction to decision-making, but synthetic data alone misses the critical edge cases and the nuanced texture of real human decision-making.
Industrial AI fails when it doesn't understand how humans actually operate under safety and workflow constraints.
Real-world ground truth. We are harvesting human mastery.
Over the past decade we have built a proprietary data platform on real OEM hardware, collecting high-quality, multimodal behavioral data that captures the operational reality of logistics environments.
“Every action is captured with its intent and its consequence for step-level causality.”
Causality, action and outcome — on one deterministic clock
Most datasets show behavior. We make the reward function and intent explicit. Our data is instrumented at every layer and locked to a single, deterministic clock.
The “why”
Explicit task objectives, gaze rays, anticipatory head tracking and step-level curriculum context.
The “how”
High-fidelity 50 Hz stream of continuous control inputs, vehicle kinematics and environmental interactions.
The “what happened”
Recorded success, failure, collisions and safety violations, scored against OSHA standards.
Built for policy learning
The first publicly available action dataset for warehouse physical AI that captures operator intent at the task level. Distributed as JSONL and a pre-built trajectory.parquet, aligned to a deterministic 50 Hz physics grid.
ScaleOver 180,000 expert operator episodes.
Edge casesRegistered mistakes, safety violations and edge cases too dangerous to collect in the real world.
Observation vectorVehicle body state, mast/carriage pose, operator HMD pose and gaze rays.
Action vectorNormalized throttle, steer, lift, tilt and sideshift inputs.
Built-in reward signalsCollision penalties, step completion, task success and time efficiency.
from datasets import load_dataset ds = load_dataset( "fl-simulators/Human-Action-and-Intent" ) # one step on the 50 Hz grid step = ds["train"][0] step["observation"] # float[44] step["action"] # float[7] step["intent"] # task / step / gaze step["reward"] # collision, step, # success, time
Grounding AI in how humans actually operate
Frontier AI & world models
World models need causally grounded experience. Use our explicit task structure, intent labels and reward signals for goal-conditioned agents, RLHF reward modeling, and grounding world models in human behavior.
- goal-conditioned RL
- reward modeling
- world models
- imitation learning
Industrial robotics & automation
Overcome the sim-to-real gap. Reduce deployment risk by giving your autonomous mobile robots the behavioral priors and OSHA-compliant safety context they need to navigate complex, social warehouse environments.
- AMRs
- autonomous forklifts
- safety validation
- behavioral priors
Published, open, and built with the community
RL: A Structured Human Action & Intent Dataset for Physical AI and World Models
- Partner to structure, comment on and publish datasets
- Opening the environment as a gym for a community model contest
- Thought partner on use of data for general AI development
Unlocking the Power of Immersive Learning: The FAIRI Instructional Design Proposition
- 4+ year research partnership, new publications on indicators upcoming
- Ongoing research on rewards, RL agents and synthetic data modeling
- World-class xAPI contributors
If you want robots to understand the world, you need data that captures human intent.
Licensing, partnerships & custom data
Tell us what you're building. We'll get back to you within two business days.
- Commercial licensing of the full dataset
- Custom data collection on specific tasks or vehicles
- Research collaborations & benchmarks
- Access to the simulation environment as a gym