Definition
A robotics data explorer is a software tool that provides structured access to the datasets used for robot learning. It allows researchers, engineers, and data operators to browse dataset catalogs, inspect individual recording sequences, view synchronized sensor streams, review annotations, and download specific data files — all through a web-based interface.
In the context of human-in-the-loop robotics data collection, an explorer serves as the inspection and delivery layer. After human operators collect demonstrations and the data passes through annotation and quality assurance, the explorer makes the resulting datasets accessible for review and use. Humaid's data explorer is an example of this tool integrated into a production data collection platform.
What a Robotics Data Explorer Shows
Video Streams
Synchronized playback of egocentric (first-person) and third-view video captured during data collection. Frame-level controls allow stepping through individual moments to verify alignment between video and annotations.
Sensor Data
IMU readings, depth maps, force-torque profiles, and other sensor streams recorded alongside video. These are typically stored in formats like MCAP or HDF5 and displayed as time-aligned overlays or downloadable files.
Pose and Tracking Data
3D body pose, hand joint positions, and object 6-DoF poses rendered as overlays on video or explorable in 3D viewers. These data types are critical for robot training pipelines that use imitation learning or behavior cloning.
Annotations
Temporal action segmentation labels, object detection bounding boxes, grasp type classifications, and task completion flags. These annotations are what make raw sensor recordings usable for supervised learning and policy training.
Metadata
Recording context (environment, scenario, duration), video specifications (FPS, resolution, codec), annotation statistics (action count, object count, tracking coverage), device information, and storage details. Good metadata is essential for filtering and selecting training subsets.
Downloadable Files
Per-sequence access to individual data streams: raw video, processed annotations, pose data in standard formats (JSON, NPZ), and raw sensor recordings (MCAP). This eliminates the need for bulk data transfers when only specific sequences are needed.
How It Works
A typical robotics data explorer follows a three-level hierarchy:
Catalog level. The top level shows all available datasets grouped by domain or task type. Each dataset has a name, description, and summary statistics. Users select a dataset to drill down.
Dataset level. Inside a dataset, users see individual recording sequences — each representing one demonstration episode. Sequences can be filtered by metadata properties, searched by name, and paginated for large collections.
Sequence level. The detail view for a single sequence shows synchronized video playback, a metadata sidebar with all recorded properties, annotation overlays, and a download panel for accessing individual files. Advanced explorers also provide 3D visualization for spatial data.
Use Cases
Quality Assurance
QA teams use the explorer to verify annotation accuracy, check for sensor synchronization issues, confirm action label boundaries, and flag episodes that do not meet quality standards. This is faster and more reliable than reviewing raw files.
Training Data Validation
ML engineers inspect individual sequences before including them in training batches. The explorer helps identify edge cases, verify that data distributions match expectations, and catch labeling errors that could degrade model performance.
Debugging Model Failures
When a trained model fails on specific tasks, engineers trace the failure back to training data. The explorer makes it possible to find and inspect the exact sequences that correspond to failure scenarios.
Data Delivery
Data collection providers use explorers to deliver datasets to clients. Instead of transferring opaque file archives, clients can browse, preview, and selectively download exactly the data they need.
Collection Protocol Iteration
During the design phase of a data collection campaign, teams review early batches in the explorer to refine operator instructions, adjust sensor configurations, and update annotation guidelines before scaling up.
Cross-Team Communication
Shareable URLs for specific datasets and sequences allow robotics teams to reference exact data points in discussions, bug reports, and documentation — eliminating ambiguity about which recording is being discussed.
Example Workflow
A robotics team building a policy for kitchen manipulation tasks would use an explorer like this:
- Open the explorer and navigate to the household dataset catalog
- Select the "Kitchen" dataset to see all recorded cooking and preparation sequences
- Filter sequences by action type or duration to find relevant demonstrations
- Open a specific sequence to watch synchronized egocentric and hand-pose video
- Review the 60+ metadata properties to confirm sensor configuration and annotation coverage
- Check temporal action segmentation labels against the video to verify annotation accuracy
- Download the hand pose JSON, object detection annotations, and raw MCAP file for that sequence
- Repeat for additional sequences until the training batch meets quality and distribution requirements
This entire process happens in the browser. No custom scripts, no file parsing, no ambiguity about what the data contains.
Try Humaid's Data Explorer
Humaid's data explorer is a production implementation of the concepts described above. Browse open robotics datasets — household tasks and manufacturing assembly — with synchronized video, full metadata, and downloadable files.