In this article
Short answer: first-person (egocentric) video is footage filmed from a camera worn on a person's head or chest, showing their hands and the objects they handle. It is the closest thing to a robot's point of view, so robotics and embodied-AI teams use it to learn manipulation and recognise activities. Public sets such as Ego4D and EPIC-KITCHENS are the usual starting point; production teams typically add their own collections for the tasks, homes and regions they care about.
Why first-person video matters for robots
Third-person video shows what happened; first-person video shows how it was done: where the hands go, which tool is picked up, what the person looks at before acting. That makes it valuable for:
- imitation learning and pre-training manipulation policies
- action recognition and step segmentation in long tasks
- video-language models that answer "what should I do next?"
- assistive technology that follows daily-living activities
Public datasets teams start with
| Dataset | What it is | Notes |
|---|---|---|
| Ego4D | About 3,670 hours of daily-life video from hundreds of camera wearers in several countries | Broad activities; access requires agreeing to its license |
| EPIC-KITCHENS-100 | About 100 hours of kitchen activity in 45 kitchens, densely annotated | Kitchens only; check terms for commercial use |
| Ego-Exo4D | Skilled activities filmed from first- and third-person views at once | Good for skill and pose research |
Read each license carefully. Several large egocentric datasets were released primarily for research, and terms can differ between the video and the annotations.
Where public data falls short
- Coverage: most footage comes from a limited set of countries, so kitchens, tools and habits elsewhere are under-represented.
- Task fit: your robot may need many repetitions of a handful of tasks, not one example of thousands.
- Annotation: labels may not match your action vocabulary.
- Licensing: research terms rarely cover a commercial robot.
How to specify your own collection
- Task list: the exact tasks, how many repetitions, and acceptable variations.
- Camera setup: head or chest mount, resolution, frame rate and field of view. Wide-angle at 30–60 fps is common.
- Settings: number of homes, kitchen types, lighting, clutter levels.
- Participants: adults, paid, with signed consent covering AI training.
- Annotation: step segments with start and end times and a description per step; optionally hand-object labels.
- Privacy: faces of other people, documents and screens blurred; audio removed unless consented.
- Delivery: clips, metadata (device, fps, duration, setting), a datasheet and the license.
Start with a pilot of a few hours across several homes, check that the footage and labels work in your pipeline, then scale.
Our first-person video
We film first-person household task video in Southeast Asian homes, with step-level descriptions, and street and traffic video for perception. See video training data for what every delivery includes.
first person video training dataegocentric video datasetvideo training datarobotics