Think together · Start with a story
A girl records a video of a ball rolling off a table. If the robot sees only the last frame, it may not know how the ball fell.
Break down the process
- Sequence of framesPut them in time order
- TimingNotice what changes
- Audio and other cluesAdd them when supported
- Task outputDescribe an event
Understanding video means processing how scenes change over time, and may also involve audio.
Some systems select a sample of frames; others use special representations of timing. Whether they capture tiny or fast movements depends on sampling and the model.
If an important moment is not seen, the model may miss the cause. It should not turn a guess based on common sense into a definite fact.
Your turn to try
If the model sees only a ball on the floor, can it be sure who pushed it off?
Ready? Open to see the explanation
No. It is missing information from when the action happened, so it needs to inspect the relevant part of the video.
One idea to remember
Understanding video means processing how scenes change over time, and may also involve audio.
Try covering the picture above and explaining each step to your family. If one step is hard to explain, revisit that card.