Have you ever watched a video and wondered how fast something in it was moving? A car accelerating on the motorway, a sprinter coming out of the blocks, a bird crossing the frame. With no calibrated camera, no object of known size and no reference measurement, getting a number out of that footage sounds like something from a crime drama.
It is harder than it looks, but it isn't impossible. Modern computer vision can get surprisingly reasonable speed estimates out of ordinary video, as long as you understand where the missing information has to come from.
The problem: speed without a ruler
Speed is distance divided by time. In a video, time is the easy part: at 30 frames per second, consecutive frames are 1/30 of a second apart. Distance is the hard part. What you can measure directly is how many pixels an object moved, and pixels aren't metres.
Worse, a single camera can't tell a small object moving slowly close to the lens from a large object moving fast far away. Both can produce exactly the same motion on screen. This is called scale ambiguity, and it is why "without prior information" always needs a footnote: somewhere, a piece of real-world scale has to enter the calculation. The techniques below differ mainly in where that scale comes from.
The classic shortcut is to use something of known size in the scene: a car (roughly 4.5 metres long), a lane marking, a door. When nothing like that is available, you need heavier tools.
Structure from Motion: from 2D to 3D
You may have seen 3D models built from drone footage. That is Structure from Motion (SfM), and it is useful for estimating speed as well.
SfM works a little like human depth perception: look at a scene from slightly different angles and you get a sense of depth. SfM analyses many frames and reconstructs both the 3D scene and the camera's path through it:
- Find distinctive visual features such as corners and textures in each frame, with detectors like SIFT or ORB.
- Match those features between frames and estimate how the camera moved, usually with the five-point algorithm inside RANSAC so that bad matches get thrown out.
- Triangulate the matched points into 3D and refine everything together with bundle adjustment, which minimises the gap between where each point should appear in each image and where it actually does.
- Measure how far the tracked object travelled in that reconstructed space.
The strength of SfM is that you understand the whole scene, not just the pixels. Its limitation is that the reconstruction is only correct up to an unknown scale factor. You get the right shape, but not whether you are looking at a model railway or a real city. One known distance, such as the height at which the camera is mounted or the width of a traffic lane, settles that.
Monocular depth estimation: AI's sense of depth
A newer option is a neural network that estimates depth from a single image. These models were trained on millions of images and picked up the same cues people use: perspective, texture gradients, objects partly hidden behind others, and the typical size of familiar things.
There are two kinds, and for speed the difference matters:
- Relative depth models such as MiDaS or the original Depth Anything tell you which surfaces are closer and which are farther, but not by how many metres. They are good for ordering a scene, not for measuring it.
- Metric depth models such as ZoeDepth, Metric3D or the metric versions of Depth Anything V2 try to output real distances directly, using what they learned about the world as their source of scale. Their estimates are approximate, and more reliable in scenes that resemble their training data, such as streets and rooms.
To estimate speed, run a metric depth model on every frame, track the object, turn its image position plus depth into a 3D position, and divide the distance it moved by the time that passed. Average over many frames, because individual depth estimates jitter.
A practical pipeline
- Detect and track. A YOLO-family detector combined with a tracker such as ByteTrack or DeepSORT keeps each object's identity from one frame to the next.
- Remove camera motion. If the camera moves, estimate the background motion with optical flow (Farnebäck, or a learned model such as RAFT) and subtract it, so you only measure the object's own movement.
- Bring in scale. Use whatever you have: a known object size, the camera's height above the road, vanishing-point geometry, or a metric depth model.
- Smooth the result. Speeds computed from single pairs of frames are noisy, so fit a smooth trajectory or average over a window.
Which approach when?
| Technique | Works best when | Typical failure |
|---|---|---|
| Structure from Motion | The camera moves and the scene has plenty of texture | Pure rotation, very little parallax, blank surfaces |
| Monocular depth | The camera is static, or the footage is blurry or low-resolution | Scale drifts from frame to frame; slow on modest hardware |
| Optical flow and tracking | You need speeds in real time | Motion blur, and the aperture problem on edges without texture |
In practice the best results come from combining them: SfM or known geometry for the overall scale, a depth model to fill the gaps where geometry fails, and tracking with optical flow for the motion from frame to frame.
Getting started
- For SfM: COLMAP is a mature open-source package.
- For depth: MiDaS and Depth Anything are easy to use from PyTorch; pick a metric variant if you need metres.
- For tracking: a YOLO detector with ByteTrack or DeepSORT.
Always test on a video of something whose speed you know, a car filmed together with its speedometer for example. It's the only way to find out how far off your pipeline is.
Where it's used
- Traffic monitoring without expensive radar equipment
- Sports analysis and performance tracking
- Wildlife research, where animals can be filmed without being disturbed
- Developing and testing autonomous vehicles
- Forensic analysis of video evidence
Wrapping up
Estimating speed from uncalibrated video is no longer science fiction, but it isn't magic either. Every method has to get its sense of real-world scale from somewhere: geometry, a familiar object, or a neural network's learned idea of how big things usually are. Once you know where your scale comes from, you also know how far to trust the number.
What would you use this for? Tell me in the comments.
