AI #9 - Crash Test
Using AI models to predict car crashes
Vehicle dash camera maker Nexar is running a competition (Moura, Zhu, and Zvitia 2025). The challenge? Can contestants make an artificial intelligence that predicts car crashes before they happen?
After completing a 7-month deep dive course in AI & Machine Learning, I decided to try this contest. While it’s nowhere near the higher paying contests ($2.1+ million!), this one does have a $5,000 first place prize.
The Setup
After logging in and agreeing to competition terms, competitors have access to 1,500 real driving videos. The videos come from forward facing, dash-mounted cameras in cars and trucks on the road. Half of the 40-second videos show real accidents and the other half show normal driving. The trick is to train an AI algorithm to “learn” what elements of the video are predictors of a crash.
How Machines “Learn”
Computers don’t see videos like we humans do. Instead, the best a computer can do is to track the movement of all the tiny pixels that make up videos. In fact, those videos you watch on Netflix or any computer screen are actually a set of many still images shown at high speed, typically 30 frames per second, to simulate movement.
So our 40-second crash video is actually a series of 1,200 frames played fast enough that you see smooth motion instead of individual frames.
For our contest, the task involves tracking and measuring movements of all those pixels across hundreds or thousands of frames and building an association of those movements with “crash” or “not a crash.”
Features are Important
I’ll take a little creative license here for illustration. Imaging you are teaching someone to recognize different car models. All cars have wheels, headlights, tires, windshields and more. Those features are common across all cars, but vary wildly by brand and model of car. You could say, “Oh see that wheel, that’s definitely a Ford.” Or, “That tail fin is a key mark of a BMW.”
Essentially an AI model creates a feature set describing what it sees across all those video pixels. Now this get a little weird: the features that the model detects are abstract representations, not easily namable things we humans know like “headlights” or “bumpers.” The AI is detecting patterns which may or may not mean anything coherent to us if we inspected what it sees. To represent what it “sees” the AI creates long lists of numbers (aka vectors) that represent discreet features in each video. So, imagine this list going on for 2,048 elements:
[0.346
0.788
0.821
0.249
…
0.677]
That list of 2,048 numbers would represent a single video to the AI.
Now that we have a feature set (numbers) for each video, the AI begins to draw correlations. By doing some mathematical operations, it observes those features present in a crash versus those present in a normal driving. During this phase (known as training), the videos contain labels that clearly show whether a video has an accident or not. It’s sort of like flashcards for your friend who is learning car models: “Ford,” “Chevy,” “Toyota.” In this case, “crash” and “no crash.”
Survey Says
Once an AI has been trained we call it a “model” and use it for making predictions. In the contest, we actually do two types of predictions. In the first type, called validation, we know the right answer (crash or no-crash) and so at this phase we use data to reinforce the training, making it better and better over a series of iterations. So we show the AI a set of videos and ask it to make predictions. When it is right, great! When it is wrong, the AI uses that to incrementally adjust to make itself more and more accurate. Over and over and over, it trains.
In the second type of prediction, called “testing,” we show the AI entirely new data with the “crash” or “no-crash” labels hidden. In our case, we show it pre-crash videos and ask it to predict: crash or no-crash.
And that’s exactly how our contest works. We do lots of tweaking, training and testing and then finally we present our best estimates to the competition judge. The judges (in this case a computer) know the answers but we don’t so it’s a good test of how accurate our models are at predicting crashes.
The top scorer in the contest currently has an accuracy rate of over 90%. I was hovering around 20th from the top last time I checked. I’ve learned a ton about AI but I’m not in the upper echelons (yet!).
Saving Lives
Well that’s the big question, isn’t it? Will this technology reduce crashes and fatalities. My guess is that it will. Automotive safety has been on a long, steady improvement trend since the passenger car was introduced, as this data attests.
Sources
Moura, Daniel C., Shizhan Zhu, and Orly Zvitia. Nexar Dashcam Collision Prediction Dataset and Challenge. 2025. https://arxiv.org/abs/2503.03848.