Why the AI steward is built to say \"I don't know\"

The most dangerous thing an AI steward can do is not make a wrong call. It's make a wrong call that sounds sure. A confident, tidy paragraph with a rule number gets believed, even when the evidence behind it is thin.

So a large part of how Sim Racing Steward works is about stopping that. Most of the accuracy comes from the system declining to answer things it can't know. Here are the main guardrails.

The reporter's description is a claim, not evidence

When someone reports an incident, they usually write a description: "he dive-bombed me at turn 3 and punted me off." That's useful context, but it's also one driver's opinion.

The model that watches the footage is told to treat the description as an unverified claim. Every driver, car colour, position and contact it writes down has to be something it can actually see in the video. If the description says something the footage doesn't show, it goes into a separate "Unresolved / Unverifiable" list labelled as what the reporter stated, instead of being copied into the sequence of events.

This matters because models are easily led. Tell one that a car was punted and it will happily "see" a punt. Separating what was claimed from what was seen removes that pull.

"Is there even a race in this clip?"

Before the real analysis, the system asks a context-free question: does this clip show racing at all? It's asked without the title, category or description, because those would prime the model to find the reported incident whether or not it's there.

If the answer is no (a menu screen, a wrong file, an unreadable clip), the report says plainly that no incident is visible and describes what the footage does show. It does not reconstruct an incident from the text. If the check itself fails for some technical reason, the analysis simply carries on as normal, so a hiccup never blocks a real case.

A list of what can't be determined

Every fact report ends with a section for things the footage can't settle: overlap that can't be judged from this camera, a car out of frame, lighting or compression hiding the contact. Timestamps follow the same idea. They're only included when the model is confident of the time, and an omitted timestamp is considered better than a guessed one.

That list is carried into the later stages, which is how uncertainty reaches the final decision. It's also useful to you. It tells you exactly what to go and check, or what extra camera angle to ask the drivers for.

It can only cite your rules

Language models love to fill gaps from memory. Give one a thin or missing rulebook and it will cheerfully cite FIA articles or the International Sporting Code, which your league never adopted.

We handle that in two layers. The prompt forbids citing anything not in the supplied rulebook, and if there's no usable rulebook it says so and reasons only from general principles like avoiding contact and leaving racing room. Then, as a hard check after the fact, the infringement line of a decision is compared against your actual ruleset text. If it cites an article or a governing body that isn't in your rules, the line is replaced with a plain description of the alleged offence, such as "alleged avoidable contact", rather than letting an invented rule stand.

A decision that quotes a rule you don't have is worse than one that quotes none, because it looks authoritative.

"Needs more info" is a real outcome

The final stage can choose needs more info as its decision, and when it does it must say exactly what evidence would resolve it: another camera angle, a longer clip, telemetry, a replay from the other driver.

In practice this is one of the most valuable outputs. If a model is forced to pick between penalty and no penalty on a clip where the answer genuinely isn't visible, it will guess, and a guess presented as a decision is how a league ends up with a bad ruling. A system that's willing to say it can't tell is one whose confident calls you can lean on more.

What this means for accuracy

None of this makes the model infallible. A depth judgement from a flat video can still be wrong, and it can't see intent or netcode. But the design aims for something specific: when the system is unsure, that should show up on the page, not stay hidden behind good writing.

Two habits make the guardrails work for you:

1. Read the unresolved list first. It's the quickest way to see how solid the report really is.
2. Treat "needs more info" as useful. It's the system doing its job, and it usually points to a better clip.

If you want to see how this plays out, try it free with a clip you already know the answer to. And for the bigger picture on where AI helps and where it doesn't, read can AI steward sim racing incidents?