We read 6,563 'was I at fault?' threads. Here's the short version

Every day, sim racers post a clip and ask strangers the same question: was I at fault? Put together, those threads are a public record of how the community actually judges incidents, with no rulebook, no panel and no training. We wanted to look at that record before saying anything else about consistency, so we read a large slice of it.

This is part one of Fault lines, a four-part series on what the data shows. This post covers the headline numbers and how we got them. The next three go deeper on which moves get posted, what makes agreement drop, and whether different sims judge differently.

What we looked at

We took 6,563 threads from r/Simracingstewards where the poster is one of the drivers ("am I at fault?", "I'm the Porsche"), mostly from 2022 to 2026, and read the 77,573 top-level comments under them. Replies to replies were left out, so one person arguing with the poster doesn't count as extra votes.

Each comment was sorted into one of three verdicts: the poster is at fault, the poster is cleared (the other driver is to blame, or the move was clean), or it was shared or a racing incident. Only about a quarter of comments (26%) state a verdict at all; the rest are jokes, questions, requests for another angle or general chat. We kept threads with at least three clear verdicts, which left 2,694 threads and 15,838 verdicts.

The headline: you are cleared more often than blamed

The most common verdict in a thread was "the poster is cleared".

Majority verdict per thread. "Split" means no verdict had more votes than another.

Before anyone reads that as "most drivers are innocent": people choose what to post. A driver who thinks the other car drove into them is more likely to post than one who knows they dived in from five car lengths back. The number describes the posts, not the drivers.

The less comfortable number: reviewers agree about half the time

Pick two commenters in the same thread at random. They give the same verdict 51% of the time. If verdicts were handed out at random with the same overall mix, you'd expect 36%. So there is real agreement, but not much. Only 1 in 5 threads is unanimous.

The usual statistic for this, Krippendorff's alpha, comes out at 0.24 (95% range 0.22 to 0.25), where 0 is chance and 1 is perfect agreement. That's low by the standards of any rating task.

Two reasons to treat it as a floor rather than the full story:

• One verdict per comment hides multi-incident clips. A long clip often contains three contacts. A commenter who writes "first contact is his fault, the second is yours" is giving two verdicts, but gets counted as one "shared". Some of the disagreement is just an unfair measurement.
• Our verdict detector is imperfect. It reads comments with keyword rules, and when we checked it against fresh comments we had labelled by hand it was right 66% of the time (75% on the clearer fault-or-cleared calls). Errors push agreement down.

How we measured it, and what didn't work

We first tried the obvious route: ask a language model, or an embedding classifier, what each comment's verdict is. Against 160 comments we labelled ourselves they scored 40% and about 50%, no better than always guessing "cleared" (46%). The reason is simple. "You" and "he" change meaning depending on which car belongs to the poster, and the model can't see the video. Plain keyword rules, which only fire on explicit statements like "your fault" or "racing incident", did better, at the cost of covering only a quarter of comments.

It's worth saying because it applies to any tool that judges incidents: reading the words around a clip is not the same as watching the clip.

What this means for a league

• One reviewer's verdict is one opinion. If half of random pairs disagree, a single steward's call on a hard incident is not a measurement of the truth. Panels of three or more, or a written process every steward follows, give you something sturdier. Our post on consistent steward decisions in a league covers how.
• Write down what you mean. The next parts show that a handful of words, "alongside" above all, do much of the damage. Agreement improves when the rulebook defines them.
• Look at the evidence first. Many threads turn on what the clip shows, not on the rule. See why multiple angles matter.

A small note about us

Sim Racing Steward turns a clip and your ruleset into a structured steward report: what happened, which rules apply, and a draft decision, so every steward starts from the same facts. It's meant to make the first read faster and more consistent, not to replace the person who decides. Free accounts get 15 credits a month, enough to run a few real incidents through it. Sign up now.

Data: 6,563 first-person threads and 77,573 top-level comments from r/Simracingstewards, retrieved from a public Reddit archive. Verdicts are detected automatically and are not perfect. Numbers in this series come from the same sample.