Survivorship Bias Is the Original Sin of Startup Scoring

Here's a failure mode I see in almost every startup-scoring model, including early versions of my own: it has never been shown a company that died.

Not by design. By convenience. The clean, available, labelled data in venture is the positive class — the decks that raised, the companies with a funding round on record, the founders who did a podcast about the journey. The negative class — the ones that quietly folded — leaves almost no structured trace. So the training set fills up with survivors, the model fits to them, and you ship a scorer that has effectively learned one lesson: most things are promising. It is calibrated to a world that doesn't exist.

You can see it in the score distribution. A model trained without failures bunches its predictions high and refuses to use the bottom of the range. It has no anchor for "bad," because it has never met "bad." Every deck looks like a pre-IPO winner viewed from the seed stage.

The fix isn't a cleverer architecture. It's the negative class.

Wiring in a real failure class

This week we wired a real one in. An accelerator outcome graph — thousands of companies from public cohorts — carries something most datasets don't: an explicit dead status. Roughly a thousand companies that were funded by top programmes and still didn't make it. We turned those into labelled negatives and put them in the corpus next to the exits, at full weight. Positive labels from acquisitions and IPOs; negative labels from shutdowns; the two classes finally in the same room.

The tradeoff is real and I'll name it, because a scorer that only celebrates is worse than useless — it's confidently wrong. Adding a failure class costs you some apparent accuracy on the easy positives and forces the model to actually separate "confident and well-produced" from "likely to survive," which turn out to be nearly uncorrelated. Good. That separation is the whole job.

Two things broke, and they're the useful part

The first was entity resolution. Linking a startup's investors to the platform's directory sounds trivial until a normalised key of "e" — the leftover of a firm named "e.ventures" after you strip the suffix — matches essentially every string it's tested against, and you find yourself asserting that a Melbourne accelerator programme is a San Francisco seed fund. The fix was unglamorous: require both sides of a fuzzy match to be substantial, and flag anything that doesn't resolve as a new entity rather than forcing it onto the nearest name. Under-linking is recoverable. A false link poisons every downstream query.

The second was label contamination. Enriching "did this company raise?" from the web, you'll happily record that a company "raised $120k with investors including its accelerator" — which is not an outcome, it's the accelerator's own entry cheque, the input dressed as a result. Label that as a fundraising success and you've taught the model that getting in equals winning, which is exactly the circularity you were trying to escape. We filtered those out by amount and investor. Some judgment calls; documented, not hidden.

A model is a mirror for its labels. Show it only winners and it will only ever tell you what you want to hear.

Frequently asked questions

What is survivorship bias in startup scoring?

It's when a model is trained mostly on companies that survived — the ones with a recorded raise or a headline — because failures leave little structured trace. The model then learns that almost everything is promising, and has no anchor for what a bad outcome looks like.

How do you fix survivorship bias in a training corpus?

Add the negative class. Take real, verifiable shutdowns — companies explicitly marked dead — label them as negatives, and weight them equally with the exits. This forces the model to separate 'confident and well-produced' from 'likely to survive', which turn out to be nearly uncorrelated.

Why is label contamination dangerous in venture data?

Because the errors look correct. Recording an accelerator's own entry cheque as a company's 'fundraise' is technically true and completely misleading — it teaches the model that acceptance equals success, the exact circularity you were trying to escape. The plausible, well-formed false label is the one that gets through.