Human Bias in Business Plan Evaluation — Where AI Compensates for Halo, Anchoring, and Recency

#A Century-Old Finding: Human Evaluation Isn't Consistent

In 1920, Edward Thorndike analyzed commanding officers' ratings of 137 military officers. Categories that should have been independent — physique, intelligence, character — were correlated at abnormally high levels. A good impression on one trait was bleeding into every other rating. He named this the 'halo effect.'

Thorndike's conclusion was simple: raters can't see a person as the sum of independent traits. An overall impression of "good" or "bad" colors every individual judgment. A century later, this finding is still a textbook staple in hiring, promotion, and review literature.

#Four Biases That Show Up in Business Plan Evaluation

Halo is just the beginning. Among the cognitive biases repeatedly confirmed across hiring, evaluation, and review contexts generally, we've narrowed it down to four that are highly likely to operate through the same mechanism in business plan evaluation. Large-scale empirical research specific to business plan review is still scarce, but the consensus view is that a reviewer's cognitive system doesn't change just because the subject shifts from a person to a document.

#How Large Is the Effect, Academically Measured?

Bias isn't an abstract worry. Here's how large the effect measures out to be in repeated experiments.

• Anchoring — In Tversky and Kahneman's classic 1974 experiment, subjects spun a roulette wheel that landed on a random number (10 or 65), then estimated the percentage of African countries in the UN. The group that saw 10 had a median estimate of 25%; the group that saw 65 estimated 45%. A random number with zero relevance to the question produced a 20-point swing.
• Anchoring — In the same paper, a group asked to estimate 8×7×6×5×4×3×2×1 gave a median guess of 2,250, while a group asked to estimate 1×2×3×4×5×6×7×8 guessed 512. The correct answer is identical (40,320) — only the order of presentation differed.
• VC decision-making — In Gompers et al. (2020, JFE), a survey of 885 VCs found that team was the top factor in investment decisions, weighted more heavily than business model, technology, or market. Because team carries the most weight in the decision — if evaluation isn't separated by domain — a strong first impression of the team has more room to bleed into every other judgment through the halo effect. (The fact that team carries heavy weight is not, on its own, direct evidence of halo effect.)

#Where AI-Based Divided Review Compensates

One reason OpenSeed splits review across domain specialists instead of relying on a single generalist is to shorten the pathways these biases travel through. We're not calling it a structural blockade — division of labor doesn't reduce bias to zero. Here's how each bias is mitigated.

On top of this, there's a consistency guarantee — the same input produces the same result. When variance is controlled across two submissions of the same business plan, you can learn "revising this section raises the score" instead of shrugging it off as "just bad luck this time."

#AI Isn't Immune Either — an Honest Look at Its Limits

Claiming AI is completely free of human bias would itself be a kind of halo effect. AI-based divided review carries its own set of limitations.

1. Training data bias — if past successful business plans over-represent certain industries, regions, or founder demographics, AI learns that same bias
2. Position bias — the phenomenon where information at the start and end of a long context is weighted more heavily than the middle has been observed in LLMs too (the "lost in the middle" effect, among others). Human recency bias is weakened here, not eliminated
3. Shared source text — because every domain reviewer still reads the same underlying document, writing that appeals strongly in one section can still exert a small influence on scores in other domains
4. Prompt dependency — if the instructions given to a reviewer are poorly designed, AI reasoning can still be steered toward a predetermined conclusion (a variant of confirmation bias)
5. Validation data limits — without ground-truth labels for actual funding outcomes (accepted vs. rejected), there's no way to measure calibration accuracy from AI review results alone

#Well-Designed Human Review Can Reduce Bias Too

For a fair comparison, one point needs to be made. The fact that bias occurs in human evaluation doesn't mean every human review process operates fully exposed to it. The following design choices meaningfully reduce bias in human review.

So the real comparison isn't "human vs. AI" — it's closer to "unstructured solo human review" vs. "well-designed human review plus AI-based divided review." The latter is the strongest combination.

#So How Should AI and Humans Actually Work Together?

The conclusion isn't that AI replaces human reviewers. The two systems fail in different ways, which is exactly why combining them produces the most reliable outcome.

• AI-based divided review → first pass: fast, domain-by-domain checks for gaps, weaknesses, and consistency, without human cognitive fatigue or impression transfer
• Human reviewers → final verdict: qualitative judgment on novel industries, founder authenticity, social value, and the like — paired with the bias-mitigating design practices outlined above
• You, the founder → decision-maker: use the gap between the two results to identify where your business plan needs further work