The most uncomfortable self-knowledge: The less you know about something, the more confident you tend to feel. The more you know, the more uncertain you become. I’ve been on both sides of that curve in multiple domains — and understanding why it happens has changed how I evaluate my own certainty and how much I trust other people’s.
The time I was confidently, expensively wrong
About four years ago I decided to learn investing actively — to pick individual stocks rather than index funds. I read three books, opened a brokerage account, and within six weeks felt I had a fairly solid handle on the basics of business valuation. I made eight investments based on my own analysis. I was confident. I had done my research.
Fourteen months later, I had underperformed the S&P 500 by 18 percentage points. Not because I was unlucky — because I had systematically misunderstood key accounting concepts, misread competitive moats in two industries I thought I understood, and overestimated my ability to interpret management guidance in earnings calls.
The painful part was not the underperformance. It was looking back at my notes from those early weeks and reading how certain I had sounded. I had written things like “this seems straightforward” and “the thesis is clear” about situations that experienced analysts would have found genuinely complex. I had possessed just enough knowledge to feel confident — and not nearly enough to be right.
That experience is now my personal reference point for the Dunning-Kruger Effect. Not as an abstract psychological curiosity but as something I lived through in real money and real time.
Read also: How I Use a Decision Journal to Make Better Choices
What Dunning and Kruger actually found
David Dunning and Justin Kruger published their foundational research in 1999 in the Journal of Personality and Social Psychology, under the title “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments.”
Their experiments tested participants on logical reasoning, grammar, and humor. They then asked participants to estimate their own performance relative to others. The consistent finding: participants who scored in the bottom quartile dramatically overestimated their performance — placing themselves on average in the 62nd percentile when they were actually at the 12th.
“The skills needed to produce correct responses are the same skills needed to evaluate whether responses are correct. Incompetence robs people of the metacognitive ability to recognize their own incompetence.”— David Dunning & Justin Kruger, 1999
This is the core of the finding: the very skills you lack are the skills required to know you lack them. Poor logical reasoners cannot identify poor logical reasoning — in their own work or in others’. Poor writers cannot accurately evaluate writing quality. The absence of skill produces both the poor performance and the inability to recognize the poor performance.
Why it happens — the metacognitive explanation
The mechanism is metacognition — thinking about thinking. To accurately assess your own competence in any domain, you need a mental model of what competence in that domain actually looks like. That model is itself a function of domain knowledge.
When you know very little about a subject, your mental model of “what good looks like” is correspondingly simple. You evaluate your performance against a simple standard — and find yourself meeting it easily. As you gain knowledge, your model of excellence becomes more sophisticated, and the distance between your current performance and that standard becomes more visible. Expertise doesn’t make you better at the task while making you feel worse — it makes you more able to see how far you still have to go.
This produces the counterintuitive pattern: novices are most confident, intermediate learners most self-critical, and experts most accurately calibrated — often still expressing significant uncertainty even at very high levels of competence.
Read also: How Better Thinking Leads to Better Outcomes
The four stages of the competence curve
Where I’ve observed it in business and investing
Personal observation: early investors
First-year investors who have read a few books are often the most confident voices in a room about stock selection and market timing. Experienced investors with 20-year track records are more likely to say “I don’t know” — because they have lived through enough market cycles to understand what they genuinely cannot predict.
Personal observation: new entrepreneurs
People who have never run a business often have the most detailed and certain opinions about what businesses should do. People who have run businesses for a decade are more likely to say “it depends” — because they have learned that context dominates strategy in ways that simple frameworks don’t capture.
Personal observation: online commentary
The most confident, certain, unqualified opinions on complex topics — economics, medicine, geopolitics — are almost never expressed by domain experts. They are expressed by people who have read enough to form a confident view but not enough to understand why the question is more complicated than they believe.
The expert side: why genuine knowledge produces doubt
The Dunning-Kruger discussion almost always focuses on the novice’s overconfidence. The expert’s experience of the same phenomenon is equally important and less often discussed.
Genuine expertise produces calibrated uncertainty — a clear understanding of what is known, what is contested, and what is genuinely unknown in a domain. Experts often express doubt and nuance that novices interpret as weakness or hedging. It is neither. It is accuracy.
The physicist who says “we don’t fully understand this phenomenon” is not less knowledgeable than the internet commenter who explains it with certainty in four sentences. They are more knowledgeable — and their knowledge includes a clear picture of where the edges of current understanding lie, which the commenter’s simpler model completely misses.
This produces an unfortunate dynamic: in public discourse, confident novices often sound more authoritative than calibrated experts — because certainty reads as competence, even when it reflects its opposite.
Read also: The Hidden Distribution of Breakthrough Ideas
How to use this insight — practically
- Treat your own certainty as a signal worth interrogating When you feel very confident about something in a domain where your experience is limited, treat that confidence as a prompt to investigate further rather than a reason to act. Ask: have I been in the valley of this domain yet? If not, you’re probably still on the peak.
- Seek out experts and notice how they express uncertainty In any domain where you’re forming opinions, find the most credible practitioners and note how they qualify their claims. If their language is significantly more cautious than yours, that gap is informative. It is probably not because they know less — it is because they know more about what they don’t know.
- Interpret the Valley of Despair as progress, not failure If you enter a new domain and initially feel confident — then suddenly feel overwhelmed by how much you don’t understand — you have not gone backwards. You have progressed from Stage 1 to Stage 2. The discomfort is the cognitive experience of a more accurate mental model forming. Stay in it.
- Use calibration questions before significant decisions Before any significant decision in a domain where you might be at Stage 1, ask: “What would a genuine expert in this domain say I’m missing?” Then try to answer that question honestly — and seek out the actual answer if the stakes are high enough to warrant it.
The most useful question I now ask myself
“Am I confident here because I know a lot — or because I haven’t yet learned enough to know what I don’t know?”
Those two types of confidence feel identical from the inside. The only way to distinguish them is to ask the question explicitly and answer it honestly. In my experience, the domains where I’m most certain I know the answer are often the ones where I’m most likely to be at the peak.
