Overthinking Overthinking
Hi everyone,
I was sitting down doing morning pages thinking about what to write about. I started with domesticity and living environments, partly prompted by the flat move, partly by a discussion about a blocked sink in the old flat. Before long I’d pulled in concepts from a book on the nature of classification systems, linked in a book about the evolutionary game theory of gender norms, was thinking back to a previous post I’d written connecting the two up to a book on the early history of grain agriculture and how it lead to the development of the state…
It is, in fact, a very good way of looking at living environments. I’m going to sit on it a bit more and might write up something about it for a future newsletter. But it definitely put me in mind of my second favourite piece of feedback on my writing (the first is “There ought to be a rule against making unspeakable monsters adorable.”), which was John Regehr on a piece I wrote about bad incentives in correctness research. The tweet in question seems to have been deleted but was something along the lines of “I basically agree with all of this but think you might be overthinking it a wee bit”.
He’s not wrong. I’m very prone to overthinking things. However, I maintain that this is Good Actually.
Before I launch into a defence of overthinking, a small announcement, which is that as you might have noticed this has prompted a rebrand: This newsletter was previously called “How to do hard things” (named after this post about skill acquisition), but that name never really fit. The new name, “Overthinking Everything”, definitely fits. There will be no change in content, because this is what I was doing already.
What is overthinking?
It’s good to define terms, so I’m going to use wiktionary’s “To think or analyze too much.” as a working definition, but I’d like to note that “too much” is intrinsically a judgement call. Too much for what? Labelling something overthinking is a judgement, in the same way that calling something beautiful, or disgusting is. As a result, different people will regularly disagree as to whether something is overthinking.
From the inside, if you thought you were overthinking you’d probably not be doing it, so this is the most common source of disagreement: You think I’m overthinking, and I think I’m not. Which of us is right depends on a lot of factors - if I’m currently trapped in an anxiety loop, I’m probably wrong when I say I’m not overthinking it (although I’m reasonably well calibrated and tend to know that I’m overthinking and just not able to stop when this happens). If I’m an expert in a subject and can see problems that you, a non-expert, cannot, you’re probably wrong when you say I’m overthinking it.
Another reason people can disagree about “overthinking” is that they can disagree about the goal. As a judgement overthinking is generally considered a negative one (“too much”), and this is mostly fair, but it’s important to ask “too much for what?”. Sometimes something that is too much for one purpose is actually serving another - you eat too much at a feast, but the reason for that is that you are eating for enjoyment rather than sustenance (or comfort) - it is only too much by the latter standards, not by the former.
Overthinking as Transitional
One example where people tend to overthink things is in decision making. People suffer from analysis paralysis, thinking about the problem far more than the decision is worth.
This is a legitimate failure mode, and when I talk about decision making I emphasise that you’re supposed to do as little thinking as is needed for the decision in question (cf. "How to make decisions”). In this sort of case where you have a clear objective, there’s a relatively straightforward cost/benefit analysis: The benefit you get from thinking about the decision should be greater than the cost of thinking about it.
This tends to go against our instincts: Uncertainty is uncomfortable, so we will typically want to spend a significant amount of effort reducing it, often far more than that reduction in uncertainty is worth. Thus, in order to get better at decision making, we need to reduce overthinking.
I’m a big fan of the four stages of competence model as a way to reason about how we get better at things. The idea is that you pass through four stages of any given skill: Unconscious incompetence (you’re bad at it and you don’t know what being good at it would be like), conscious incompetence (you’re bad at it and you know why), conscious competence (you’re good at it as long as you actively work at it), and unconscious competence (you’re good at it without thinking about it). The model is oversimplified, but tends to track pretty well with my experience of getting good at things. In particular you can very rarely skip any of the stages (For some physical skills it’s arguable that you can skip the conscious competence stage, but I’m not fully convinced).
The relevance is that overthinking is generally a sign that you are in the conscious stages. If you’re not overthinking it and are getting good results, that’s unconscious competence, if you’re not overthinking it and are getting bad results, that’s unconscious incompetence.
This means that going from conscious competence to unconscious competence involves “stopping overthinking it”, but that doesn’t mean you can “just stop overthinking it” to get to unconscious competence: If you do that you’ll more likely backslide to to conscious incompetence. The move upwards instead requires figuring out which bits of thinking you can drop, and which bits you have to retain. You don’t avoid overthinking my merely thinking less, you avoid overthinking by learning which bits you can drop.
Often the way to figure that out is to overthink even more - go over decisions you’ve made and figure out what you could have done instead, figure out where you can cut corners, and generally just pay a lot of attention to the problem until the important bits become clear to you.
When doing this the transitional steps are still overthinking, in the sense that you are thinking about the decision more than it is worth and more than is ideal, but this is “too much” compared to two options that are not currently available to you: Not making the decision at all, or being someone who can make the decision without overthinking. Until you’ve managed the transition to becoming that person, overthinking is a good and necessary step.
Overthinking as Expertise Crafting
One of the ways you can tell the difference between a professional and an extremely competent amateur (in the original sense of “unpaid” rather than “bad”) is that often the amateur often does a much better job of things.
The reason for this is not that professionals are lazy (at least, not always that) but that professionals have spent a lot of time learning what is actually important - they know what corners it’s entirely reasonable to cut, and where they actually need to spend time.
Learning these shortcuts is generally worth it for a professional in a way that it’s not for an amateur. Spending an hour to learn how to do a five minute job in one minute is obviously worth it if you’re going to do that job a thousand times, and obviously not worth it if you’re going to do it twice.
From the amateur’s perspective, the professional is absolutely overthinking their craft: They spend what seems like an unreasonable amount of time on things that aren’t that important, but the reason for this is that the trade offs are different between the two.
This can also go in the other direction: There are skills that it is simply not worth acquiring unless you’re obsessed with a subject because even if they’re not that hard to do once you have them, they require an up front investment that most people can’t be bothered with it. Again, if you’re going to do something twice it’s not worth investing huge amounts of effort in perfecting it (unless you’d enjoy that), if you’re going to do it thousands of times it might be.
As a result of this, experts in a subject will often look slightly obsessed about minutiae of their craft that looks like it can’t possibly matter, because the benefit of understanding “small” details of their craft is so much larger for them.
Overthinking as Insight Generation
A thing I am often praised for in my writing is that I explain things that people kinda knew but lacked the tools to articulate or quite make sense of. This is true, this is a thing that I am good at.
There are basically two reasons I’m able to do this:
I’m good at explaining things (see How to explain anything to anyone).
I overthink things intensely in ways that you don’t.
Inevitably every time I give you one of these nice clear explanations it’s because I’ve spent ages chewing on much more complicated explanations first - it starts like the opening paragraph of this letter, where I connect up a bunch of disparate ideas, see how they fit together, use them as a frame for looking at things, etc. Often this goes nowhere, or leads to interesting things that don’t need the original ideas.
If you look at the final explanation, almost none of that is present, because I’ve put in the effort to simplify it at the end of the discovery process.
This is well explained by a recent visualisation from Grant Sanderson (3blue1brown):
An initial meandering exploration of the space of possibilities eventually leads to a solution, and then one refines the discovered solution into a simpler straight line path to the goal.
From the point of view of the end result, the research process looks like overthinking: You put far more effort into the problem than the eventual solution seems to warrant, but this is only obvious in retrospect.
Additionally from the point of view of someone on the outside, you may look like you’re overthinking before you find the solution, because you seem to be spending a lot of time thinking about something that isn’t obviously going anywhere. Often all you can do is justify it as a sort of intuitive sense that there’s something interesting there that you’re looking for (as per “Maintaining Niche Interests”, my theory of interestingness is basically that something is interesting if it feels worth thinking about more). Because that feeling of interest is not easily transferable, this will often look like overthinking while just feeling like thinking.
You’re Underthinking Overthinking
Despite my impassioned defence of overthinking, I acknowledge that for most people overthinking is a major problem - it’s less often the sort of positive forms of overthinking I’m talking about, and more often associated with analysis paralysis, or the result of anxiety. Overthinking can be a genuine problem.
But there’s a very simple reason for that: It’s because you’re not very good at it.
Overthinking is a skill like any other, and most people are stuck in the unconscious incompetence stage because they don’t know how or when to use it. For most people this often looks like overthinking in useless ways (e.g. thinking in circles as the result of anxiety) or inappropriate contexts (e.g. analysis paralysis while making decisions) or failing to overthink in cases where it would be useful to do so (the scenarios I’ve described in this letter).
I don’t quite know what to suggest if you’re in the latter camp (it’s not really my area of expertise), but if you’re in the former camp my advice is to lean into the overthinking you’re already doing. Rather than treating overthinking as bad, revel in it. If you’re overthinking something because you’re anxious about it, OK! Why not sit down and write a plan? Maybe you’re right and things will go terribly wrong. What will you do then?
Once you start deliberately overthinking things, you have the opportunity to get better at it - start noticing the bits that it turns out you didn’t really need to overthink after all. Overthink your overthinking, until you’ve got the skill of deciding what really needs it down to unconscious competence.
Even before you get to that point, if your overthinking is anxiety driven chances are good that you’ll naturally feel less inclined to do it in inappropriate situations as a result of this. Properly planning for these scenarios is a good way to raise your confidence that you can deal with whatever comes up, which tends to lower the anxiety naturally.
In general, this seems to be a good approach to a lot of strong feelings: Many feelings tend to rise to whatever strength is required for you to take notice of them, so working through those feelings by taking them more seriously than it seems like they warrant is a good way to figure out and deal with the underlying problem and lower the strength of feeling to a manageable level. In short, you need to overthink your feelings, but first you should probably overthink how you go about doing that.
I love this. It all resonates with my experience so much. As a fellow inveterate overthinker let me toss in a couple more observation.
Here's one: the motivation for "good" overthinking depends on an intuitive sense that the conventional explanation of a situation is shallow or imperfect, and that a deeper, more insightful explanation is possible.
When people resist this kind of thinking, I find it's for a couple reasons. First, as you say, they don't know how to think like this, in that they are uncomfortable with the searching phase, which feels aimless compared to plain old means-end analysis and problem solving. But also, they simply lack that intuitive sense that the conventional explanation is flawed or shallow, and that there is something deeper to search for.
I think this sense is strongly domain dependent.
For instance, if you know some area of mathematics well, you build up your sense of what is the essence of the issue involved in solving a problem. That's why you can tackle it quickly and clearly, instead of grinding through unnecessary calculations to reach the same solution. But it's only through exposure that you build this sense, this intuition, that a deeper insight is possible.
But the same is true in culture, interpersonal affairs, and personal psychology. It's only through time and exposure that one builds up the ability to sense that a deeper insight is possible, and available through productive overthinking.
Because the sense of possibility is so domain dependent, you of course often find people who are onboard for "productive overthinking" in one area, but are impatient with thinking very deeply in another area of life. In the area that interests them less, they just don't see that there's more to see. You can be insightful about linear algebra and shallow about unconscious motivations.
Perhaps we are all "overthinkers" in some areas and not in others, and the interesting distinction between people is how many areas and which areas.
A last observation: one frustration of being an overthinker is the curse of knowledge. Once you've found a compact, simple explanation of an interesting area, the whole area seems simple, less interesting, and not worth thinking about or writing about further. Indeed, looking at everything I've written above, part of me feels, "Well isn't this just a longwinded way of saying *thoughtfulness* is good, some people are more thoughtful than others, and are thoughtful about different things"? Sort of.
Okay but the bit about overthinking and the competence model reminds me of:
I sometimes help a self-employed builder out. On one house, i put lagging (insulation) on the hot water pipes beneath. It's the first time I've ever done this. (It's like small pool noodles with a slit right along it, which is efficient if awkward at joins and corners)
And when the house was inspected, the inspector said that it was by far the best and tidiest lagging he's ever seen.
And it's not so much because I'm better at it than experienced plumbers so much as, i just took a bit longer and did a really good job *because* I'm not experienced enough to know what shortcuts i /can/ take to get it done faster and still do an adequate job.
(Also, since I'm finally commenting anyway: i really appreciate your writings. I've been told i tend to be insightful, but those insights tend to lean more on the "how people tend to work" and "systemic issues" a little bit moreso than practical insights. I'm also pretty decent when it comes to self insight & understanding too, i suppose, but it seems your writing manages to hit a lot of the gaps in my understandings. It's pretty excellent, and amazingly helpful for me! So, thank you for putting your understandings on the internet.)
~CS