Hedgehogs (experts) know one big thing, while foxes (generalists) know many little things. However, foxes seem to be better at predicting the future than hedgehogs, because they constantly challenge their worldview and see their own ideas as hypotheses in need of testing. So maybe the reason why foxes are more successful forecasters is that they focus on learning, their "one big thing" strategy.
> However, [generalists] seem to be better at predicting the future than [experts], because they constantly challenge their worldview and see their own ideas as hypotheses in need of testing.
What? No. You're implying that experts do not challenge their own beleives and ideas about the world.
I've seen this fairly often. I believe it's akin to the Dunning-Kruger effect in that experts begin to over-assess the effectiveness of the theories they hold most deeply to their expert identity the more entrenched they become in that identity.
It's usually easy to spot experts less susceptible to this problem (and more open to change) since they'll almost never consider themselves "experts" and avoid the label. They know how susceptible to change their theories could be or potential for being misguided/wrong. Their deep degree of understanding shows them just how little they truly know.
Furthermore, in our society, you must project confidence to be successful. Many experts are forced to appear confident when they're really not (does not make them less of an expert). If you're an expert and don't come off as exact, non-experts (and some experts even) may falsely assume you're not an expert because they expect you can produce an answer to some question they pose with certainty.
I believe this structure is partly to blame for the problem. Some play this confidence charade and are able to separate out the act of appearing certain, but still understanding the uncertainties they don't share and keeping them held tight.
The others get caught in this feedback loop where appearing confident and being right translates emotionally to being more confident. Their confidence is rewarded so they assume the the perspectives they have confidence in are sound the more it occurs over time. When a case comes a long that breaks that flow, they're still confident, but have positive emotional ties to those beliefs and become a victim of their own successes. Something else must be wrong because I've been rewarded so much for being right about this.
I believe those doing translational/applied sciences and engineering are more susceptible to problem than those on the theoretical side due to the way things are incentivized as a society. On the theoretical side there's a greater acceptance of being wrong (that's becoming less true as research is privatized more). On the applied side, people expect a high degree of confidence and certainty because that's what they pay for and there's little-to-no acceptance for error even though we're all human and make errors daily.
So I think the problem is really due to how society views experts and exerts its expectations upon them.
Nevertheless, I think that work defines us and it becomes our identity (e.g. Joseph the Carpenter, Bob the Builder), but at the same time there is so much more in you than just work. From what you are writing, I could identify what you don’t stand for and don’t want :-) this realization is a good starting point. However, now is the time to get curious, observe, reflect and start choosing your battles wisely, because you might wake up one day and realize that all you have been doing is wasting your time and energy on fights you didn’t actually care about that much… And this is precisely why it helps to know yourself, know your values, what you stand for, your talents and strengths, and maybe even your vision or dreams of how would you like to live in the future, and how would you like the world to look like in the future… Knowing this, I think, would allow you to focus your energy on something that is meaningful for you. Because meaning exists only in your head. So whatever we do, whatever happens in the world or the universe is meaningless per se. The meaning is interpretative. You create and give meaning to whatever you do, or whatever happens to you or in the world.
> but I want to know I'm doing something for a good purpose.
I think, you can never know in advance whether the purpose turns out to be “good” or “bad”, here is a little illustration of that :-) https://www.conures.net/stories/horse.shtml
Interesting example with 'guilt' as an emotion that may help to foster cooperation. This reminds me of an idea of evolution being treated as a thermodynamic process and 'punishment' as a crucial mechanism in the evolution of cooperation. The authors argue that "punishment acts like a magnetic field that leads to an 'alignment' between players, thus encouraging cooperation."[1]
