There was a proverb, though, about that very road he had just left: Whoever sets out from Mount Mirror seeking the impossible, will surely return.
When you considered Jeffreyssai’s last warning—and that the proverb said nothing of succeeding at the impossible task itself—it was a less optimistic saying than it sounded.
Brennan shook his head wonderingly. How could Jeffreyssai possibly have known before Brennan knew himself?
Well, beyond a certain point, it is futile to inquire how a beisutsukai master knows a thing—
Brennan halted in mid-thought.
No.
No, if he was going to become a beisutsukai master himself someday, then he ought to figure it out.
It was, Brennan realized, a stupid proverb.
So he walked, and this time, he thought about it carefully.
As the sun was setting, red-golden, shading his footsteps in light.
*
Part Z
The Craft and the Community
312
Raising the Sanity Waterline
To paraphrase the Black Belt Bayesian: Behind every exciting, dramatic failure, there is a more important story about a larger and less dramatic failure that made the first failure possible.
If every trace of religion were magically eliminated from the world tomorrow, then—however much improved the lives of many people would be—we would not even have come close to solving the larger failures of sanity that made religion possible in the first place.
We have good cause to spend some of our efforts on trying to eliminate religion directly, because it is a direct problem. But religion also serves the function of an asphyxiated canary in a coal mine—religion is a sign, a symptom, of larger problems that don’t go away just because someone loses their religion.
Consider this thought experiment—what could you teach people that is not directly about religion, that is true and useful as a general method of rationality, which would cause them to lose their religions? In fact—imagine that we’re going to go and survey all your students five years later, and see how many of them have lost their religions compared to a control group; if you make the slightest move at fighting religion directly, you will invalidate the experiment. You may not make a single mention of religion or any religious belief in your classroom; you may not even hint at it in any obvious way. All your examples must center about real-world cases that have nothing to do with religion.
If you can’t fight religion directly, what do you teach that raises the general waterline of sanity to the point that religion goes underwater?
Here are some such topics I’ve already covered—not avoiding all mention of religion, but it could be done:
Affective Death Spirals—plenty of non-supernaturalist examples.
How to avoid cached thoughts and fake wisdom; the pressure of conformity.
Evidence and Occam’s Razor—the rules of probability.
The Bottom Line / Engines of Cognition—the causal reasons why Reason works.
Mysterious Answers to Mysterious Questions—and the whole associated sequence, like making beliefs pay rent and curiosity stoppers—have excellent historical examples in vitalism and phlogiston.
Non-existence of ontologically fundamental mental things—apply the Mind Projection Fallacy to probability, move on to reductionism versus holism, then brains and cognitive science.
The many sub-arts of Crisis of Faith—though you’d better find something else to call this ultimate high master-level technique of actually updating on evidence.
Dark Side Epistemology—teaching this with no mention of religion would be hard, but perhaps you could videotape the interrogation of some snake-oil sales agent as your real-world example.
Fun Theory—teach as a literary theory of utopian fiction, without the direct application to theodicy.
Joy in the Merely Real, naturalistic metaethics, et cetera, et cetera, et cetera, and so on.
But to look at it another way—
Suppose we have a scientist who’s still religious, either full-blown scriptural-religion, or in the sense of tossing around vague casual endorsements of “spirituality.”
We now know this person is not applying any technical, explicit understanding of . . .
. . . what constitutes evidence and why;
. . . Occam’s Razor;
. . . how the above two rules derive from the lawful and causal operation of minds as mapping engines, and do not switch off when you talk about tooth fairies;
. . . how to tell the difference between a real answer and a curiosity-stopper;
. . . how to rethink matters for themselves instead of just repeating things they heard;
. . . certain general trends of science over the last three thousand years;
. . . the difficult arts of actually updating on new evidence and relinquishing old beliefs;
. . . epistemology 101;
. . . self-honesty 201;
. . . et cetera, et cetera, et cetera, and so on.
When you consider it—these are all rather basic matters of study, as such things go. A quick introduction to all of them (well, except naturalistic metaethics) would be . . . a four-credit undergraduate course with no prerequisites?
But there are Nobel laureates who haven’t taken that course! Richard Smalley if you’re looking for a cheap shot, or Robert Aumann if you’re looking for a scary shot.
And they can’t be isolated exceptions. If all of their professional compatriots had taken that course, then Smalley or Aumann would either have been corrected (as their colleagues kindly took them aside and explained the bare fundamentals) or else regarded with too much pity and concern to win a Nobel Prize. Could you—realistically speaking, regardless of fairness—win a Nobel while advocating the existence of Santa Claus?
That’s what the dead canary, religion, is telling us: that the general sanity waterline is currently really ridiculously low. Even in the highest halls of science.
If we throw out that dead and rotting canary, then our mine may stink a bit less, but the sanity waterline may not rise much higher.
This is not to criticize the neo-atheist movement. The harm done by religion is clear and present danger, or rather, current and ongoing disaster. Fighting religion’s directly harmful effects takes precedence over its use as a canary or experimental indicator. But even if Dawkins, and Dennett, and Harris, and Hitchens, should somehow win utterly and absolutely to the last corner of the human sphere, the real work of rationalists will be only just beginning.
*
313
A Sense That More Is Possible
To teach people about a topic you’ve labeled “rationality,” it helps for them to be interested in “rationality.” (There are less direct ways to teach people how to attain the map that reflects the territory, or optimize reality according to their values; but the explicit method is the course I tend to take.)
And when people explain why they’re not interested in rationality, one of the most commonly proffered reasons tends to be like: “Oh, I’ve known a couple of rational people and they didn’t seem any happier.”
Who are they thinking of? Probably an Objectivist or some such. Maybe someone they know who’s an ordinary scientist. Or an ordinary atheist.
That’s really not a whole lot of rationality, as I have previously said.
Even if you limit yourself to people who can derive Bayes’s Theorem—which is going to eliminate, what, 98% of the above personnel?—that’s still not a whole lot of rationality. I mean, it’s a pretty basic theorem.
Since the beginning I’ve had a sense that there ought to be some discipline of cognition, some art of thinking, the studying of which would make its students visibly more competent, more formidable: the equivalent of Taking a Level in Awesome.
But when I look around me in the real world, I don’t see that. Sometimes I see a hint, an echo, of what I think should be possible, when I read the writings of folks like Robyn Dawes, Daniel
Gilbert, John Tooby, and Leda Cosmides. A few very rare and very senior researchers in psychological sciences, who visibly care a lot about rationality—to the point, I suspect, of making their colleagues feel uncomfortable, because it’s not cool to care that much. I can see that they’ve found a rhythm, a unity that begins to pervade their arguments—
Yet even that . . . isn’t really a whole lot of rationality either.
Even among those few who impress me with a hint of dawning formidability—I don’t think that their mastery of rationality could compare to, say, John Conway’s mastery of math. The base knowledge that we drew upon to build our understanding—if you extracted only the parts we used, and not everything we had to study to find it—it’s probably not comparable to what a professional nuclear engineer knows about nuclear engineering. It may not even be comparable to what a construction engineer knows about bridges. We practice our skills, we do, in the ad-hoc ways we taught ourselves; but that practice probably doesn’t compare to the training regimen an Olympic runner goes through, or maybe even an ordinary professional tennis player.
And the root of this problem, I do suspect, is that we haven’t really gotten together and systematized our skills. We’ve had to create all of this for ourselves, ad-hoc, and there’s a limit to how much one mind can do, even if it can manage to draw upon work done in outside fields.
The chief obstacle to doing this the way it really should be done is the difficulty of testing the results of rationality training programs, so you can have evidence-based training methods. I will write more about this, because I think that recognizing successful training and distinguishing it from failure is the essential, blocking obstacle.
There are experiments done now and again on debiasing interventions for particular biases, but it tends to be something like, “Make the students practice this for an hour, then test them two weeks later.” Not, “Run half the signups through version A of the three-month summer training program, and half through version B, and survey them five years later.” You can see, here, the implied amount of effort that I think would go into a training program for people who were Really Serious about rationality, as opposed to the attitude of taking Casual Potshots That Require Like An Hour Of Effort Or Something.
Daniel Burfoot brilliantly suggests that this is why intelligence seems to be such a big factor in rationality—that when you’re improvising everything ad-hoc with very little training or systematic practice, intelligence ends up being the most important factor in what’s left.
Why aren’t “rationalists” surrounded by a visible aura of formidability? Why aren’t they found at the top level of every elite selected on any basis that has anything to do with thought? Why do most “rationalists” just seem like ordinary people, perhaps of moderately above-average intelligence, with one more hobbyhorse to ride?
Of this there are several answers; but one of them, surely, is that they have received less systematic training of rationality in a less systematic context than a first-dan black belt gets in hitting people.
I do not except myself from this criticism. I am no beisutsukai, because there are limits to how much Art you can create on your own, and how well you can guess without evidence-based statistics on the results. I know about a single use of rationality, which might be termed “reduction of confusing cognitions.” This I asked of my brain; this it has given me. There are other arts, I think, that a mature rationality training program would not neglect to teach, which would make me stronger and happier and more effective—if I could just go through a standardized training program using the cream of teaching methods experimentally demonstrated to be effective. But the kind of tremendous, focused effort that I put into creating my single sub-art of rationality from scratch—my life doesn’t have room for more than one of those.
I consider myself something more than a first-dan black belt, and less. I can punch through brick and I’m working on steel along my way to adamantine, but I have a mere casual street-fighter’s grasp of how to kick or throw or block.
Why are there schools of martial arts, but not rationality dojos? (This was the first question I asked in my first blog post.) Is it more important to hit people than to think?
No, but it’s easier to verify when you have hit someone. That’s part of it, a highly central part.
But maybe even more importantly—there are people out there who want to hit, and who have the idea that there ought to be a systematic art of hitting that makes you into a visibly more formidable fighter, with a speed and grace and strength beyond the struggles of the unpracticed. So they go to a school that promises to teach that. And that school exists because, long ago, some people had the sense that more was possible. And they got together and shared their techniques and practiced and formalized and practiced and developed the Systematic Art of Hitting. They pushed themselves that far because they thought they should be awesome and they were willing to put some back into it.
Now—they got somewhere with that aspiration, unlike a thousand other aspirations of awesomeness that failed, because they could tell when they had hit someone; and the schools competed against each other regularly in realistic contests with clearly-defined winners.
But before even that—there was first the aspiration, the wish to become stronger, a sense that more was possible. A vision of a speed and grace and strength that they did not already possess, but could possess, if they were willing to put in a lot of work, that drove them to systematize and train and test.
Why don’t we have an Art of Rationality?
Third, because current “rationalists” have trouble working in groups: of this I shall speak more.
Second, because it is hard to verify success in training, or which of two schools is the stronger.
But first, because people lack the sense that rationality is something that should be systematized and trained and tested like a martial art, that should have as much knowledge behind it as nuclear engineering, whose superstars should practice as hard as chess grandmasters, whose successful practitioners should be surrounded by an evident aura of awesome.
And conversely they don’t look at the lack of visibly greater formidability, and say, “We must be doing something wrong.”
“Rationality” just seems like one more hobby or hobbyhorse, that people talk about at parties; an adopted mode of conversational attire with few or no real consequences; and it doesn’t seem like there’s anything wrong about that, either.
*
314
Epistemic Viciousness
Someone deserves a large hat tip for this, but I’m having trouble remembering who; my records don’t seem to show any email or Overcoming Bias comment which told me of this 12-page essay, “Epistemic Viciousness in the Martial Arts” by Gillian Russell.1 Maybe Anna Salamon?
We all lined up in our ties and sensible shoes (this was England) and copied him—left, right, left, right—and afterwards he told us that if we practised in the air with sufficient devotion for three years, then we would be able to use our punches to kill a bull with one blow.
I worshipped Mr Howard (though I would sooner have died than told him that) and so, as a skinny, eleven-year-old girl, I came to believe that if I practised, I would be able to kill a bull with one blow by the time I was fourteen.
This essay is about epistemic viciousness in the martial arts, and this story illustrates just that. Though the word “viciousness” normally suggests deliberate cruelty and violence, I will be using it here with the more old-fashioned meaning, possessing of vices.
It all generalizes amazingly. To summarize some of the key observations for how epistemic viciousness arises:
The art, the dojo, and the sensei are seen as sacred. “Having red toe-nails in the dojo is like going to church in a mini-skirt and halter-top . . . The students of other martial arts are talked about like they are practicing the wrong religion.”
If your teacher takes you aside and teaches you a special move and you practice it for twenty years,
you have a large emotional investment in it, and you’ll want to discard any incoming evidence against the move.
Incoming students don’t have much choice: a martial art can’t be learned from a book, so they have to trust the teacher.
Deference to famous historical masters. “Runners think that the contemporary staff of Runner’s World know more about running than all the ancient Greeks put together. And it’s not just running, or other physical activities, where history is kept in its place; the same is true in any well-developed area of study. It is not considered disrespectful for a physicist to say that Isaac Newton’s theories are false . . .” (Sound familiar?)
“We martial artists struggle with a kind of poverty—data-poverty—which makes our beliefs hard to test . . . Unless you’re unfortunate enough to be fighting a hand-to-hand war you cannot check to see how much force and exactly which angle a neck-break requires . . .”
“If you can’t test the effectiveness of a technique, then it is hard to test methods for improving the technique. Should you practice your nukite in the air, or will that just encourage you to overextend? . . . Our inability to test our fighting methods restricts our ability to test our training methods.”
“But the real problem isn’t just that we live in data poverty—I think that’s true for some perfectly respectable disciplines, including theoretical physics—the problem is that we live in poverty but continue to act as though we live in luxury, as though we can safely afford to believe whatever we’re told . . .” (+10!)
One thing that I remembered being in this essay, but, on a second reading, wasn’t actually there, was the degeneration of martial arts after the decline of real fights—by which I mean, fights where people were really trying to hurt each other and someone occasionally got killed.
In those days, you had some idea of who the real masters were, and which school could defeat others.
Rationality- From AI to Zombies Page 149