In English
What would happen if a superintelligent AI was aligned with your values?
The details here are a little too much in the “superintelligence is magic that can achieve anything” direction to my taste (I don’t think that anything will just be instantly teleported into safety, superintelligent AI or not), but I don’t doubt that the same results could be achieved via more mundane means. And it’s nice to have some more uplifting visions of the future.
The Choose Your Own Adventure Book or Ghost Ship Model of Will
I first put the core idea into words when someone I met at a workshop said she often had trouble being on time for things. She would notice that it was time to go a meeting soon but that she still had three minutes, so she could, keep reading her novel. And then, ten minutes later she’d actually stop and end up late, which she didn’t like. To this, I said something like “Ah. Apparently, your choice points don’t happen very often when you’re reading a novel. So, if you want to avoid being late, maybe you should seize choice points shortly before you need to leave, because you probably won’t get another one in time.”
The metaphor here is that your agency has a structure somewhat more like a choose your own adventure book than the completely free “I can do whatever I want whenever I want” which we often see it as. Of course, the chose your own adventure book metaphor is too constraining; it offers too few choices compared to one’s experience of the real world. Nonetheless, it captures the sporadicalness of the choice points.
Patri Friedman on political views.
In a comment someone implied that one should have “a stable set of political views that are interrelated and coherent to some degree.”
I think I might disagree with this, and thought y’all might find it interesting.
So my claim is that to the degree that political views are describing mechanisms and outcomes in the real world, the real world is so complex that I’m actually not sure that an accurate description would be “interrelated and coherent” to a significant degree.
In fact I’ll hypothesize that most of the time when people choose view B partly based on how related and coherent it is with view A, they are making a worse choice (for building an accurate model of reality) than if they chose view B solely based on how it seems to empirically fit the world.
Coherence is beautiful and appealing, our mind likes simpler models, but except in the few cases where reality has a strong simplicity orientation (laws of physics), generally a move towards simplicity is a move towards a smaller, more impoverished space of models which is thus less likely to be an accurate description of complex reality. You are throwing away degrees of freedom when trying to fit a very irregular curve.
I think the laws of physics & mathematics have spoiled us because they are so universal, present everywhere, extremely important when they apply, and have so much simplicity and elegance. And I feel like I may be becoming (through a combination of reflecting on past idiocy, and getting really into meditation) such a radical empiricist that I view the desire to find simple models for the world as an omnipresent human foolishness.
I will caveat that the legibility & computational tractability of simple models do matter, our brains can only manage a certain size of model, I just think we will generally understand reality better by viewing it with curiosity and openness to it being modeled by incoherent, unrelated sub-models, rather than trying to force it to conform to our current set of (imperfect and incomplete) models.
On Anchor Collapse and Actually Deciding
Say you’re afraid of dogs. You don’t want to be afraid of dogs, of course, because you like dogs and everyone knows that only some dogs are mean. […] and you don’t want to be stupid, so you deny that the other side of the argument even exists. “There’s no reason for it”/”its irrational”/”I have a phobia”. […]
But let’s be real here. Dogs bite. I’ve been bit. If you’re phobic, you’ve probably been bit too. If you give yourself some room to not worry about looking stupid and look at the facts, there’s a reason to be afraid of dogs. You can’t guarantee you won’t get bit again, and getting bit really freaked you out. You really don’t want it to happen again. Once you admit this you can start to frame it as a decision. […]
So you’ve admitted that yes, the dog might bite you, and that would be really bad. But you still want to pet the dog! So you tell me “jimmmy! I want to not be afraid of dogs so I can pet them!”
“So pet the dog”
“But it might bite me!”
“It might”
“But I don’t want it to bite me!
“You don’t. And if it does, it will be real hurty. Have you considered that maybe you shouldn’t pet the dog?”
“But I want to pet the dog!”
“Then pet the dog”
“But it might bite me!”
…And we can go on all day like this. You’re wanting to pet the dog and not be afraid, but you’re also not wanting to get bit. As if there’s anything I can do about it. The risk is part of the territory. […]
And the way people often handle these is to just get sick of the struggle and suppress one side. “Okay, I know its a nice doggy so I’m gonna pretend that I’m okay with risking getting bit when really I’m not and I’ll just suppress that”. Only what they actually say to themselves is more like “I know its safe. I already decided. The fear is irrational and I want it gone.”But that’s not shitting or getting off the pot. That’s not collapsing the anchors. The two desires are still separate, so that’s not actually deciding. […]
But that’s nonsense. Of course you don’t want to get bit. Who wants to get bit? Getting bit is hurty and bad. And you want to pet the doggy. At the same time. Of course you want to pet the doggy. Doggies are cute and nice. And you haven’t let yourself go there because “I can’t have it so I’m not allowed to think it” but you really wish you could pet the dog with no risk of it biting you. It’s the best of both worlds. It would be really nice to pet the dog with no risk of it biting you. […]
The interesting thing is what happens the moment you stop holding the desires apart and experience them both simultaneously. This is collapsing anchors.
And it goes something like this…
I want to pet the doggie, and if I do, I might get bit.
…
…
…
(Seriously, give it a moment. Shit takes time.)
…
Is it worth it?
Am I willing to stick my hand out and pet that dog knowing that there is some chance that the dog is going to bite it?
And then you sigh a bit. And then you’re silent. And you picture not the separate issues of petting (good!) and being bit (bad!) but the combination package of getting to pet the dog but maaaaaybe getting bit. […]
If your answer is yes, then you can say “yes, I want to pet the dog, even knowing that I might get bit. I still want to pet the dog because it’s worth it. I want that package deal where my hand gets bit sometimes.”
Or if your answer is no, then you say “No, I don’t want to pet the dog. It’s not worth the chance of getting bit”. And that’s the end of it. It’s not “but I wish I could pet it and it wouldn’t bite me!” because you know that comes with the territory – it’s a dog and you can’t predict them perfectly. […]
And either way, there’s no conflict. No two separate desires. Just a congruent choice coming from a decision you had not made before.
See also:
Duncan Sabien’s concept of “Fabricated options“, which is when you don’t collapse the two and insist that of course there has to be a way where you pet the dog and are guaranteed to not get bitten.
Internal Double Crux, which is a technique for integrating both the “I want to pet the dog” and “I might get bitten”.
Adam Davis on a student with an apparent trauma history.
This semester’s unusual student experience was an office hour in which I heard “I can’t be told I’m wrong. It upsets me, and I freak. I shut down. You have to say things like, ‘there’s another way of looking at it,’ or ‘have you thought about it this way?’”
I tried “You might not know about this… (?)” Nod.
If you were expecting a rant on the lines of “Ach! These young snowflakes today!” keep scrolling. There’s plenty out there.
There is not the least question that this was a person already fragile, damaged, subjected to sustained abuse, who cultivated withdrawal as a firstline coping mechanism. Also not in doubt: without her courageously frank explanation and clear request, I and my colleagues would surely have done something to drive her away. Years ago, when no such conversation would have been imaginable, she’d simply have quit showing up one day, and we’d maybe have wondered, briefly, why.
Oh stop. There’s no prospect of my building her “resilience,” “grit,” whatever word we use to make it feel all right to be callous, by asserting my right — and I do indeed have the right, and the privilege — to say whatever I want to say, however I want to say it. The world is better, not worse, because she declares a need for accommodation, and gets it. We are not weaker. Our precious bodily fluids are just fine.
She’s finishing the semester. That’s big. That’s a real thing.
And here’s another real thing: it occurs to me that what she was asking wasn’t, actually, the least bit unreasonable. In fact, although I can’t specifically recall telling a student, in so many words, “you’re wrong,” mmm, there are lots of things we quite routinely say that come close enough, and there are many fully functional alternatives that get us through the necessaries just fine. Her request makes me reflect on how I communicate with all kinds of people who maybe don’t have the kind of guts and poise and self-understanding she has.
Oh, she is not weak.
Somebody said, “we’re all just walking each other home.” When kindness does not come to us easily or naturally, as it assuredly does not come easily or naturally to me, noop, it’s all the more important to sink effort into it.
There was a lot of debate of this on my Facebook.
Some people were saying that “we should get this person to work on her problem rather than accommodating her in this way”. But actually accommodating her can be an important first step in helping her fix the problem!
Often people with these kinds of issues experience strong shame about it because others send the message that it’s unreasonable/not okay. That shame then makes it harder to deal with the original issue because it’s painful to think about.
If there are people who communicate with their behavior that the person doesn’t need to feel deep shame about their problem, then that actually makes it easier to work on the problem itself.
There was also a bunch of debate about whether this was reasonable, whether it’s even possible to finish a degree without being told you’re wrong, etc.. I think this was a bit ambiguous from the original post. My interpretation was that the specific phrasing of “you’re wrong” was triggering to the student, but she was open to having the same point communicated with a different phrasing, and that e.g. having assignments marked down for problems wouldn’t be an issue. The fact that her lecturer didn’t consider there to be an issue would be in line with this interpretation.
Some people with experience in education also chimed in, pointing out that they’ve never had a reason to say “you’re wrong” to a student – that there’s always a better way of expressing it, and it just seems like common decency not to embarrass a student.
In any case, even if it was the case that “she can’t realistically expect people to accommodate her this way”… if someone has this issue, it’s likely due to something like cPTSD, which can easily take years to recover from. So it’s simultaneously true that it will be a major problem for her until she gets over it, and that getting over it may take very long and require accommodation and external support to get there. That combination of facts sucks, but just saying “she should get over it” isn’t going to solve anything.
How the incels warped my research
I have generally tried to ignore the manosphere. But as an evolutionary psychologist, I’ve found that hard to do. You can hardly read two paragraphs of incel ideology without coming across references to my field.
Louis Bachaud and Sarah Johns recently published a content analysis of manosphere messaging in the journal Evolutionary Human Sciences, explaining the ways in which our research gets appropriated by manosphere circles.
For example, incels maintain a wiki page of scientific citations they claim support their worldview — an annotated bibliography of misogyny. In one case, in a sort of Russian nesting doll of misrepresentation, the incel wiki quotes a paper citing a study of mine as demonstrating that women prefer dominant men — which they further twist into the incel notion that women actually prefer violent men as romantic partners.
Reading this entry, I thought, “That’s odd, I don’t remember ever publishing on dominance preferences — do the incels know my work better than I do?” No. I double-checked: That study didn’t even mention dominance preferences.
Curiously overlooked in this whole wiki section on women’s preferences is the fact that kindness is repeatedly found to be among the most desired qualities in large-scale, cross-cultural studies of mate preferences. […]
Like any biological approach to behavior, evolutionary psychology has always been controversial. In part, this is owing to some truly bad actors in the field. All it takes is some thoughtless tweets or blog posts for the entire field to earn a reputation as a safe space for provocateurs. […] This allows the manosphere to sell its audience a scientific consensus around its ideology that simply does not exist. Its members appropriate and mischaracterize the literature on evolutionary psychology to lend a scientific patina to their hateful, misogynistic, and dangerous ideas.
For instance, incels are obsessed with the “dual mating strategy” hypothesis, a divisive idea that interprets fluctuations in women’s sexual desire as evidence that women have evolved to seek out men with “good genes” at the most fertile point in their menstrual cycle. Incels use this hypothesis to explain, in their eyes, why relationships are doomed: No matter how good a partner you are, women will always be looking to sleep around with someone better.
Part of the problem is that the dual mating strategy hypothesis was indeed a popular idea among evolutionary psychologists until about 2016. After that, it became one of the more prominent epicenters of psychology’s replication crisis, which revealed that large swaths of psychology research were based on unreliable findings. But even before this major setback, the dual mating strategy hypothesis was critiqued by some evolutionary psychologists like my friend and colleague Jim Roney. Nonetheless, Jim’s work gets hardly any play in manosphere circles, and the hypothesis has since morphed into a version quite unlike the one promoted by incels.
At the end of the day, incels attempt to draw from evolutionary theory a power it does not have. Evolution is not destiny. It is a powerful tool for explaining how we came to be who we are today, but it cannot tell us who we should be today or who we can be tomorrow.
“I lost trust”: Why the OpenAI team in charge of safeguarding humanity imploded
Ilya Sutskever and Jan Leike announced their departures from OpenAI, the maker of ChatGPT, on Tuesday. They were the leaders of the company’s superalignment team — the team tasked with ensuring that AI stays aligned with the goals of its makers, rather than acting unpredictably and harming humanity.
They’re not the only ones who’ve left. Since last November — when OpenAI’s board tried to fire CEO Sam Altman only to see him quickly claw his way back to power — at least five more of the company’s most safety-conscious employees have either quit or been pushed out. […]
… the real answer may have less to do with pessimism about technology and more to do with pessimism about humans — and one human in particular: Altman. According to sources familiar with the company, safety-minded employees have lost faith in him.
“It’s a process of trust collapsing bit by bit, like dominoes falling one by one,” a person with inside knowledge of the company told me, speaking on condition of anonymity.
Compare enough statistics, some of them are going to correlate closely just by random chance. For example, the popularity of the first name “Eleanor” in the US closely correlates with the amount of wind power generated in Poland, r=0.993, p<0.01.
Apparently I organised a student protest against a teacher
This is the best thing I’ve read in a long time. Autistic child who has problems understanding social norms reacts to a mean teacher, without realizing it ends up organizing a student revolt and causing the teacher to get replaced.
First, we need to understand what kind of polity the Old Republic – and thus the Empire – is. And here, the phrasing I go to (somewhat imprecise) is that the Republic was a ‘Republic of Princes’ in the same sense that the Holy Roman Empire was an empire of ‘princes’ or more technically ‘imperial states. […]
In short, the Republic was not a democracy of people but a republic of states, the ‘princes’ which in turn governed their own territory internally. These ‘princes’ could be any form of government. And indeed, the imperial states of the Holy Roman Empire could be noble rulers, but also bishops ruling cities (the ‘prince-archbishops’), monks running abbeys (Imperial prelates), grandmasters running holy orders, and even cities governing themselves (free and imperial cities). So too with the Republic, which is why the Trade Federation can sit on the Senate alongside democratic Naboo and monarchic Alderaan. […]
… what I think a historian of this period, looking back would conclude about the Star Wars story would be this: the Clone Wars were essentially a civil war between the princes of the Rim territories against the princes of the core regions (as the later effectively ruled the senate). That civil war produced political momentum among some of the core princes towards centralization, which fuels the career of Palpatine. Palpatine’s reign and the Empire in general is thus understood as a reaction to the Clone Wars primarily aimed at centralizing power at the expense of the princes. […]
[…] now there are simmering tensions which the Imperial Navy is supposed to tamp down. As a result, imperial designers reach for escalation dominance in their designs, aiming to build ships which can, on their own, intimidate the militaries of the princes – because remember, the ‘princes’ (planetary governments of whatever form) all have their own small navies – in order to avoid a conflict. The [Imperial Star Destroyer] is the end result of that design philosophy: a gun-platform powerful enough to be effectively beyond the ability of any planetary princely navy to fight effectively.
Certain days driving home in Hanoi, metis would take me.
The hundreds of bikes and cars moving unpredictably required of me an intense focus in all directions at once, a broad awareness and an intense focus working as one. If I couldn’t track the speed, direction, and distance of every vehicle behind and ahead of me, and stay open to noticing potholes, sudden braking cars, or swerving buses – I might not survive the drive home. I’d seen enough dead bodies and broken bones on the roads to know that traffic was powerful and indifferent to me, the way the ocean is to sailors, the way the jungle is to hunters.
On certain magic days, when the traffic and my focus merged into a liquid exchange, something would happen and I’d be beyond focus and awareness. Beyond my self and my neck-swiveling calculations of swerving trajectories. On those days, I was a fluid entity of sensory intuition – heat on the side of my face and the thick tang of stale diesel exhaust told me without looking that I had a bus to my left. The honks and revs around me, the way each one muffled, or grew shriller, or faded, became an internal picture of the vehicles around me – how they were rushing up on me, turning to a side street, falling behind… The flicker of red reflection off the edge of my glasses told me the car ahead had tapped its brake lights. The sudden drop of the bike’s engine a few feet away told me they were suddenly braking in reaction.
Wordlessly, thoughtlessly, acting simply as an aspect of the situation pouring around and through me, I banked left and revved to get ahead of the bus, before it could block me off from the gap between it and the truck in front of us. I couldn’t see the traffic ahead, but everything I could see, hear, feel, and smell (the exhaust got a touch thicker, didn’t it?) told me there was a blockage in traffic ahead on the right side – and my experience with these roads told me obstacles like that don’t stay on one side of the road for long, they spread quickly until only a trickle of traffic can make it through the gridlock. I could either break ahead of the mess right now, from the left, or I’d be stuck here for 10 minutes waiting. I slipped through the gap just before the bus closed it, and sped out ahead. Me and the 2 or 3 others who had banked left rushed out into open road as the knot behind us tightened. […]
I wish I didn’t have to say that those dangerous, exhaust-fume-reeking days in Hanoi were some of the greatest peak experiences of my life, but here we are. I have journal entries from that year, long winding devotional prose poems to the Goddess of the Gap – an embodiment of that perfect gap in traffic that moves with divine smoothness, if you can just devote yourself to it and prove yourself worthy of staying in it. […]
I came literal inches from death over and over again in pursuit of it. […]
I drove around Hanoi without a helmet for a long time. I didn’t really understand why. It was stupid, I knew it was stupid. I felt really Alive without it though, and I couldn’t figure out how I could be so smart in so many ways, and so deadly stupid about this – and how even while knowing all this and thinking about it, I kept not wearing a helmet, because some blood-deep devotion to the Goddess of the Gap was somehow making me Alive, waking up some latent essence that had been sleeping inside me my whole life.
The arrow doesn’t fly if the bowstring is never pulled taut. Without tension – true, dangerous tension – you never even get the opportunity to hit the target.
A Woman Who Left Society to Live With Bears Weighs in on “Man or Bear”
When I’m alone in the backcountry and come across a man, I feel a very low level of vigilance. Depending on the situation, I might even be happy to see him. He’s a fellow human! Maybe we’ll be friends! I’m likely to smile genuinely and say hello.
I don’t feel afraid, but I am aware. As we chat, my intuition absorbs a thousand things at once. His body language. His tone. How he looks at me and interacts. Most of the time, this produces an increased sense of security. Most men are friendly, respect my boundaries, and don’t want to hurt me. Most of the time, I feel very safe around men.
But not all the time. Sometimes, my intuition absorbs things that increase my level of vigilance. […] It could be something he says. Maybe he makes a comment about my body or my appearance. Or he asks if I’m carrying a weapon and then presses for details about where I’m camping that night. Sometimes, it’s a shift in his tone, a leer, the way he puts his body in my space. But, usually, it’s a combination of things, a totality of behaviors that add up to a singular reality: this man is either not aware that he’s making me uncomfortable, or he doesn’t care. Either way, this is the danger zone. Even if he has no intention of harming me, the outcome of that intention is no longer possible for me to assess or predict.
In this moment, my mind snaps into a single, crystalline point of focus. My intuition rises to the surface of my skin. I become a creature of exquisite perception. The world is a matrix of emotional data: visceral, clear, direct.
I need to get away from the man. But I need to do it in a way that doesn’t anger him. This is the tricky bit. Men who lack social awareness or empathy often also lack other skills in emotional management. And usually, what men in these situations actually want is closeness. They’re trying to get closer to me, physically or emotionally, in the only way they know how. That combination of poor emotional skillsets and a desire to get closer is exactly what puts me in danger.
If I deny his attempts at closeness by leaving or setting a boundary, he could feel frustrated, rejected, or ashamed. If he doesn’t know how to recognize or manage those feelings, he’s likely to experience them as anger. And then I’m a solo woman stuck in a forest with an angry man, which is exactly what women are most afraid of.
There’s no time to think, so I operate on instinct. My task is ridiculously complex. I need to deescalate any signs of aggression, guide the man into a state of emotional balance, and exit the situation safely, all at once. This process requires all of my attention, energy, and intellect. It’s really hard.
I’ve been in this position so many times that it exhausts me just to write about it. Sometimes, it’s not that I’m afraid of men; I’m just really, really tired.
Spencer Greenberg on distributions and personality traits
Important but often overlooked: when groups differ a small amount in their means, they may differ *dramatically* in their tails.
For instance, in a personality study, we found males to have a little bit lower average compassion score than females (1.4 vs. 2) […]
Small differences like this in averages are typically not noticeable or important. Most people are somewhere near the middle.
If you knew only someone’s compassion level and had to guess their sex from it, you’d be wrong more than one-third of the time (predicting optimally)!
However, small differences in means can lead to much bigger differences in the “tails” (i.e., way on the right or way on the left of the chart). In other words, whereas the percentage of people just above the mean (or just below it) may come from the two groups in roughly equal proportions, the percentage of people who have very high levels of the traits (or very low levels of it) may come from just one of the two groups most of the time.
To see this happening for the example of compassion: despite only a small difference in mean compassion levels between males and females, among just the most compassionate people in our study, there were about 2x more females than males […].
Moreover, the least compassionate people […] were almost all males! […]
Similarly, on average, females usually test only a bit higher than males on peacefulness and forgiveness.
But, if we look at the tails of behavior, we see extreme differences. Males accounted for 96% of U.S. mass shootings and 90% of homicide convictions.
I suspect that one reason so many people believe that groups differ much more, on average, than they really do (and engage in dichotomizing and stereotyping) is that tail behavior is sometimes much more visible than typical behavior.
When you meet most people, you don’t really think about whether their compassion level is slightly above average or slightly below average (and then correlate it with sex). You just wouldn’t even notice one way or the other.
But when you see that the vast majority of serial killers are male, that stands out.
Most males are not very low in compassion. But most people who are very low in compassion are males!
For instance, ~4x more males than females have psychopathy/sociopathy.
Suomeksi (In Finnish)
Paljon puhetta tyhjästä – Tekoäly ja tunteet, vieraana Kaj Sotala.
Olin vieraana Paljon puhetta tyhjästä -podcastissa, kolmena isona pääteemana tekoälyn uhkakuvat, tekoälyn mahdollisuudet, sekä mielen rakenne ja toiminta. Ei tullutkaan kuin neljän tunnin keskustelu.
Karkea sisällysluettelo:
0:00:00 – Intro
0:04:14 – Transhumanismi, teknologinen singulariteetti, tekoälyn uhkakuvat
2:06:03 – Tekoälyn myönteiset mahdollisuudet
2:46:40 – Mielen rakenne ja toiminta
(Käytänpä mä paljon “silleen” -sanaa.)
Sosiaaliturvaleikkausten vaikutukset työnteon kannustimiin.
Vuonna 2024 voimaan tulevat sosiaaliturva- ja veromuutokset muuttavat melko paljon Kelan työttömyysetuuksia saavien kotitalouksien käytettävissä olevia tuloja ja työnteon rahallisia kannustimia. Esimerkkitalouksia koskevan tarkastelumme perusteella kannustimet työskentelyyn pienillä palkoilla tai osa-aikaisesti heikkenevät. Kun toimeentulotuki huomioidaan, kannustimet voivat heikentyä jopa yllättävän suurilla, yli mediaanitulon palkoilla.
The description of traffic in Hanoi reminds me of John Lilly describing driving as American yoga, only ever so much more so.