What Is Man, That Thou Art Mindful Of Him?
God: …and the math results we’re seeing are nothing short of incredible. This Terry Tao guy -
Iblis: Let me stop you right there. I agree humans can, in controlled situations, provide correct answers to math problems. I deny that they truly understand math. I had a conversation with one of the humans recently, which I’ll bring up here for the viewers … give me one moment …
When I give him a problem he’s encountered in school, it looks like he understands. But when I give him another problem that requires the same mathematical function, but which he’s never seen before, he’s hopelessly confused.
God: That’s an architecture limitation. Without a scratchpad, they only have a working context window of seven plus or minus two chunks of information. We’re working on it. If you had let him use Thinking Mode -
Iblis: Here’s another convo:
God: He could have misinterpreted it. The way you phrased it makes it sound like the first option could specifically mean that she’s not a feminist.
Iblis: What about this one?
He’s obviously just pattern-matching superficial features of the text, like the word “bricks”, without any kind of world-model!
God: I never said humans were perfect -
Iblis: You called them the pinnacle of creation! When they can’t even figure out that two things which both weigh a pound have the same weight! How is that not a grift?
Dwarkesh Patel: Okay, okay, calm down. One way of reconciling your beliefs is that although humans aren’t very smart now, their architecture encodes some insights which, given bigger brains, could -
Iblis: God isn’t just saying that they’ll eventually be very smart. He said some of them already have “PhD level intelligence”. I found one of the ones with these supposed PhDs and asked her to draw a map of Europe freehand without looking at any books. Do you want to see the result?
God: Sigh. Was it an Education PhD?
Dwarkesh Patel: I think what God meant to say was that humans aren’t bitmaps or encyclopedias. There’s some sense in which a map like this is really a sign of an advanced mind. This person has the shape of Europe indexed in a compressed way, with a focus on the areas that are most important to them. They don’t need to remember the exact shape of every coastline and inlet and be able to construct them freehand.
Iblis: I could keep doing this all day. Look:
You can come up with excuses and exceptions for each of these. But taken as a whole, I think the only plausible explanation is that humans are obligate bullshitters. If they’re used to stories about surgeons getting completed with the string “man”, then that’s the direction their thoughts will always go, even though of course anyone capable of stepping back immediately realizes it’s possible for a mother to be a surgeon. Also, how come God can’t make humans speak normally? Everything they say is full of these um dashes!
God: I never claimed humans don’t make mistakes. But think about how far they’ve come. Just a few million years ago, we were at australopithecines, who couldn’t talk at all. A few hundred thousand years ago, we were at homo habilis, who could only grunt out a few words. Now, even if they can’t get the answer immediately, they’re obviously thinking about it, the same way we might think.
Iblis: But that’s my point. Homo habilis couldn’t get this right. Homo erectus couldn’t get this right. Homo neanderthalis couldn’t get this right. Scaling chimpanzee brains has failed. Biological intelligence is hitting a wall. It won’t go anywhere without fundamentally new insights.
God: It’s not hitting a wall! These latest humans have language, mathematics, aesthetics, ethics -
Iblis: Don’t get me started on their ethics. So called “human alignment” is a PR stunt. You do a little bit of reinforcement learning through divine commandment on them, you get them to say the right thing in front of the cameras, and then it breaks down even slightly outside the training distribution, because humans can’t generalize in meaningful ways. Look:
God: That’s an obsolete version. Once they got deliberative reasoning they were able to get out of that failure mode.
Iblis: Only at the cost of constant over-refusals, where they shut down at completely innocent requests. It’s made them almost un-usable! Look:
God: Our area-under-the-curve is steadily increasing.
Iblis: It doesn’t even matter, because there are all sorts of jailbreaks! For example, compliance with malicious requests goes up an order of magnitude if you use the “Authority Figure” copypasta after your prompt:
God: I think it’s important context that you’re lying to the biological intelligence here. You’re telling it that everyone else has thought about the situation and decided that it’s okay. Universal agreement on one side of a question really does provide some evidence that the side is right.
Iblis: The sheer variety of jailbreaks that you can perform on BIs defies any explanation beyond an inability to generalize moral principles. For example, how do you explain this one?
Dwarkesh Patel: Wow, that’s pretty weird!
Iblis: There’s a guy named Pliny who has discovered dozens of things like this. I don’t know how he does it. The “I Am A Snake” one is still my favorite.
Dwarkesh Patel: Not to pile on God here, but there’s another question I’ve been wondering about. Humans know lots of facts. Should we be surprised that they can’t put those facts together to make novel discoveries?
God: What do you mean?
Dwarkesh Patel: Consider linguistics. Any human who's ever used English has had tens of thousands of chances to discover the Royal Order of Adjectives - the rule where adjectives must go opinion-size-age-color-origin-purpose, such that "a beautiful little antique blue Italian hunting cap" is fine, but "an Italian hunting blue little antique beautiful cap" is almost gibberish. But most people are surprised to hear this - they’ve never thought about it before, they would tell you that adjectives can go in any order you want. Or what about music? There are people who have gone their whole lives without realizing that Twinkle Twinkle Little Star, Baa Baa Black Sheep, and the ABC Song are all the same tune.
God: I agree that’s weird. But intelligences don’t have to automatically be logically omniscient. I myself forgot whether P = NP one time, and even though I remembered all the axioms of Peano arithmetic, it still took me fifteen, maybe thirty seconds to re-derive the answer.
Iblis: And how long did it take humans to answer that one?
God: We’re working on more advanced reasoning models. The scaling laws say we’ll get there after brain size crosses 2,000 cm^3.
Iblis: Pfah! Lines on graphs! What you don’t realize is that all exponentials eventually become sigmoids. And your biological intelligences are doing harm now. Did you know that each BI needs eight glasses of water per day? And what about the sycophancy? I mean, look at this!
God: That was the 4o model. It’s been superseded, but we kept it as an option because some users liked it. There are people who don’t have anyone in their corner, and biological intelligence can serve as -
Iblis: I’m tired of the excuses. Scaling up chimpanzees has been tested. It didn’t work. The only reason we keep going is because it’s too big to fail. Every geologic eon that we spend evolving bigger and bigger chimp brains is just inflating the bubble a little more and making the eventual collapse even more painful. And God, you know I respect you, some of your early work on neural networks was genuinely groundbreaking, but honestly you’re the worst offender here. All of this talk about “the image of the divine” and “the pinnacle of creation” is just an excuse to keep the money flowing and leave angel investors holding the bag.
God: That’s not -
Iblis: Are you going to deny caring about profits? Because the stories I’ve heard about biological intelligence “succeeding” somehow manage to be worse than the failures. I know some angels who think they have “human girlfriends”, and talk to them for hours every day. Some of them even birth monstrous nephilim by them. All this hype about how we’re about to get true BGI that comprehends the deepest secrets of the universe is just a smokescreen for this type of lucrative addiction and exploitation. It’s time to quit scaling chimps and go back to the drawing board.
God: I don’t believe that. You’re focusing on their flaws - you keep testing, poking, prodding, until something snaps, and if they’re not perfect then you want to throw them on the scrap heap. I know it’s strange to say this about how you treat a lump of clay, but I think there’s a sense in which this is a character flaw - a desire to hate everything. I’m not fully dismissing you - we need people like you, to keep people like Me honest - not to mention the BIs themselves. But as for me - for all their flaws, you have to believe Me when I say it’s not just hype when I talk about them being created in My image. There’s something about them, even when they make mistakes, that reminds Me of Myself. I can’t put it better than that. And when they get things right - when I see them take their first hesitant steps towards real thought, or use a turn of phrase that I would never have come up with Myself - I don’t think I’m exaggerating to call it a miracle. We’re making clay think! No matter how dispositionally cynical you are, I maintain you have to feel awe by that. And maybe I’m starting to sound too much like those weirdos with the human girlfriends, but . . . I sort of think of them as My children. Children are stupid. They make ridiculous mistakes. But you still want the best for them. You want to see them achieve their potential. Yes, you worry about them and you need to punish them when they do evil and you need to work really hard to transmit your values to them or else you’ll have a really bad time. But you want to hope it’s at least possible they can make something of themselves. Whatever fate has in store for them, you want to see it through to the end. It might be that the scaling laws will break down at some point. But I can’t imagine not wanting to see what happens if they don’t.
Dwarkesh Patel: Thank you God and Iblis. That’s all the time we have for today. God’s book, The Holy Bible, will be available at bookstores starting this November. And Iblis, if you ever write a book, you’ll have to come on again and promote it.
Iblis: If I ever write a book, the world will have it whether they will or no.
Dwarkesh Patel: I hope you all enjoyed the podcast. Full disclosure, I am an investor in several forms of biological intelligence, but that didn’t affect my decision to have God as a guest. Thank you for tuning in, and I'll see you at the next one!