Time is the Fifth Dimension of Utility

Say you were offered a choice between these two possible universe trajectories:

A) a 100% chance of 10 billion beings living at each point for the next billion years

B) a 25% chance of 100 billion beings living at each point for the next billion years, and a 75% chance of 1 billion beings

Which do you choose?

Continue reading

The Summoning


Found on the whiteboard of the Computer Safety Research Group.

Berkeley, California

Kobe and Hunter sat hunched over a laptop in Kobe’s living room.  Black and white stones slowly popped up on the rectangular grid; the commentators tried to grasp the mind of the masters.

A black stone appeared, and Sin furred his brow.  It was man vs machine, and game one wasn’t looking so good for man.

“Go go BetaGo!”  shouted Kobe.

Hunter cocked his head to the side.  “Remember how we’ve spent the last year of our lives trying to avoid scenarios where an AI inadvertently destroys the world?”

Kobe looked down and mumbled something about sports.  They sat in a somewhat awkward silence for the next ten minutes, watching Sin’s carefully designed structure crumble.

After the game, Kobe and Hunter headed over to the offices of the Computer Safety Research Group (CSRG) for the weekly strategy meeting.  The topic of discussion for the day: what to do when the Big Brain Countdown reaches 11pm?

Their positions were mostly predictable.  Hunter, Svengali, and Clapperton though that if they could just deter Big Brain, the world would have enough time to come up with some better frameworks for AI.  Kobe, Wilbert, and Cherish were skeptical that Big Brain was really going to make much progress.  (Hunter was quick to remind them about the afternoon’s go match.)  Rushmore and Melanie thought that eventually, someone was going to try to summon Elua, and although Big Brain wouldn’t be the worst group to do it, why not CSRG?

Lumpkins stood up to speak and everyone quieted down.

“In case you guys haven’t forgotten, this is the computer safety research group.  Our goal is to make sure no one summons Moloch, and that’s exactly what’s going to happen if we rush into this.  Big Brain has to be stopped.  And if you don’t agree, then maybe this isn’t the organization for you.”

Continue reading