Now, this is often caricatured, as I have here, as a fear that armies of malicious robots will attack us.现在我们时常会看到这样一些讽刺漫画，我们总会担心受到一些不怀好意的机器人军队的攻击。But that isn&#39t the most likely scenario.但这不是最可能出现的事情。It&#39s not that our machines will become spontaneously malevolent.我们的机器不会自动变得邪恶。The concern is r

It&#39s as though we stand before two doors. Behind door number one, we stop making progress in building intelligent machines.我们就像站在了两扇门前。在第一扇门后面，我们停下打造智能机器的脚步。Our computer hardware and software just stops getting better for some reason.某些原因也使我们停止了对电脑软件和硬件的升级。Now take a moment to consider why this m

Another reason we&#39re told not to worry is that these machines can&#39t help but share our values because they will be literally extensions of ourselves.另外一个试图安慰我们的理由是，那些机器必须拥有和我们一样的价值观，因为它们将会是我们自身的延伸。They&#39ll be grafted onto our brains, and we&#39ll essentially become their limbic systems.它们将会被

So what would apes like ourselves do in this circumstance?那在这种情况下，像我们这样的“大猩猩”还能有什么用呢？Well, we&#39d be free to play Frisbee and give each other massages.我们可以悠闲地玩飞盘，给彼此做按摩。Add some LSD and some questionable wardrobe choices, and the whole world could be like Burning Man.服用一些迷药，穿一些奇装异服，整个世界都沉浸在狂欢节之中。No

No one seems to notice that referencing the time horizon is a total non sequitur.似乎没有人注意到以时间作为参考系是得不出合理的结论的。If intelligence is just a matter of information processing,如果说智慧只包括信息处理，and we continue to improve our machines, we will produce some form of superintelligence.然后我们继续改善这些机器，那么我们终将生产出超级智能。And w

Sorry, a chicken. There&#39s no reason for me to make this talk more depressing than it needs to be.抱歉，母鸡的位置应该在这。这个演讲已经够严肃了，开个玩笑轻松一下。It seems overwhelmingly likely, however, that the spectrum of intelligence extends much further than we currently conceive,然而，很可能的情况是，智慧谱线上的内容已远远超出了我们的认知，and if we bui

It&#39s crucial to realize that the rate of progress doesn&#39t matter, because any progress is enough to get us into the end zone.我们要知道这个进程的速度并不重要，因为任何进程都足够让我们走进死胡同。We don&#39t need Moore&#39s law to continue. We don&#39t need exponential progress. We just need to keep going.甚至不需要考虑摩尔定律，也不需要用指数函数来衡

I&#39m going to talk about a failure of intuition that many of us suffer from.我想谈论一种我们很多人都经历过的来自于直觉上的失误。It&#39s really a failure to detect a certain kind of danger.它让人们无法察觉到一种特定危险的存在。I&#39m going to describe a scenario that I think is both terrifying and likely to occur, and that&#39s not a good com