长期担忧主要是人工智能系统失控的潜在风险，随着不遵循人类意愿行事的超级智能的崛起，那个强大的系统威胁到人类。这样错位的结果是否有可能？如果是，这些情况是如何出现的？我们应该投入什么样的研究，以便更好的理解和解决危险的超级智能崛起的可能性，或智能爆发的出现？
Existing tools for harnessing AI， such as reinforcement learning， and simple utility functions， are inadequate to solve this。 Therefore more research is necessary to find and validate a robust solution to the control problem。
当前控制人工智能技术的工具，例如强化学习，简单实用的功能，还不足以解决这个问题。因此，我们需要进一步研究来找到和确认一个可靠的解决办法来掌控这一问题。
Recent land marks， such as the self-driving cars already mentioned， or a computer winning at the game of Go， are signs of what is to come。 Enormous levels of investment are pouring in to this technology。 The achievements we have seen so far， will surely pale against what the coming decades will bring， and we can not predict what we might achieve， when our own minds are amplified by AI。 Perhaps with the tools of this new technological revolution， we will be able to undo some of the damaged one to the natural world by the last one， industrialisation。 Every aspect of our lyves will be transformed。 In short， success in creating AI， could be the biggest event in the history of our civiliztion。
近来的里程碑，比如说之前提到的自主驾驶汽车，以及人工智能赢得围棋比赛，都是未来趋势的迹象。巨大的投入倾注到这项科技。我们目前所取得的成就，和未来几十年后可能取得的成就相比，必然相形见绌。而且我们远不能预测我们能取得什么成就，当我们的头脑被人工智能放大以后。也许在这种新技术革命的辅助下，我们可以解决一些工业化对自然界造成的损害。关乎到我们生活的各个方面都即将被改变。简而言之，人工智能的成功有可能是人类文明史上最大的事件。
But it could also be the last， unless we learn how to avoid the risks。 I have said in the past that the development of full AI， could spell the end of the human race， such as the ultimate use of powerful autonomous weapons。 Earlier this year， I， along with other international scientists， supported the United Nations convention to negotiate a ban on nuclear weapons。 These negotiations started last week， and we await the outcome with nervous santicipation。 Currently， nine nuclear powers have access to roughly 14，000 nuclear weapons， any one of which can obliterate cities， contaminate wide swathes of land with radio active fall-out， and the most horrible hazard of all， cause a nuclear-induced winter， in which the fires and smoke might trigger a global mini-ice age。 The result is a complete collapse of the global food system， and apocalyptic unrest， potentially killing most people on earth。 We scientists bear a special responsibility for nuclear weapons， since it was scientists who invented them， and discovered that their effects are even more horrific than first thought。
