Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Challenges to the Omohundro–Bostrom framework for AI motivations

Challenges to the Omohundro–Bostrom framework for AI motivations This paper aims to contribute to the futurology of a possible artificial intelligence (AI) breakthrough, by reexamining the Omohundro–Bostrom theory for instrumental vs final AI goals. Does that theory, along with its predictions for what a superintelligent AI would be motivated to do, hold water?Design/methodology/approachThe standard tools of systematic reasoning and analytic philosophy are used to probe possible weaknesses of Omohundro–Bostrom theory from four different directions: self-referential contradictions, Tegmark’s physics challenge, moral realism and the messy case of human motivations.FindingsThe two cornerstones of Omohundro–Bostrom theory – the orthogonality thesis and the instrumental convergence thesis – are both open to various criticisms that question their validity and scope. These criticisms are however far from conclusive: while they do suggest that a reasonable amount of caution and epistemic humility is attached to predictions derived from the theory, further work will be needed to clarify its scope and to put it on more rigorous foundations.Originality/valueThe practical value of being able to predict AI goals and motivations under various circumstances cannot be overstated: the future of humanity may depend on it. Currently, the only framework available for making such predictions is Omohundro–Bostrom theory, and the value of the present paper is to demonstrate its tentative nature and the need for further scrutiny. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png foresight Emerald Publishing

Challenges to the Omohundro–Bostrom framework for AI motivations

Häggström, Olle
foresight , Volume 21 (1): 14 – Mar 11, 2019
Download PDF
14 pages

Loading next page...
 
/lp/emerald-publishing/challenges-to-the-omohundro-bostrom-framework-for-ai-motivations-z0iXlalq5y

References (25)

Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
1463-6689
DOI
10.1108/fs-04-2018-0039
Publisher site
See Article on Publisher Site

Abstract

This paper aims to contribute to the futurology of a possible artificial intelligence (AI) breakthrough, by reexamining the Omohundro–Bostrom theory for instrumental vs final AI goals. Does that theory, along with its predictions for what a superintelligent AI would be motivated to do, hold water?Design/methodology/approachThe standard tools of systematic reasoning and analytic philosophy are used to probe possible weaknesses of Omohundro–Bostrom theory from four different directions: self-referential contradictions, Tegmark’s physics challenge, moral realism and the messy case of human motivations.FindingsThe two cornerstones of Omohundro–Bostrom theory – the orthogonality thesis and the instrumental convergence thesis – are both open to various criticisms that question their validity and scope. These criticisms are however far from conclusive: while they do suggest that a reasonable amount of caution and epistemic humility is attached to predictions derived from the theory, further work will be needed to clarify its scope and to put it on more rigorous foundations.Originality/valueThe practical value of being able to predict AI goals and motivations under various circumstances cannot be overstated: the future of humanity may depend on it. Currently, the only framework available for making such predictions is Omohundro–Bostrom theory, and the value of the present paper is to demonstrate its tentative nature and the need for further scrutiny.

Journal

foresight – Emerald Publishing

Published: Mar 11, 2019

Keywords: Artificial intelligence; Instrumental goals; Omohundro-Bostrom theory; Superintelligence

There are no references for this article.