Machine Intelligence Research Institute

In the future, the world might want international agreements to govern AI development. Such agreements are likely to include provisions about what AI chips can and cannot be used for (e.g. no undeclared models are in deployment), and verification mechanisms to ensure every side is following the rules. One proposed method for this chip use verification is network taps.

A defensive military coalition is a key frame for thinking about our international agreement aimed at forestalling the development of superintelligence. We introduce historical examples of former rivals or enemies forming defensive coalitions in response to an urgent mutual threat, and detail key aspects of our proposal which are analogous. We also contrast our proposal with the oft-employed analogy to nonproliferation agreements, and highlight shortcomings.

(Update Jan. 12: We released an FAQ last month, with more details. Last updated Jan. 7.) (Update Jan. 19: We now have an example of a successful partial run, which you can use to inform how you do your runs. Details.) We at MIRI are soliciting help with an AI-alignment project centered around building a dataset, described below.

The following is a partially redacted and lightly edited transcript of a chat conversation about AGI between Eliezer Yudkowsky and a set of invitees in early September 2021. By default, all other participants are anonymized as “Anonymous”.

Edward Kmett Edward Kmett is a prominent Haskell developer known for popularizing the use of lenses in functional programming. Edward maintains a significant chunk of all libraries around the Haskell core libraries, covering everything from automatic differentiation to category theory to graphics, SAT bindings, RCU schemes, tools for writing compilers, and more.