ch4

Superintelligence will either emerge quickly via strategic dominance or as a result of long collaborative efforts.

Example: Consider the Manhattan Project, the group that developed the atom bomb. The group’s activities were kept secret because the U.S. government feared that the USSR would use their research to build nuclear weapons of their own.

If SI developed in isolation would have a strategic advantage over all others.

Also prone to danger of falling into nefarious hands and be used as a weapon of mass destruction. Or suffer malfunctioned and try o do something terrible – kill all humans, say

One developed with collaboration of multiple groups of scientists, sharing advances in technology, humankind would be build gradually but is more likely to be subjected to being checked every step of the process, ensuring that the best choices have been made.

Example: A good precedent for such collaboration is the Human Genome Project, an effort that brought together scientists from multiple countries to map human DNA. Another good technique would be public oversight – instating government safety regulations and funding stipulations that deter scientists from working independently.

So, while the rapid development of a single SI could still occur during such a slow collaborative process, an open team effort would be more likely to have safety protocols in place.