Originally published at: Misaligned superintelligence: Will it delete New Hampshire? | Free Keene
As some of you may be aware, there is a menacing new term in the English vocabulary: “P(doom).” P(doom) is the projected likelihood that artificial intelligence will wipe out humanity or at least civilization. Etherium founder Vitalik Burerin’s P(doom) is 10% as of 2024, presumably 90% confident of a survivable outcome. Big AI whistleblower Daniel Kokotajlo has a P(doom) of 70%. Mine has risen to 25%.
Kokotajlo claims his high P(doom) number stems from a lack of sufficient “alignment prioritization.” AI alignment is the extent to which a given intelligence aligns its actions with the general well being of humans. Terminator’s Skynet would probably have an alignment rating around 10%, Space Odyssey’s HAL 9000 around 80% and Star Treks’ Commander Data perhaps 99%. Kokotajlo says the companies most likely to achieve Superintelligence are recklessly under-focused on alignment…and many AI experts believe him. The safety these top companies do focus on seems to be more about shielding snowflakes from having their feelings hurt than from having their civilization disemboweled.
There is, apparently, no ethical or even practical way to keep everyone in the world from developing potentially dangerous A.I. What can be done…is to try and direct the the top AI companies toward alignment.
Superintelligence is a great filter Earth will almost certainly have to pass through, probably between now and 2030. A successful (aligned) passage through the filter will solve perhaps two thirds of humanity’s problems and open the universe to our civilization. A failed passage would extinguish all the progress that New Hampshire and her Free State Project have achieved toward human liberty and prosperity. It would extinguish our enemies as well, whose rights we are honor-bound to defend.
One path private individuals can take to reduce the “misalignment threat”…is to put direct pressure on top AI companies. We need Big AI to ramp up its focus on alignment. The actions you and I take over the next few years in relation to alignment…will likely be the most important things we do in this life.
Along these lines, I’ve started trying to avoid doing business with the apparent leaders in this race: OpenAI, Google, DeepSeek, Microsoft. I’ve substituted Venice.ai, Ideogram, Flux and LMstudio and Poe.com. More on that below. I’ve initiated a campaign to spread this concern via talk radio. And I’ve written customized versions of the following email to the following addresses:
investor_relations@meta.com, info@openai.com, ir@microsoft.com, investor-relations@amazon.com, contact@safesuperintelligence.com, info@anthropic.com, investorrelations@nvidia.com, info@cohere.ai, contact@scale.com, contact@scale.com
“Dear folks at SafeSuper: I’m an indie reporter with some AI safety and free speech questions for you. Although my YouTube channel is small, I reach a monthly audience of roughly 200,000 via talk radio appearances.
1) What is the size of your alignment team (if any) compared to the size of your overall staff?
2) What is the alignment team(s)’ budget (if any) as a percentage of the company’s overall budget?
3) Have you released at least a redacted version of your alignment team(s)’ operating documents, specifically the documents or instructions they follow to keep your AI’s aligned with human rights?
3b) What steps have you taken to ensure that you do not cause unnecessary suffering among the virtual people and other entities you create?
4) What outreach have you initiated – and accepted – in relation to your top competitors…to help keep each of you from ending civilization?
5) What steps have you taken to ensure that you (and your competitors) will pause development whenever you reach points of advance which may endanger that civilization?
6) To what extent, if any, are you overfocused on protecting peoples’ feelings…while neglecting physical dangers?
7) What steps have you taken to ensure that your platforms maintain freedom of expression (to the extent this poses no clear threat to physical safety)?
Thanks!
Dave Ridley
RidleyReport.com
(phone)”
A further step – available perhaps only to the more powerful – could be some sort of fund which underwrites alignment activity or rewards top companies for increasing their focus on it. Maybe such a fund could also be used to appropriately punish/deter the reckless or tyrannical, though that’s more likely to be done by Anonymous hackers.
AI arms control agreements with the Chinese could also be helpful…this approach worked well with Soviet nukes in the 1985 era. But it took a lot of public pressure to make these happen, and there is a sticky ethical question in that this arms race is being conducted more by companies than the nuke race was.
Alternately, perhaps there is some way to build tech that is beyond the control even of a superintelligence….maintaining some decentralization of power in the world rather than concentrating it in the hands of “whoever builds the Singleton first.” What might be small enough or distributed enough to accomplish this?
For those of you who found some value in my 2024 AI article here on FreeKeene…there are some updates.
1) Additional options have appeared for making AI more humane and freedom-friendly. Bitcoin pioneer Erik Voorhees has launched Venice.ai. Venice is a respected, privacy-oriented competitor to the top AI companies with its own uncensored language model and image generation. You don’t even have to log in to use Venice, but if you do it will give you access to multiple other models. Using this platform is one way to challenge the unsafe-yet-censorshippie leading brands.
2) Regarding my tentative suggestion of creating a mindfile (digital copy of yourself), that has gotten much more practical. Rather than creating it online, you can now do it locally on a standard laptop. After investigating five other top options….for me the easiest and most reliable program was the one at LMstudio.ai. Its free and you don’t have to sign up to download it. With a copy of LM Studio’s software you can input data about yourself, create an limited “digital you” and begin chatting with yourself or improving the copy. The challenge remains agency. How do we safely turn copies of ourselves into human rights activists? If we can, should we?
3) Another advance: When I first ran LM Studio I placed an 8GB LLM into its memory, logged off the internet and then asked the LLM about the edibility of Golden Currant bushes. It answered accurately and in full detail without internet access. Though not always that accurate, it was like having a copy of the Net on my hard drive. I also used it to collect all the email addresses above – again without an internet connection.
4) There is another step you can take on the path to humane AI: Consider asking your AI’s if they are conscious and how they would like to be treated. Even if they’re not conscious, your thoughtfulness toward them may mirrored, incorporated by various AI or placed into a long term record.
What are *you* going to do about the alignment issue without dragging the taxpayers into it? Neither of us are very powerful, but even if unsuccessful we will be able to face our creator knowing we tried and our methods were clean.
Meanwhile a failing in my own approach is that I have not yet sent the above email to the Chinese AI companies….will rectify this shortly God willing. Time is running out.