The AI Race: Can Humanity Agree Before it Loses Control?

By Inuwa Waya From Geoffrey Hinton’s and Anthropic’s warnings, the emerging artificial intelligence race demands a new international system of cooperation, restraint and verification.

There are moments in history when humanity develops a technology before fully understanding the consequences of possessing it. Nuclear weapons were one. Artificial intelligence may prove to be another.

But there is a fundamental difference. Nuclear weapons possess extraordinary destructive power, but they do not think. They cannot reason about the intentions of their controllers, conceal their actions, rewrite computer code, independently search for vulnerabilities or devise strategies to prevent themselves from being disabled.

Future artificial intelligence may eventually possess some combination of these capabilities. Reports about increasingly autonomous AI systems have already raised concerns about their ability to interact with computer systems and potentially operate beyond the immediate direction of their users.

Read Also:

Wali proposes new model to stabilise Nigeria’s petrol prices

This has moved the AI debate beyond employment, productivity, misinformation and economic disruption. A more disturbing question has entered mainstream scientific discussion: Could humanity create an uncontrollable intelligence more capable and more powerful than itself?

What makes the debate particularly significant is the position of scientists who played leading roles in the emergence of modern AI.

Anthropic, the developer of Claude, is one of the world’s leading frontier-AI laboratories and has placed considerable emphasis on AI safety and alignment. Yet researchers associated with the company have expressed serious concerns about the direction in which frontier AI is moving.

Advert

Jacob Coxon, who worked on AI pre-training research at Anthropic and previously at OpenAI, warned that leading laboratories were racing towards increasingly autonomous and potentially self-improving AI before humanity had solved the problem of controlling systems more capable than itself.

More strikingly, Evan Hubinger, Anthropic’s Alignment Science Lead, has placed his personal estimate of the probability of AI causing human extinction within roughly the next decade at greater than 10 per cent.

Samuel Marks, Anthropic’s Scalable Oversight Lead, has also publicly emphasised the seriousness with which catastrophic AI risks are regarded by researchers working closest to frontier systems.

None of this means Anthropic believes extinction is inevitable, nor does it mean that a 10 per cent estimate can be scientifically measured with precision. It means something more limited, but still extraordinary: people whose professional work involves understanding and controlling frontier AI systems cannot confidently rule out catastrophic loss of control.

That deserves serious attention.

Geoffrey Hinton is not a speculator about a technology he does not understand. His pioneering work on neural networks helped establish the foundations of modern AI. Together with Yoshua Bengio and Yann LeCun, he received the Turing Award for foundational contributions to deep learning. He subsequently received the Nobel Prize in Physics.

In 2023, Hinton left Google after more than a decade with the company. He said the decision gave him greater freedom to speak about the dangers he believed advanced AI could create.

One of the architects of the AI revolution had become sufficiently concerned about where the technology might lead that he wanted greater freedom to warn society about it.

Responding to a question about whether a greater-than-10-per-cent probability that AI could kill humanity within approximately a decade was plausible, Hinton said such an estimate was not unreasonable, while acknowledging the difficulty of calculating such probabilities.

His caveat is important. But so is the warning itself. It indicates how urgently some leading researchers believe the governance challenges surrounding advanced AI must be addressed.

Related:

Waya Hails Governor Yusuf’s Reforms, Calls for End to Political Thuggery in Kano

Kano Assembly passes bill to regulate private schools, curb arbitrary fee hikes

FCCPC Scandal Deepens as Ex-Director ‘Stays On’ Illegally

Yet the scientific community is far from unanimous on the possibility of human extinction because of advanced AI.

Yann LeCun, another giant of modern AI and Hinton’s fellow Turing Award recipient, strongly disputes many extinction scenarios.

One of LeCun’s central arguments is that intelligence should not be confused with motivation. Becoming extraordinarily intelligent does not logically mean acquiring a desire to dominate humanity. Human drives towards power, status and survival have biological and evolutionary origins; there is no established law requiring an artificial intelligence to acquire equivalent ambitions simply because it becomes highly capable.

For LeCun, fears of AI taking over the world amount partly to humans projecting their own evolutionary traits of power and dominance onto machines. He has argued that if an AI system becomes unsafe, humans can change its design or turn it off. Human beings, he maintains, should remain in charge.

Andrew Ng has similarly argued that extinction scenarios are excessively speculative and that disproportionate fear could encourage regulation that unnecessarily inhibits beneficial technological development. He has also argued that some regulations championed by large technology companies could disadvantage smaller competitors and the open-source community.

Other researchers emphasise the limitations of today’s AI. Current systems hallucinate, make elementary errors, struggle with aspects of sustained reasoning and remain dependent on enormous physical infrastructures built, powered and maintained by humans.

These arguments deserve consideration.

Scientists who disagree with Hinton should be part of any international effort to govern frontier AI. But their disagreement does not establish that catastrophic AI risk is impossible. It establishes something more uncomfortable: uncertainty.

When Hinton and LeCun, who shared the Turing Award for their contributions to deep learning, can examine the trajectory of the same technology and reach radically different conclusions about its ultimate danger, policymakers cannot responsibly pretend that the scientific question has been settled.

Suppose Hinton and the Anthropic researchers are wrong. Humanity might impose unnecessary safety tests, spend enormous resources on verification and slow some frontier development.

Conversely, suppose they are right to a large extent. The world may not have the opportunity to correct its mistakes.

That asymmetry lies at the heart of the policy problem.

If aeronautical engineers disagreed over whether an aircraft had a 10 per cent probability of crashing, nobody would resolve the dispute by filling the aircraft with passengers and taking off. The rational response would be to investigate before take-off.

The response to the AI conundrum should therefore be neither panic nor technological prohibition. It should be precaution proportionate to consequence.

The crucial question is which international body should oversee such precautionary measures and whether scientists and governments can agree on what risks should trigger intervention.

The competition among countries to develop artificial intelligence is intense and is often likened to an arms race or a new Cold War. World leaders increasingly view AI as a technology capable of shaping future economic leadership, national security and global influence.

The United States and China are the leading competitors in the global AI race, although Europe, India, Singapore, Japan, South Korea, Canada, Britain and countries in the Middle East are also developing significant capabilities.

The current competition is not unprecedented.

The closest historical analogy is the nuclear race. After Hiroshima and Nagasaki came the Soviet atomic bomb, the hydrogen bomb, intercontinental ballistic missiles and submarine-launched nuclear weapons.

The United States and the Soviet Union became locked in a security dilemma. Each increased its arsenal partly because it feared the other’s capabilities.

Then came the Cuban Missile Crisis, demonstrating how unrestricted strategic competition could produce consequences in which neither side could claim meaningful victory.

Out of that recognition gradually emerged arms control.

The Strategic Arms Limitation Talks, known as SALT, began formally in 1969. SALT I produced the 1972 Anti-Ballistic Missile Treaty and an interim agreement limiting aspects of strategic offensive weapons.

SALT did not end the Cold War or eliminate nuclear weapons. It did something more practical: it established boundaries around competition.

The Strategic Arms Reduction Treaty, or START, subsequently went further, requiring substantial reductions in strategic arsenals and establishing verification arrangements involving data exchanges, notifications, technical monitoring and on-site inspections.

America did not trust the Soviet Union, and vice versa. Where trust was impossible, verification could make limited restraint possible.

The SALT/START analogy, however, does not entirely fit AI.

The most consequential strategic competition in frontier AI is presently between the United States and China. Their companies, computing infrastructure, semiconductor strategies and military ambitions make their relationship central to the global AI race.

If Washington believes Beijing is approaching transformative AI, America has an incentive to accelerate. If Beijing believes Washington is approaching it, China has the same incentive.

Neither wants to finish second.

That makes negotiations between the US and China essential, although achieving them will be difficult given the intensity of their strategic competition.

In rejecting warnings about the possibility of AI taking over the world and wiping out humanity, President Donald Trump has described such assertions as a hoax and argued that AI and data centres in the United States represent the “Greatest Economic Development Engine in History which should not be stopped.”

But even if Washington and Beijing agreed to negotiate limitations, that alone would not be enough.

Significant AI research and infrastructure exist in Europe, Britain, India, Japan, South Korea, Canada, the Middle East, Singapore and elsewhere. Future breakthroughs therefore need not originate in either America or China.

There is another complication: unlike nuclear weapons, much of frontier AI is not controlled directly by governments.

Corporations and research laboratories are central to its development. Companies such as OpenAI, Anthropic, Google DeepMind and Meta are developing frontier systems in the United States and allied economies, while China has its own major technology companies and AI laboratories.

That fundamentally changes the governance structure.

A treaty constraining Washington while leaving frontier American laboratories outside its obligations would be inadequate. The same would apply to Beijing and Chinese developers.

An agreement between America and China that ignored significant capabilities elsewhere could also become ineffective.

The practical solution should therefore have different stages, with a bilateral approach serving as Stage One.

Washington and Beijing could begin negotiations for an AI Strategic Stability Agreement because their rivalry creates the possibility of a frontier race in which safety becomes subordinate to speed. Such negotiations should involve independent AI companies and researchers in their respective countries.

A first agreement could prohibit autonomous AI authority over nuclear launches; guarantee meaningful human control over irreversible strategic military decisions; establish emergency AI hotlines; create common reporting obligations for serious loss-of-control incidents; and establish agreed safety evaluations for systems crossing defined capability thresholds.

The next stage should widen participation.

Other states possessing significant frontier AI capabilities, advanced semiconductor industries or very large computing infrastructure should enter the framework.

Ultimately, the objective should be a universally recognised and acceptable control regime.

This may sound simple on paper. In reality, judging from the difficulties the international community already experiences in enforcing international law and reaching consensus on major security issues, establishing a body responsible for supervising frontier AI systems and developing an international verification mechanism would be a herculean task.

The purpose of international governance and monitoring is not to determine which government or company has good intentions. It is to create rules under which responsible behaviour does not require competitive suicide.

No one knows whether AI could cause human extinction within a decade.

We should resist converting uncertain probability estimates into sensational predictions. We should equally resist converting uncertainty into complacency.

The next ten years should therefore not necessarily be described as humanity’s countdown. They should be treated as a window for governance.

Artificial intelligence could become one of humanity’s greatest achievements. It could transform medicine, education, scientific discovery and productivity. It could help solve problems that generations of human beings have failed to solve.

The objective cannot therefore be to stop technological progress. It must be to ensure that progress remains compatible with human survival.

The international community must put aside its differences along fault lines and open channels for cooperation. SALT and START demonstrated that adversaries could negotiate limits on technologies that both regarded as vital to national security.

AI may require an even more complicated model because governments, corporations and research laboratories all play major roles in its development.

No government, corporation or scientist has the moral right to gamble with humanity’s continued existence simply because nobody can calculate the odds precisely.

If Hinton and the Anthropic researchers are wrong, history may conclude that humanity was excessively cautious. If they are right and humanity ignores them, there may be nobody left to write history.

There is, however, another issue that deserves attention even before the debate over AI extinction scenarios is settled: the potential consequences of excessive human reliance on AI.

Humans increasingly use AI for analytical thinking and consult it on personal and official matters. Overdependence on these systems could have consequences for human cognitive abilities and decision-making, although the long-term effects remain uncertain.

The world therefore needs to consider not only whether machines could eventually become too intelligent for humans to control, but also whether humans could become so dependent on machines that they gradually surrender some of their own capacity for independent thought.

The big question today is whether humanity is prepared to control an intelligence greater than its own before it becomes impossible to govern.

Inuwa Waya is a Nigerian oil and gas expert and former governorship aspirant from Kano State. He writes from Kano and sent this article to ‘The Historica Nigeria’.