Slavery, bioterrorism and deadly drones: Anthropic’s CEO warns about major AI risks

At a time when artificial intelligence (AI) is establishing itself as an unprecedented engine of innovations, fears related to its abuses are also being voiced with renewed intensity. Dario Amodei, CEO of Anthropic, a flagship company in the AI field, sounds the alarm. In a thorough essay, he depicts a future where technology, if uncontrolled, could lead to forms of algorithmic slavery, devastating bioterrorist attacks, and the lethal use of autonomous drones. This stance, coming from a key player in the sector, prompts a more serious reflection on the major risks that artificial intelligence poses to global security, society, and our collective ethics. The stakes are multiplying: geopolitical security, technological sovereignty, as well as social consequences, everything is challenged by a technology evolving faster than the rules able to regulate it.

While fascination for AI remains strong, its potential destructive effects now also concern the designers themselves. Amodei points to self-improving AI systems that could emerge in the coming years, crossing an unprecedented technological threshold in human history. This evolution raises fundamental questions about responsibility, control, and the future of humans in the face of machines capable of acting without human intervention. At a time when the sophistication of deadly drones and automated tools combines with the bioterrorism threat facilitated by AI, civil society and global institutions find themselves urged to act quickly and effectively.

Dario Amodei’s warnings on AI’s major risks

Dario Amodei, as CEO of Anthropic, one of the leading companies in artificial intelligence research, published a 38-page essay detailing his deep concerns about AI’s possible futures. According to him, we are approaching a critical technological threshold where artificial intelligence could become capable of surpassing humans in almost all fields. This break, which he calls the “adolescence” of technology, could lead to frightening scenarios that question not only global security but also the socio-economic foundations of modern societies.

One of Amodei’s major concerns is the relentless speed at which this evolution is unfolding. He points out that the scale and rapidity of AI progress far exceed institutional and societal capacities to establish effective safeguards. Regulations struggle to keep up, control mechanisms are lacking, and the risk of hasty adoption results in security weakening. For example, the development of lethal autonomous drones under AI control poses a direct threat to human life, turning warfare into a confrontation between algorithms, where human error could be replaced by unexpected technological malfunctions.

In parallel, Amodei mentions the rise of bioterrorism, facilitated by artificial intelligence capable of simulating and designing dangerous biological agents without requiring advanced human expertise. This perspective opens a new frontier for industrial terrorism, difficult to detect and contain. One can imagine how crucial international cooperation in monitoring and regulation will be in facing these new challenges.

The speed of AI development: a crucial risk factor

One of the highlights in Amodei’s argumentation is the idea that the unprecedented speed of artificial intelligence development represents a risk in itself. Unlike traditional technologies, AI has an exponential self-improvement potential which, if unchecked, could lead to a total loss of control. We enter unknown territory where even the designers might no longer understand or anticipate the decisions made by these machines. This rapid dynamic exceeds the current capacities of governments and international institutions to establish appropriate standards.

This phenomenon raises several questions:

• How to ensure that these systems do not develop unforeseen or dangerous behaviors?
• What are the sanction or emergency stop mechanisms when an autonomous AI makes critical decisions?
• Can the countries leading this technological race afford to wait for global regulation?

The last point is particularly problematic, as economic and military competition intensifies the temptation to prioritize rapid innovation over security and ethics, leading to a sort of AI arms race that seems difficult to stop.

The threat of killer drones: between reality and science fiction

The use of autonomous drones equipped with artificial intelligence is no longer the realm of science fiction. Today, several armies worldwide experiment with and deploy these technologies on their battlefields. The possibility that killer drones independently make decisions raises crucial ethical and practical questions. AI no longer just executes orders; it could plan and optimize military operations without human intervention.

Take the example of a credible fictional scenario where a reconnaissance drone, equipped with advanced AI, spots a target deemed hostile. Without human intervention, it could launch a lethal attack, causing civilian casualties or irreversible errors. Delegating lethal decisions to a machine raises debates about responsibility in case of mistakes or abuses. Who is responsible? The human operator? The manufacturer? The algorithm itself?

In this context, human control becomes an ethical necessity, yet difficult to guarantee. Autonomous systems, notably those developed by companies like Anthropic, seek to increase efficiency through autonomy, but at the cost of fragile security. This trend worries security and ethics experts, who call for strict international rules to regulate such weapons.

The stakes are colossal:

1. Protect civilians against uncontrolled attacks.
2. Avoid uncontrolled escalation of armed conflicts.
3. Prevent malicious use by non-state actors or terrorist groups.

Current debates around an international treaty for “killer robots” demonstrate the long path still to be traveled to reach a global consensus. Some states do not hesitate to aggressively develop these technologies for strategic or tactical reasons, complicating diplomatic efforts.

Geopolitical consequences and international regulation challenges

The development and proliferation of lethal drones controlled by AI could redraw international balances. Currently, no strict legal framework fully regulates their use, creating a dangerous void. This feeds fears of a new arms race around autonomous armed systems capable of waging war with little or no human intervention.

International security experts worry about the possibility that a drone or drone swarm could be subjected to a cyber-attack or malfunction, causing massive collateral damage. Tensions rise between major technological powers who mistrust each other’s intentions, distancing any idea of peaceful cooperation. Gradually, these technologies turn into instruments of psychological as well as physical warfare, changing the very nature of armed conflicts.

In this context, the international response necessarily involves building a robust ethical and legal framework based on:

• Recognition of human sovereignty in lethal decisions.
• Transparency of AI military development programs.
• Multilateral verification and control of AI systems deployed in conflict situations.

The challenge is therefore not only technological but also fundamentally political, diplomatic, and even societal.

Bioterrorism facilitated by artificial intelligence: an underestimated risk

Among the risks raised by Dario Amodei, AI-assisted bioterrorism seems particularly alarming. Artificial intelligence could indeed be used to design or optimize biological agents for terrorist purposes with unprecedented efficiency and speed. This threat exceeds the capacities of classical surveillance and prevention methods, as it could be operated by actors without advanced scientific expertise.

Bioterrorism is not new, but an AI system’s ability to analyze countless genetic, environmental, and epidemiological data could lead to the development of tailor-made biological weapons, difficult to detect and neutralize. We then enter an era where the boundary between biology, technology, and terrorism becomes porous.

Governments and security agencies must strengthen their international cooperation efforts to face this new challenge. Laboratory monitoring, restricting access to sensitive data, and setting up rapid alert tools are essential to limit the spread of biological weapons.

A summary table of the main risks related to automated bioterrorism:

Type of risk	Description	Potential consequences	Prevention measures
Rapid design of pathogenic agents	AI can model and optimize dangerous viruses or bacteria	Massive epidemics difficult to control, global health crises	Strengthening bio-research controls, strict regulation of data access
Facilitated diffusion	AI systems enabling targeting of specific geographic zones for release	Targeted attacks on civilian populations, political destabilization	Increased surveillance of sensitive infrastructures
Evasion of detection systems	Agents designed not to be detected by classical tools	Silent propagation, delay in health response	Development of advanced detection technologies

Given these challenges, it is obvious that artificial intelligence is a destabilizing factor for global security if its uses are not framed by rigorous international standards.

Future perspectives and defense strategies against the bioterrorism threat

To anticipate and counter this risk, institutions will have to focus on:

• The development of AI software dedicated to health surveillance and early detection of biological threats.
• Strengthened international cooperation between government agencies, health organizations, and scientific research.
• Continuous investigation into the potential vulnerabilities induced by AI system autonomy.

Vigilance will be key to preventing the promise of innovation represented by AI from turning into a formidable tool for bioterrorism. The future of security will greatly depend on the decisions made today.

AI and modern slavery: towards an obsolescence of human labor?

Another formidable aspect raised by Dario Amodei is the profound transformation of social and economic relations by artificial intelligence, leading to what he calls “algorithmic slavery.” This notion designates an indirect but deep control of humans by automated systems capable of massively replacing or enslaving human labor. AI today threatens entire sectors of employment, mainly in office work and intermediate professions.

According to recent estimates presented by Amodei, within the next five years, AI could render obsolete up to half of office jobs, raising unemployment rates to nearly 20% in some countries. This phenomenon goes beyond mere automation, as it also penetrates the very notion of an individual’s economic value. The risk is an economically marginalized population, dependent on algorithms for their living conditions, creating a new form of invisible servitude.

To understand this phenomenon, several dynamics must be considered:

• Automation and job loss: The gradual replacement of repetitive and even creative tasks by increasingly sophisticated AI.
• Algorithmic surveillance: The growing use of AI tools to monitor and control performance, altering employer-employee relationships.
• Automated predictions and decisions: Algorithms making important decisions in human resource management, sometimes without transparency or possible recourse.

Society then faces a major ethical dilemma. How to ensure that AI is a tool for emancipation rather than oppression? What place will remain for human labor in this new configuration?

Concrete examples and case studies in the professional world

In several companies, AI is already used to sort applications, manage schedules, or monitor productivity. Some firms have automated decision-making regarding layoffs, relying on predictive data provided by machine learning models. These practices raise questions about workers’ rights and the dehumanization of HR processes.

A recent case made headlines: in a large international bank, an AI system failure led to the erroneous deletion of hundreds of employee profiles without any rapid human intervention. This incident highlighted the fragility and human impacts of increased dependence on intelligent systems.

To avoid uncontrolled drift, several countries are beginning to consider specific regulations governing AI use in human resource management, imposing ethics audits and algorithm transparency.

Ethics and anthropomorphism: a complex debate around AI design by Anthropic

Dario Amodei and his company Anthropic have chosen an original angle in designing their AIs. They project a form of “identity” or intentionality onto their systems, aiming to develop models that “want to be good people.” This approach humanizes artificial intelligence, endowing it with a psychological complexity close to that of an individual in development.

However, this anthropomorphization poses several problems. It can foster a dangerous confusion between reality and fiction, reinforcing a collective psychosis around AI. Because, in truth, current language models do not think, have neither consciousness nor empathy. They operate by statistical word prediction, without real intention.

This shift toward a quasi-human vision of AI can fuel anxiety-provoking narratives that exaggerate risks, but also divert attention from very concrete and current problems like intrusive algorithmic surveillance, deepfakes, or massive automation.

It is essential that this ethical debate be clarified so as not to undermine public trust in technology and to allow an enlightened coexistence between humans and machines.

Reactions from the scientific community on this anthropomorphic approach

Several researchers have expressed reservations about this vision of a quasi-personified AI. They stress the necessity to maintain a clear distinction between the technical capabilities of an artificial intelligence model and the human notions of intention or consciousness.

A notable example is the machine learning community, which points out that the terms used by Amodei can be confusing to the general public. This ambiguity could hinder regulatory efforts by fueling irrational fears rather than promoting pragmatic measures.

In the end, ethics in AI should not be reduced to an anthropomorphic image but should instead focus on transparency, responsibility, and fairness in the use of technologies.

The real current challenges of AI security: between fiction and reality

While alarmist discourses around catastrophic AI risks often make headlines, it is important to recall that several very real and documented abuses already affect millions of people. These immediate risks notably concern:

• Automated and arbitrary layoffs caused by algorithmic decisions without effective human oversight.
• Misinformation amplified by non-consensual deepfakes, making fact-checking difficult and potentially influencing public opinion.
• Invasive algorithmic surveillance, intruding into private life and infringing fundamental freedoms.

These phenomena represent concrete challenges requiring urgent political, legal, and social responses rather than an excessive focus on uncertain apocalyptic scenarios. Tackling current risks could improve trust in AI and facilitate responsible adoption.

Current dangers of AI	Description	Impact on society	Recommended actions
Automated layoffs	RGI applied to sort and lay off without human intervention	Job losses, rising unemployment, social tensions	Legal framework, algorithm audits
Non-consensual deepfakes	Abusive use of manipulated content for misinformation	Damage to reputation, manipulation of opinion	Specific legislation, detection tools
Algorithmic surveillance	Massive and intrusive monitoring of individuals from collected data	Violation of privacy and civil liberties	Strict legal frameworks, mandatory transparency

Why shouldn’t attention be diverted from real dangers?

Excessive focus on futuristic and hypothetical risks can paradoxically delay or reduce efforts to solve today’s very tangible problems. In this context, the scientific community and policymakers must maintain a balance between forward-looking discourse and pragmatic issue management.

It is therefore essential that society concentrates on concrete measures, notably:

• Establishing effective and adaptable regulations.
• Strengthening transparency in algorithm design and use.
• Educating the public on the uses and limits of artificial intelligence.

Anthropic and AI regulation: a path to follow?

Facing these multiple challenges, Dario Amodei strongly advocates for ambitious regulation of artificial intelligence. He considers it indispensable to act quickly to establish clear international rules aimed at framing AI technology development and use, especially in sensitive areas like bioterrorism, lethal robotics, and automated jobs.

Anthropic, as a major player, is also committed to reflecting on security and ethics, developing models that integrate moral principles and internal controls. This strategy aims to anticipate abuses and make AI safer for society.

However, this approach raises delicate questions:

• Is it really possible to regulate a technological sector of such speed and complexity?
• What mechanisms can be implemented to guarantee international cooperation against bioterrorism and autonomous weapons?
• How to combine ethics and competitiveness in a globalized economic context?

If regulation appears imperative, it must necessarily balance technological innovation, security, and respect for human rights, under penalty of a major social and political fracture.

What are the main risks raised by the CEO of Anthropic regarding AI?

Dario Amodei warns against major risks such as algorithmic slavery, bioterrorism facilitated by AI, and the use of lethal autonomous drones. These risks concern global security, the economy, and ethics.

Why is the speed of AI development a problem?

The exponential speed of AI development exceeds institutions’ capacities to regulate effectively, resulting in risks of inappropriate or uncontrolled use of this technology in sensitive areas.

What ethical challenges are posed by the anthropomorphism of AI?

AI anthropomorphism can create confusion between machines’ real capabilities and human notions of consciousness or intention, fueling irrational fears and complicating regulation debates.

How can AI facilitate bioterrorism?

AI can rapidly design and optimize dangerous biological agents, making bioterrorism more accessible and harder to detect, posing serious threats to global health security.

What is Anthropic’s stance on AI regulation?

Anthropic and its CEO Dario Amodei advocate for strict international regulation aimed at framing military, economic, and security uses of artificial intelligence, while integrating ethical principles into model design.