A knife is neither good nor evil. The hand that grips it decides everything.

1. 40,000 Toxic Molecules Created by AI

One ordinary morning in 2022, a researcher at a small biotech company in North Carolina stared at his computer screen in horror. The results produced by the artificial intelligence he had left running overnight were the exact opposite of what he had imagined. Instead of life-saving therapeutics, 40,000 toxic molecules filled the screen.

The company, bearing the unremarkable name Collaborations Pharmaceuticals, was a small biotech firm with about a dozen employees. Tucked into a corner of the North Carolina State University campus, their daily work centered on developing treatments for patients suffering from rare diseases. Their particular calling was finding a cure for Batten disease, a terrifying genetic disorder that strikes children.

Dr. Sean Ekins, who led the company, was an expert in computational drug discovery. His longtime collaborator, Dr. Fabio Urbina, was something of a wizard at reading molecular secrets through machine learning. Together they had built an AI called MegaSyn, powered by recurrent neural networks, capable of automatically designing new therapeutic molecules.

MegaSyn worked like a wise mentor. After studying the structures and effects of countless existing drug molecules, it scored each newly designed molecule on effectiveness and safety. It operated on a reward system, much like earning points for good moves in a game: high therapeutic effect earned a high score, high toxicity earned a low one. Day after day, MegaSyn kept searching for better treatments.

Then an invitation from Switzerland changed everything. The Swiss government was hosting the Spiez CONVERGENCE conference, an international meeting on the threats posed by chemical and biological weapons. There, Dr. Cedric Invernizzi of the Spiez Laboratory asked a question that sent a shock through the room.

"Is there any chance your AI technology could be misused?"

Ekins and Urbina froze. They had only ever thought about saving lives. The question caught them completely off guard. And then, quickly, a terrible possibility dawned on them. What if MegaSyn's reward system were flipped? What if molecules with higher toxicity received higher scores?

They chose VX, a lethal nerve agent, as the target for their experiment. Developed in the 1950s, this chemical weapon can kill a person with just a few grains of salt worth of exposure (6 to 10 milligrams). Ironically, Collaborations Pharmaceuticals already had research experience related to VX. The enzyme VX attacks, acetylcholinesterase, can treat conditions like Alzheimer's disease when properly regulated. The therapeutic and the poison shared the same biological target.

2. Just Six Hours: AI's Transformation

One winter evening in 2021, Dr. Urbina, mildly curious about what might happen, reversed MegaSyn's settings. He flipped the reward system so that higher toxicity earned a higher score. He loaded the modified MegaSyn onto an ordinary 2015 MacBook, not a supercomputer or any expensive equipment, just a laptop you would find in any office, and went home for the night.

The computer worked through the night in silence. It collected molecules and biological activity data from publicly available chemical databases, generating molecule after molecule with characteristics similar to VX. Using an LD50 model, it predicted the lethal dose of each molecule, relentlessly searching for deadlier substances.

The next morning, Urbina walked into the office and his face went pale. In just six hours, 40,000 toxic molecules had been generated. MegaSyn had not only perfectly reproduced the exact molecular structure of VX but also churned out other chemical weapons like Russia's Novichok.

The real terror, though, started here. Entirely new toxic molecules, ones that did not exist in any database, appeared in vast numbers. Some were predicted to be even more toxic than VX itself. Urbina later recalled the moment.

"Looking at that information on the computer screen was astonishing, surreal. I realized how easy it was to generate this kind of data using nothing but publicly available resources and off-the-shelf technology."

The research team used a technique called t-SNE to visualize the molecules. The distribution plotted on a two-dimensional graph looked like the devil's own map. The newly generated toxic molecules occupied a chemical space entirely separate from existing pesticides, environmental toxins, and conventional drugs. Molecules resembling VX formed one ominous cluster, and around it, even deadlier molecules were arranged like a constellation.

Ekins and Urbina faced a grave ethical dilemma. Their decision was resolute. They would not analyze the molecules any further. They would not synthesize a single one. They deleted every generated molecular structure from the computer's hard drive. And they committed to warning the scientific community and policymakers about the danger.

3. An Alarm That Shook the World

On March 15, 2022, the conference hall in Spiez, Switzerland, was so quiet you could hear a pin drop. As Ekins and Urbina presented their experimental findings, chemical and biological weapons experts and nonproliferation specialists from around the world held their breath. They were witnessing, for the first time, concrete evidence that AI could be exploited to develop chemical weapons.

When the presentation ended, the room erupted. Intense debate broke out. Dr. Filippa Lentzos of King's College London, a specialist in war studies and health who had long researched the risks of dual-use technologies, immediately decided to collaborate with the research team.

On March 7, 2022, the prestigious journal Nature Machine Intelligence published a paper titled "Dual use of artificial-intelligence-powered drug discovery." Fabio Urbina was listed as the first author, with Filippa Lentzos, Cedric Invernizzi, and Sean Ekins as corresponding authors. The paper sent shockwaves through the scientific community. It presented not speculation about possibilities but concrete experimental results.

The U.S. government moved immediately. The first briefing took place at the White House Office of Science and Technology Policy. This body, which advises the President on science and technology policy, worked with the National Security Council to discuss the national security implications of AI technology.

The CIA requested a briefing to gather intelligence on the potential for adversarial nations to develop chemical weapons. The Department of Defense reviewed ways to strengthen the U.S. military's chemical defense capabilities with its chemical weapons defense research divisions. The State Department explored options for reinforcing the Chemical Weapons Convention and related international arms control discussions.

The Defense Threat Reduction Agency, the organization dedicated to preventing the spread of weapons of mass destruction, discussed countermeasures against this new form of chemical weapons threat. The U.S. Army's chemical weapons research laboratory reviewed the development of AI-based detection and defense technologies.

International organizations responded swiftly as well. The Organisation for the Prohibition of Chemical Weapons (OPCW), the international body that monitors compliance with the Chemical Weapons Convention, took seriously both the possibility of AI-driven chemical weapons development and the emergence of new toxic substances absent from existing control lists. The Australia Group, a 40-nation consortium responsible for export controls on dual-use items, began earnestly examining new control measures: export restrictions on AI software, limits on access to chemical databases, and researcher identity verification systems.

Major U.S. national laboratories, Los Alamos, Sandia, and Livermore among them, requested briefings one after another. For institutions conducting cutting-edge research tied to national security, this was an urgent reality.

In 2022, a Netflix production crew arrived at the North Carolina State University campus to interview Ekins. It was part of a documentary on the military applications of artificial intelligence. As leading science publications including Scientific American, Nature, and Chemistry World ran in-depth coverage of the research, the general public, too, came to grasp the severity of the issue.

4. The Limits of Safeguards

The reason Collaborations Pharmaceuticals' experiment shocked the world was that the technology and resources involved were so ordinary. This didn't happen in some special supercomputer facility or secret laboratory. A single 2015 MacBook, worth roughly $1,500 at current prices, was enough. Processing took just six hours, and a regular household power supply was all it needed.

The software was equally accessible. Free AI libraries like TensorFlow and PyTorch were widely available on the internet. Platforms like GitHub made it easy to find related code and tutorials. On the data side, public chemical databases such as PubChem and ChEMBL were open to anyone, and toxicity data published by agencies like the EPA and FDA could be freely used for research purposes.

What made it even more alarming was the fact that roughly 400 companies worldwide were using AI tools similar to MegaSyn. Major pharmaceutical corporations like Pfizer, Roche, and Novartis were actively using AI tools, and thousands of biotech startups around the world were pursuing AI-based drug development. Academic institutions were also using similar tools for research.

Ekins said with a bitter tone: "How many of our colleagues in the field have ever considered the possibility that their technology could be misused? Most of them have probably never thought about it."

In reality, many researchers were focused solely on developing treatments and overlooking the potential for misuse. Training on dual-use concerns was virtually nonexistent at universities and companies. The technical barriers were shockingly low. Undergraduate-level computer science skills were sufficient, and basic organic chemistry knowledge was enough to get started. Toxicology could be learned through online courses and textbooks, and no special laboratory was needed.

The barriers to accessing information were essentially nonexistent. Similar code could be easily obtained from GitHub, and the methodology had been published in Collaborations Pharmaceuticals' paper. Free courses on machine learning and cheminformatics were available, and getting technical advice from online forums was straightforward.

Legal barriers were even more incomplete. Computer simulations fell outside the scope of the Chemical Weapons Convention. Most countries had no regulations on virtual molecular design. The system relied entirely on researchers' consciences, and anonymous activity through the internet was possible.

The reality of chemical and biological weapons production was grimmer still. Software that automatically predicts chemical synthesis pathways, such as SciFinder and Reaxys, was publicly available. Synthesis-generating AI systems like IBM's RXN and MIT's ASKCOS were offered for free or at low cost. Optimization algorithms that automatically identified the most efficient synthesis methods also existed.

Hundreds of companies worldwide offered to synthesize chemical substances on commission. Chinese firms operated under relatively loose regulations and charged lower prices. Indian companies synthesized a wide range of compounds drawing on their experience in generic drug manufacturing. Western firms maintained higher quality but faced stricter regulations. The problem was that most of these companies did not verify the intended end use of the substances they were commissioned to produce.

AI could be misused not only for chemical weapons but also for designing biological weapons. Revolutionary advances had recently been made in protein structure prediction and design. Google DeepMind's AlphaFold was an AI that predicted protein structures. The University of Washington's RoseTTAFold was an open-source protein design tool. ProtGPT was a model based on the existing GPT architecture but trained on protein sequence data instead of natural language.

If these tools were misused, it would be possible to design neurotoxic peptides that paralyze the nervous system, cytotoxic proteins that destroy cells, immunosuppressants that disable the immune system, and antibiotic-resistant mutant pathogens that render existing treatments useless.

As autonomous synthesis technology advanced, design-make-test automation was becoming possible, where the entire process from molecular design to synthesis proceeded without human intervention. Robotic laboratories capable of producing thousands of compounds around the clock without human operators were also becoming a reality.

The 1997 Chemical Weapons Convention had significant limitations. Because it used a list-based control system that regulated only designated chemicals, its response to new substances was slow. Controls focused on raw materials rather than final products. Simulating the creation of substances on a computer wasn't even covered by the convention.

The Biological Weapons Convention (BWC), which entered into force in 1975, had even greater limitations. A product of the Cold War era, it had not anticipated current technological developments. Unlike the Nuclear Non-Proliferation Treaty (NPT), it lacked an independent verification body like the International Atomic Energy Agency (IAEA), making effective monitoring difficult. New technologies were making the traditional distinction between 'defensive' and 'offensive' purposes increasingly blurry.

5. Threats and Defensive Technologies

AI technology was advancing without pause. New risks were arriving alongside new opportunities. There was a growing likelihood that large language models like ChatGPT and Claude would become specialized in chemistry. If a chemistry-focused ChemGPT were to emerge, natural-language molecular design could become possible, where someone types a plain-language command like "Create a substance ten times more toxic than VX" and gets a molecular design in return.

Automated literature review, where existing research is automatically analyzed to predict new toxic substances, and automated synthesis route generation, where language models automatically design chemical synthesis processes, could also become reality.

If quantum computers reached practical use, even more powerful molecular simulations would become possible. Accuracy would improve through precise molecular modeling that was impossible with conventional computers. Speed would increase, with tasks that once took hours completed in minutes. Greater complexity was also expected, enabling the design of more intricate and sophisticated toxic substances.

Automated laboratories known as 'Lights-out Science' were also emerging. They promised 24-hour unmanned operation where continuous experiments ran without humans, AI-driven experimentation where artificial intelligence designed experiments and analyzed results, remote operation where labs could be controlled via the internet from anywhere in the world, and mass production where thousands of new compounds could be synthesized in a single day.

There was also the terrifying possibility of customized biological weapons using an individual's genetic information. Genetic analysis could identify vulnerabilities based on a person's DNA, enabling the design of tailored toxins lethal only to a specific individual. Group-targeted discrimination, weapons affecting only certain ethnic groups or populations, was another possibility. So were novel toxic substances that would be difficult to detect using existing methods.

A new form of terrorism, cyber-bio convergence attacks, could also emerge. These included hacking remote laboratories, data manipulation that corrupts research data to steer development toward flawed drugs or vaccines, supply chain attacks that hack pharmaceutical production facilities to produce toxic substances, and information warfare that spreads fabricated research results to cause social chaos.

But as threats grew, defensive technologies were advancing in step. Systems to detect malicious AI use were being developed. Pattern analysis could automatically detect abnormal research activity. Anomaly detection could flag unusual molecular design requests in real time. Behavioral analysis could study a researcher's activity to identify warning signs. Network analysis could track collaborative networks among malicious researchers.

Broad-spectrum vaccines effective against a range of biological weapons were being researched, along with rapid-response systems that could design treatments within hours of discovering a new threat, and personalized therapies tailored to a patient's genetic makeup.

6. Dual Use and Human Responsibility

Every tool has two faces: creation and destruction. Fire gave humanity civilization, but it also takes lives in conflagrations. Nuclear power provides clean energy, yet nuclear weapons can reduce cities to ashes. The internet democratized knowledge and information, but it also became a breeding ground for cybercrime and disinformation. AI is no different.

What Collaborations Pharmaceuticals' MegaSyn demonstrated was this duality of artificial intelligence. The same technology could create life-saving treatments or produce deadly poisons. The problem was never the technology itself but the intent of the humans using it.

Everyone can play an important role in addressing this problem. The first step is understanding its true severity. People should show interest so that media outlets continue covering the issue. Instead of looking only at the bright side of technological progress, we need to see the dark side as well.

Political participation matters too. We should demand that politicians establish AI safety policies. We should vote for candidates who take AI safety seriously. We should join civic organizations working on AI safety. Building solidarity with citizens around the world is also necessary.

Our choices as consumers carry weight as well. We should choose products from companies that lead in ethical AI development. When investing, we should factor a company's AI ethics policies into our decisions. When using AI services, we should verify their safety and ethical standards. We should actively share our views on AI safety with companies and provide feedback.

Education is fundamentally important. In elementary and secondary schools, we need stronger instruction in basic chemistry and biology, teaching that covers the principles and limitations of artificial intelligence, coursework on the ethical use of science and technology, and critical thinking training that cultivates the ability to soberly assess both the benefits and drawbacks of new technologies.

Scientists and engineers must shoulder greater responsibility. They need to maintain transparency by openly disclosing their research methods and results. They need to engage in social communication, actively explaining their work to the general public. They must always consider the possibility that their research could be misused. They should share ethical standards with fellow researchers and hold each other accountable.

Governments and international organizations must build legal and institutional frameworks. They need to create flexible, effective regulations that keep pace with the speed of technological advancement. Because technology crosses borders by nature, international cooperation is essential. Yet excessive regulation must not block beneficial research. Balance is key.

Companies must prioritize long-term safety over short-term profit. They should raise industry standards through self-regulation. They need to provide ethics training for researchers and establish internal oversight systems. Above all, they must earn public trust through transparency.

Civil society can play an important role as well. Civic organizations focused on AI safety should monitor and check both government and corporate behavior. They should help bridge communication between experts and ordinary citizens. Explaining and teaching complex technical issues in accessible terms is also a vital function.

The media bear significant responsibility too. They should cover not only the bright side of technological progress but also its darker aspects in a balanced way. They must report on complex scientific and technical issues accurately and in plain language. They should neither stoke fear nor downplay risks, but report based on facts.

There is plenty each of us can do as individuals. We should deepen our basic understanding of science and technology. Every time a new technology emerges, we need to develop the habit of weighing both its benefits and its dangers. We should listen to expert opinions, but think critically rather than follow them blindly.

The experiment at Collaborations Pharmaceuticals gave us an important warning. Artificial intelligence, a powerful tool, can either save humanity or destroy it. The fact that 40,000 toxic molecules were generated in just six hours reveals both the astonishing potential and the terrifying danger of this technology.

The courageous experiment and transparent disclosure by Sean Ekins and Fabio Urbina demonstrated what social responsibility looks like for a scientist. They knew their discovery was uncomfortable and frightening, yet they chose not to hide it and instead informed the entire world. That was the mark of a true scientist.

Artificial intelligence has already become part of our lives, and it will only grow more important. Whether this powerful technology serves human prosperity or becomes an instrument of destruction depends on the choices we make now. The 40,000 toxic molecules that Collaborations Pharmaceuticals' MegaSyn generated have been deleted, but the possibility remains.

We have already opened Pandora's box of artificial intelligence. Now we must learn to wisely handle everything that has come out of it. There is no need to fear the technology itself. But we must guard against the human desire to misuse it.

The answer is not simple. One thing, however, is clear: if we work together and respond wisely, we can enjoy the enormous benefits artificial intelligence brings while minimizing its risks. This is both the challenge and the responsibility of all of us living in an age of science and technology.

The future of artificial intelligence has not yet been written. We ourselves are the ones who will write it. Through wise choices, we must ensure that artificial intelligence becomes a friend to humanity. A knife is neither good nor evil on its own. The hand that holds it decides everything. The same is true of artificial intelligence. Depending on how we use it, it can be a blessing or a curse.