Last month, more than 1,300 employees of frontier AI companies wrote a letter calling for the US government to build the capacity to pace automated AI research and development via international coordination.
The Institute for Progress (IFP), where I work on biosecurity policy, released 23 creative policy recommendations in response to the letter. These recommendations were also included as a guest post on Noah Smith’s newsletter, Noahpinion.
In this post, I am sharing some of my personal thoughts on how to think about “pacing” the frontier. I have also copied over the initial IFP recommendations for increasing bioresilience (i.e. our ability to withstand and defend against biological threats), which I helped to write.
Technological progress has led to massive improvements in human life. It has enabled us to live longer, healthier lives and pursue our dreams more than any other generation in human history.
There are also some technologies that can cause harm and should be regulated. For much of the population, there are many technologies that fall into this category. Others disagree. I err on the side of absolutely minimal regulation. There is a category of technology that nearly everyone, including me, agrees should be regulated, e.g. not every person should have the ability to enrich uranium for the purpose of constructing nuclear weapons.
It is as yet unclear whether the automated research and development of artificial intelligence falls into this category. What is clear is that it might.
There is an intermediate answer: build out the capacity to respond if needed.
However, capacity is analogous to power, and increasing the power of government to regulate anything, especially to potentially slow down technology, should not be taken lightly.
We are lucky to live in a world where creative policy –– policy that does not simply increase power with no strings attached, policy that even accelerates and builds in safeguards at the same time, is possible.
The policy recommendations contained in the memo are exactly such creative policies. They focus on:
Providing transparency into how automated AI research and development is proceeding so that we can make better decisions in the future.
Improving state capacity, primarily through increasing the capacity of the Center for AI Standards and Innovation (CAISI) to advise officials and frontier AI companies, making sure the roles of various agencies are clear, and directing intelligence agencies to focus on understanding foreign AI development.
Developing risk mitigation strategies that accelerate defensive capabilities and commercial use, to enable other technologies to progress as fast as possible, especially those that might help us build resilience.
Accelerating the development of AI verification technology, i.e. hardware and cryptographic tools enabling states and frontier labs to prove how their compute is being used without exposing sensitive information. Among other things, this enables treaties to be enforced.
Investing in AI resilience, increasing our defensive capacity against cyber, biological, or other categories of offensive threats that could be accelerated by AI.
Ensuring the US leads in AI, which gives us optionality to pace the frontier of technology if it becomes clear this is necessary. (I would personally argue this also increases the likelihood that the most capable models share Western values.)
Creating the foundation for international cooperation and coordination.
As mentioned, I helped write the recommendations on bioresilience.
There are many fantastic things about biosecurity as a problem set relative to other AI-enabled threats. The threat we face is more clear than other threats, as are the steps we need to take to mitigate the risks. Most of the steps we should take in biosecurity are also relatively cheap compared to other national security efforts, and have strong bipartisan support.
None of these steps meaningfully slow technology development, and some of them may even accelerate it. When Fable was released, Anthropic assessed the risk and felt they had to prevent it from answering any biology-related question. Since then, they have loosened some of these restrictions, enabled by better classifiers. This has allowed them to reduce biology refusals/fallbacks by ~85%. If we continue to create better classifiers for understanding whether a particular prompt poses a biological risk, we can make sure important biotechnological inquiries can be answered in full. This strategy aligns with the memo’s recommendation that CAISI develop standards for safeguards to reduce dangerous uplift without degrading legitimate use, which would add federal capacity to the effort.
Perhaps most importantly, we can be certain the steps we take to build resilience will create a safer future regardless of how AI development proceeds.
This does not mean it will be easy to massively improve our biosecurity, but it does mean it’s possible.
I remain highly optimistic about the tractability of this and other problems related to the development of automated AI R&D, but it will require the government, the frontier labs, and the People working together to make the future of abundance that AI promises possible.
I highly recommend reading the full memo. Now, below is the section from the memo on bioresilience policy recommendations:
We should prepare for a future in which AI could lower the barriers to creating biological threats. Whether AI will meaningfully aid novices in creating biological weapons is hotly debated, and the measured effect was not significant as of mid-2025. But frontier models have continued to improve on every biology benchmark, including those measuring the tacit knowledge required for benchwork troubleshooting. Even if we are not certain, it is worth building resilience now for a world where AI provides an uplift in bioweapon creation, since defenses will take time to build.
Biosecurity is uniquely difficult because bioattacks and biodefenses scale asymmetrically. There is a massive offensive advantage: compared with defenses such as pathogen-agnostic detection and medical countermeasures, attacks take less time to deploy, cost less, and benefit more from advances in AI. Broadly, attacks are relatively more bottlenecked by information and tacit knowledge, while defenses are relatively more bottlenecked by physical properties, such as manufacturing capacity and physical infrastructure. By removing friction in accessing information, including tacit knowledge, AI is poised to accelerate attackers’ progress more than defenders’.
The US should therefore build a strong defense to prevent, detect, and defend against biological threats. We recommend:
Prevent dangerous biological threats from being created and released: Congress should secure the DNA supply chain, such as by passing the Biosecurity Modernization and Innovation Act as part of the American Biotechnology Competitiveness Act, requiring DNA synthesis providers to screen customers and orders while minimizing burdens on legitimate research. Congress should also equip CAISI to evaluate new models for biological risks and to develop safeguards standards to reduce dangerous uplift without degrading legitimate use. Finally, OSTP should coordinate NIH and NIST pilots of tiered access controls for narrow subsets of pathogen data most likely to enable misuse and provide the gene synthesis screening framework requested in Executive Order 14292.
Detect threats before they spread widely: The CDC should invest $80M annually in a federal pathogen early-warning system, adding metagenomic sequencing to existing traveler, wastewater, and clinical surveillance networks, so the US can rapidly detect novel biological threats. Using the resulting information, the Defense Innovation Unit should continue to fund the development of better AI tools capable of scanning these billions of data points for emerging threats, and explore new options for better biosurveillance.
Defend the population and shorten the path out of the pandemic: Congress should sustainably fund the Strategic National Stockpile to grow and maintain day-zero reserves of respirators and medical countermeasures for future pandemics. Congress should also sustainably fund warm-base manufacturing capacity to ensure that we are prepared to rapidly manufacture countermeasures for novel threats. The Biomedical Advanced Research and Development Authority (BARDA) should provide advance purchase commitments for broad-spectrum antivirals. The Department of Health and Human Services should fund large-scale trials of air-cleaning technologies, such as Far-UVC and glycol vapors, and recommend evidence-based updates to ASHRAE Standard 241. Finally, Congress should provide incentives for buildings to procure these air-cleaning technologies.
Many more projects should be undertaken in biosecurity, as well as in other critical areas. This list is far from exhaustive. IFP is currently developing more concrete project proposals to increase AI resilience as part of The Launch Sequence.
Well written, Olivia! Excited to see more of these ideas take root in government.