policy report
The 2024 US Regulatory Landscape
The 2024 US Regulatory Landscape
Published by Convergence Analysis, this series is designed to be a primer for policymakers, researchers, and individuals seeking to develop a high-level overview of the current state of AI regulation.
Updated: April 28, 2024
Outline
Structure of AI Regulations
In large part due to legislative gridlock in the US Congress, the United States has taken an approach to AI governance centered around executive orders and non-binding declarations by the Biden administration. Though this approach has key limitations, such as the inability to allocate budget for additional programs, it has resulted in a significant amount of executive action over the past year.
Three key executive actions stand out in shaping the US approach:
1
US / China Semiconductor Export Controls: Launched on Oct 7, 2022, these export controls (and subsequent updates) on high-end semiconductors used to train AI models mark a significant escalation in US efforts to restrict China's access to advanced computing and AI technologies. The rules, issued by the Bureau of Industry and Security (BIS), ban the export of advanced chips, chip-making equipment, and semiconductor expertise to China. They aim to drastically slow China's AI development and protect US national security by targeting the hardware essential to develop powerful AI models.
1
US / China Semiconductor Export Controls: Launched on Oct 7, 2022, these export controls (and subsequent updates) on high-end semiconductors used to train AI models mark a significant escalation in US efforts to restrict China's access to advanced computing and AI technologies. The rules, issued by the Bureau of Industry and Security (BIS), ban the export of advanced chips, chip-making equipment, and semiconductor expertise to China. They aim to drastically slow China's AI development and protect US national security by targeting the hardware essential to develop powerful AI models.
1
US / China Semiconductor Export Controls: Launched on Oct 7, 2022, these export controls (and subsequent updates) on high-end semiconductors used to train AI models mark a significant escalation in US efforts to restrict China's access to advanced computing and AI technologies. The rules, issued by the Bureau of Industry and Security (BIS), ban the export of advanced chips, chip-making equipment, and semiconductor expertise to China. They aim to drastically slow China's AI development and protect US national security by targeting the hardware essential to develop powerful AI models.
2
Blueprint for an AI Bill of Rights: Released in October 2022, this blueprint outlines five principles to guide the design, use, and deployment of automated systems to protect the rights of the American public. These principles include safe and effective systems, algorithmic discrimination protections, data privacy, notice and explanation, and human alternatives, consideration, and fallback. While non-binding, the blueprint aims to inform policy decisions and align action across all levels of government.
2
Blueprint for an AI Bill of Rights: Released in October 2022, this blueprint outlines five principles to guide the design, use, and deployment of automated systems to protect the rights of the American public. These principles include safe and effective systems, algorithmic discrimination protections, data privacy, notice and explanation, and human alternatives, consideration, and fallback. While non-binding, the blueprint aims to inform policy decisions and align action across all levels of government.
2
Blueprint for an AI Bill of Rights: Released in October 2022, this blueprint outlines five principles to guide the design, use, and deployment of automated systems to protect the rights of the American public. These principles include safe and effective systems, algorithmic discrimination protections, data privacy, notice and explanation, and human alternatives, consideration, and fallback. While non-binding, the blueprint aims to inform policy decisions and align action across all levels of government.
3
The Executive Order on Artificial Intelligence: Issued in October 2023, this order directs various federal agencies to act to promote the responsible development and use of AI. It calls for these agencies to develop AI risk management frameworks, develop AI standards and technical guidance, create better systems for AI oversight, and foster public-private partnerships. It marks the first comprehensive and coordinated effort to shape AI governance across the federal government, but lacks binding regulation or specific details as it primarily orders individual agencies to publish reports on next steps.
3
The Executive Order on Artificial Intelligence: Issued in October 2023, this order directs various federal agencies to act to promote the responsible development and use of AI. It calls for these agencies to develop AI risk management frameworks, develop AI standards and technical guidance, create better systems for AI oversight, and foster public-private partnerships. It marks the first comprehensive and coordinated effort to shape AI governance across the federal government, but lacks binding regulation or specific details as it primarily orders individual agencies to publish reports on next steps.
3
The Executive Order on Artificial Intelligence: Issued in October 2023, this order directs various federal agencies to act to promote the responsible development and use of AI. It calls for these agencies to develop AI risk management frameworks, develop AI standards and technical guidance, create better systems for AI oversight, and foster public-private partnerships. It marks the first comprehensive and coordinated effort to shape AI governance across the federal government, but lacks binding regulation or specific details as it primarily orders individual agencies to publish reports on next steps.
What are the key traits of the US’ AI governance strategy?
The US’ initial binding regulations focus on classifying AI models by compute ability and regulating hardware:
The US has taken a distinctive approach to AI governance by controlling the hardware and computational power required to train and develop AI models. It is uniquely positioned to leverage this compute-based approach to regulation, as it is home to all leading vendors of high-end AI chips (Nvidia, AMD, Intel) and consequently has direct legislative control over these chips.
The US has taken a distinctive approach to AI governance by controlling the hardware and computational power required to train and develop AI models. It is uniquely positioned to leverage this compute-based approach to regulation, as it is home to all leading vendors of high-end AI chips (Nvidia, AMD, Intel) and consequently has direct legislative control over these chips.
The US has taken a distinctive approach to AI governance by controlling the hardware and computational power required to train and develop AI models. It is uniquely positioned to leverage this compute-based approach to regulation, as it is home to all leading vendors of high-end AI chips (Nvidia, AMD, Intel) and consequently has direct legislative control over these chips.
This is exemplified by the US-China export controls, which aim to restrict China's access to the high-end AI chips necessary for developing advanced AI systems by setting limits on the processing power & performance density of exportable chips.
This is exemplified by the US-China export controls, which aim to restrict China's access to the high-end AI chips necessary for developing advanced AI systems by setting limits on the processing power & performance density of exportable chips.
This is exemplified by the US-China export controls, which aim to restrict China's access to the high-end AI chips necessary for developing advanced AI systems by setting limits on the processing power & performance density of exportable chips.
This focus can also be seen in the Executive Order’s reporting requirements for AI models, which have thresholds for computing capacity or model training measured in floating-point operations per second (FLOP/s).
This focus can also be seen in the Executive Order’s reporting requirements for AI models, which have thresholds for computing capacity or model training measured in floating-point operations per second (FLOP/s).
This focus can also be seen in the Executive Order’s reporting requirements for AI models, which have thresholds for computing capacity or model training measured in floating-point operations per second (FLOP/s).
Beyond export controls, the US appears to be pursuing a decentralized, largely non-binding approach relying on executive action:
Due to structural challenges in passing binding legislation through a divided Congress, the US has relied primarily on executive orders and agency actions to shape its AI governance strategy, which don’t require any congressional approval. It has chosen to decentralize its research and regulatory process by distributing such work among selected agencies.
Due to structural challenges in passing binding legislation through a divided Congress, the US has relied primarily on executive orders and agency actions to shape its AI governance strategy, which don’t require any congressional approval. It has chosen to decentralize its research and regulatory process by distributing such work among selected agencies.
Due to structural challenges in passing binding legislation through a divided Congress, the US has relied primarily on executive orders and agency actions to shape its AI governance strategy, which don’t require any congressional approval. It has chosen to decentralize its research and regulatory process by distributing such work among selected agencies.
Instead of including specific binding requirements in the US Executive Order on AI, the Biden administration has preferred to task various federal agencies with developing their own frameworks, standards, and oversight mechanisms. Most of these upcoming standards are still being developed and are not yet public.
Instead of including specific binding requirements in the US Executive Order on AI, the Biden administration has preferred to task various federal agencies with developing their own frameworks, standards, and oversight mechanisms. Most of these upcoming standards are still being developed and are not yet public.
Instead of including specific binding requirements in the US Executive Order on AI, the Biden administration has preferred to task various federal agencies with developing their own frameworks, standards, and oversight mechanisms. Most of these upcoming standards are still being developed and are not yet public.
Such executive orders are limited first and foremost by the lack of jurisdiction to allocate more budget for specific policy implementations, a power controlled by Congress.
Such executive orders are limited first and foremost by the lack of jurisdiction to allocate more budget for specific policy implementations, a power controlled by Congress.
Such executive orders are limited first and foremost by the lack of jurisdiction to allocate more budget for specific policy implementations, a power controlled by Congress.
A secondary limitation is that executive orders are easy to repeal or reverse when the US presidency changes every 4 years, meaning that even binding executive orders may not be enforced long-term.
A secondary limitation is that executive orders are easy to repeal or reverse when the US presidency changes every 4 years, meaning that even binding executive orders may not be enforced long-term.
A secondary limitation is that executive orders are easy to repeal or reverse when the US presidency changes every 4 years, meaning that even binding executive orders may not be enforced long-term.
The Blueprint for an AI Bill of Rights and the Executive Order on AI provide high-level guidance and principles but lack the binding force of law. They serve more as a framework for agencies to develop their own policies and practices, rather than a centralized, comprehensive regulatory regime like the EU AI Act.
The Blueprint for an AI Bill of Rights and the Executive Order on AI provide high-level guidance and principles but lack the binding force of law. They serve more as a framework for agencies to develop their own policies and practices, rather than a centralized, comprehensive regulatory regime like the EU AI Act.
The Blueprint for an AI Bill of Rights and the Executive Order on AI provide high-level guidance and principles but lack the binding force of law. They serve more as a framework for agencies to develop their own policies and practices, rather than a centralized, comprehensive regulatory regime like the EU AI Act.
US AI policy is strongly prioritizing its geopolitical AI arms race with China:
The US AI governance strategy is heavily influenced by the perceived threat of China's rapid advancements in AI and the potential implications for national security and the global balance of power. The only binding actions taken by the US (enforcing semiconductor export controls) are explicitly designed to counter China's AI ambitions and maintain the US' technological and military superiority.
The US AI governance strategy is heavily influenced by the perceived threat of China's rapid advancements in AI and the potential implications for national security and the global balance of power. The only binding actions taken by the US (enforcing semiconductor export controls) are explicitly designed to counter China's AI ambitions and maintain the US' technological and military superiority.
The US AI governance strategy is heavily influenced by the perceived threat of China's rapid advancements in AI and the potential implications for national security and the global balance of power. The only binding actions taken by the US (enforcing semiconductor export controls) are explicitly designed to counter China's AI ambitions and maintain the US' technological and military superiority.
This geopolitical focus sets the US apart from the EU, which has prioritized the protection of individual rights and the ethical development of AI, or China, which has prioritized internal social control and alignment with party values. The US strategy appears to be more concerned with the strategic implications of AI and ensuring that the technology aligns with US interests in the global arena.
This geopolitical focus sets the US apart from the EU, which has prioritized the protection of individual rights and the ethical development of AI, or China, which has prioritized internal social control and alignment with party values. The US strategy appears to be more concerned with the strategic implications of AI and ensuring that the technology aligns with US interests in the global arena.
This geopolitical focus sets the US apart from the EU, which has prioritized the protection of individual rights and the ethical development of AI, or China, which has prioritized internal social control and alignment with party values. The US strategy appears to be more concerned with the strategic implications of AI and ensuring that the technology aligns with US interests in the global arena.
AI Evaluation & Risk Assessments
The AI Bill of Rights states that automated systems should undergo pre-deployment testing, risk identification and mitigation, and ongoing safety monitoring. Tests should:
be extensive;
be extensive;
be extensive;
follow domain-specific best practices;
follow domain-specific best practices;
follow domain-specific best practices;
take into account the specific technology and the role of human operators;
take into account the specific technology and the role of human operators;
take into account the specific technology and the role of human operators;
include automated and human-led testing;
include automated and human-led testing;
include automated and human-led testing;
mirror deployment conditions;
mirror deployment conditions;
mirror deployment conditions;
be repeated for each deployment with material differences in conditions;
be repeated for each deployment with material differences in conditions;
be repeated for each deployment with material differences in conditions;
be compared with status-quo/human performance as a baseline to meet pre-deployment.
be compared with status-quo/human performance as a baseline to meet pre-deployment.
be compared with status-quo/human performance as a baseline to meet pre-deployment.
Crucially, the bill states that possible outcomes of these evaluations should include the possibility of not deploying or even removing a system, though it does not prescribe the conditions under which deployment should be disallowed.
The bill states that risk identification should focus on impact on people’s rights, opportunities, and access, as well as risks from purposeful misuse of the system. High-impact risks should receive proportionate attention. Further, automated systems should be designed to allow for independent evaluation, such as by researchers, journalists, third-party auditors and more. Evaluations are also required to assess algorithmic discrimination, as we discuss in a separate section.
The Executive Order on AI makes these principles more concrete, and also includes calls to develop better evaluation techniques. In summary, the EO calls for several new programs to provide AI developers with guidance, benchmarks, test beds, and other tools and requirements for evaluating the safety of AI, as well as requiring AI developers to share certain information with the government (such as the results of red-team tests). In particular:
Section 4.1(a): Calls for the Secretary of Commerce, acting through NIST, to conduct the following actions within 270 days:
Section 4.1(a): Calls for the Secretary of Commerce, acting through NIST, to conduct the following actions within 270 days:
Section 4.1(a): Calls for the Secretary of Commerce, acting through NIST, to conduct the following actions within 270 days:
Section 4.1(a)(i)(C): Launch an initiative to create guidance and benchmarks for evaluating and auditing AI capabilities, focusing on capabilities through which AI could cause harm such as cybersecurity or biosecurity
Section 4.1(a)(i)(C): Launch an initiative to create guidance and benchmarks for evaluating and auditing AI capabilities, focusing on capabilities through which AI could cause harm such as cybersecurity or biosecurity
Section 4.1(a)(i)(C): Launch an initiative to create guidance and benchmarks for evaluating and auditing AI capabilities, focusing on capabilities through which AI could cause harm such as cybersecurity or biosecurity
Section 4.1(a)(ii): Establish guidelines for AI developers to conduct red-teaming tests (with an explicit exception for AI in national security) and assess the safety, security, and trustworthiness of foundation models.
Section 4.1(a)(ii): Establish guidelines for AI developers to conduct red-teaming tests (with an explicit exception for AI in national security) and assess the safety, security, and trustworthiness of foundation models.
Section 4.1(a)(ii): Establish guidelines for AI developers to conduct red-teaming tests (with an explicit exception for AI in national security) and assess the safety, security, and trustworthiness of foundation models.
Section 4.1(a)(ii)(B): Coordinate with the Sec of Energy and Director of the National Science Foundation to develop and make available testing environments (e.g. testbeds) to AI developers.
Section 4.1(a)(ii)(B): Coordinate with the Sec of Energy and Director of the National Science Foundation to develop and make available testing environments (e.g. testbeds) to AI developers.
Section 4.1(a)(ii)(B): Coordinate with the Sec of Energy and Director of the National Science Foundation to develop and make available testing environments (e.g. testbeds) to AI developers.
Section 4.1(b): calls for the Secretary of Energy to, within 270 days, implement a plan for developing the DoE’s AI model evaluation tools and testbeds, “to be capable of assessing near-term extrapolations of AI systems’ capabilities”. In particular, these evaluations should be able to “generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards.”
Section 4.1(b): calls for the Secretary of Energy to, within 270 days, implement a plan for developing the DoE’s AI model evaluation tools and testbeds, “to be capable of assessing near-term extrapolations of AI systems’ capabilities”. In particular, these evaluations should be able to “generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards.”
Section 4.1(b): calls for the Secretary of Energy to, within 270 days, implement a plan for developing the DoE’s AI model evaluation tools and testbeds, “to be capable of assessing near-term extrapolations of AI systems’ capabilities”. In particular, these evaluations should be able to “generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards.”
Section 4.2(a)(i): calls for the Secretary of Commerce to, within 90 days, require companies developing dual-use foundation models to share with the government information, reports, and records on the results of any red-team testing that’s based on the guidelines referenced in 4.1(a)(ii). These should include a description of any adjustments the company takes to meet safety objectives, “such as mitigations to improve performance on these red-team tests and strengthen overall model security”. Prior to the development of those red-teaming guidelines from 4.1(a)(ii), this description must include results of any red-teaming that may provide easier access to:
Section 4.2(a)(i): calls for the Secretary of Commerce to, within 90 days, require companies developing dual-use foundation models to share with the government information, reports, and records on the results of any red-team testing that’s based on the guidelines referenced in 4.1(a)(ii). These should include a description of any adjustments the company takes to meet safety objectives, “such as mitigations to improve performance on these red-team tests and strengthen overall model security”. Prior to the development of those red-teaming guidelines from 4.1(a)(ii), this description must include results of any red-teaming that may provide easier access to:
Section 4.2(a)(i): calls for the Secretary of Commerce to, within 90 days, require companies developing dual-use foundation models to share with the government information, reports, and records on the results of any red-team testing that’s based on the guidelines referenced in 4.1(a)(ii). These should include a description of any adjustments the company takes to meet safety objectives, “such as mitigations to improve performance on these red-team tests and strengthen overall model security”. Prior to the development of those red-teaming guidelines from 4.1(a)(ii), this description must include results of any red-teaming that may provide easier access to:
Bio-weapon development and use;
Bio-weapon development and use;
Bio-weapon development and use;
The discovery & exploitation of software vulnerabilities;
The discovery & exploitation of software vulnerabilities;
The discovery & exploitation of software vulnerabilities;
The “use of software or tools to influence real or virtual events”;
The “use of software or tools to influence real or virtual events”;
The “use of software or tools to influence real or virtual events”;
The possibility of self-replication or propagation.
The possibility of self-replication or propagation.
The possibility of self-replication or propagation.
The EO calls on individual government orgs and secretaries to provide one-off evaluations, such as:
Section 4.3(a)(i): The head of each agency with authority over critical infrastructure shall provide to the Sec of Homeland Security an assessment of potential risks related to the use of AI in critical infrastructure and how AI may make infrastructure more vulnerable to failures and physical and cyber attacks.
Section 4.3(a)(i): The head of each agency with authority over critical infrastructure shall provide to the Sec of Homeland Security an assessment of potential risks related to the use of AI in critical infrastructure and how AI may make infrastructure more vulnerable to failures and physical and cyber attacks.
Section 4.3(a)(i): The head of each agency with authority over critical infrastructure shall provide to the Sec of Homeland Security an assessment of potential risks related to the use of AI in critical infrastructure and how AI may make infrastructure more vulnerable to failures and physical and cyber attacks.
Section 4.4(a)(i): The Secretary of Homeland Security shall:
Section 4.4(a)(i): The Secretary of Homeland Security shall:
Section 4.4(a)(i): The Secretary of Homeland Security shall:
evaluate the potential for AI to be misused to develop chemical, biological, radiological, and nuclear (CBRN) threats (and their potential to counter such threats);
evaluate the potential for AI to be misused to develop chemical, biological, radiological, and nuclear (CBRN) threats (and their potential to counter such threats);
evaluate the potential for AI to be misused to develop chemical, biological, radiological, and nuclear (CBRN) threats (and their potential to counter such threats);
consult with experts in AI & CBRN issues, including third-party model evaluators, to evaluate AI capabilities to present CBRN threats;
consult with experts in AI & CBRN issues, including third-party model evaluators, to evaluate AI capabilities to present CBRN threats;
consult with experts in AI & CBRN issues, including third-party model evaluators, to evaluate AI capabilities to present CBRN threats;
Section 4.4(a)(ii): The Secretary of Defence shall contract with the NASEM and submit a study that assesses the risks from AI’s potential use in biosecurity risks.
Section 4.4(a)(ii): The Secretary of Defence shall contract with the NASEM and submit a study that assesses the risks from AI’s potential use in biosecurity risks.
Section 4.4(a)(ii): The Secretary of Defence shall contract with the NASEM and submit a study that assesses the risks from AI’s potential use in biosecurity risks.
Section 7.(b)(i): Encouraging the Directors of the FHFA and CFPB to require evaluations of models for bias affecting protected groups.
Section 7.(b)(i): Encouraging the Directors of the FHFA and CFPB to require evaluations of models for bias affecting protected groups.
Section 7.(b)(i): Encouraging the Directors of the FHFA and CFPB to require evaluations of models for bias affecting protected groups.
Section 8(b)(ii): The Secretary of HHS is to develop a strategy including an AI assurance policy to evaluate the performance of AI-enabled healthcare tools, and infrastructure needs for enabling pre-market assessment.
Section 8(b)(ii): The Secretary of HHS is to develop a strategy including an AI assurance policy to evaluate the performance of AI-enabled healthcare tools, and infrastructure needs for enabling pre-market assessment.
Section 8(b)(ii): The Secretary of HHS is to develop a strategy including an AI assurance policy to evaluate the performance of AI-enabled healthcare tools, and infrastructure needs for enabling pre-market assessment.
Section 10.1(b)(iv): The Director of OMB’s guidance shall specify required risk-management practices for Government uses of AI, including the continuous monitoring and evaluation of deployed AI.
Section 10.1(b)(iv): The Director of OMB’s guidance shall specify required risk-management practices for Government uses of AI, including the continuous monitoring and evaluation of deployed AI.
Section 10.1(b)(iv): The Director of OMB’s guidance shall specify required risk-management practices for Government uses of AI, including the continuous monitoring and evaluation of deployed AI.
AI Model Registries
The US has chosen to actively pursue “compute governance as an entry point” –- that is, it focuses on categorizing and regulating AI models by the compute power necessary to train them, rather than by the use-case of the AI model.
In particular, it has concentrated its binding AI regulations around restricting the export of high-end AI chips to China in preparation for a geopolitical AI arms race.
In particular, it has concentrated its binding AI regulations around restricting the export of high-end AI chips to China in preparation for a geopolitical AI arms race.
In particular, it has concentrated its binding AI regulations around restricting the export of high-end AI chips to China in preparation for a geopolitical AI arms race.
As of Biden’s 2023 Executive Order on AI, there is now a set of preliminary rules requiring the registration of models meeting a certain criteria of compute power. However, this threshold has currently been set beyond the compute power of any existing models, and as such is likely only to impact the next generation of LLMs.
As of Biden’s 2023 Executive Order on AI, there is now a set of preliminary rules requiring the registration of models meeting a certain criteria of compute power. However, this threshold has currently been set beyond the compute power of any existing models, and as such is likely only to impact the next generation of LLMs.
As of Biden’s 2023 Executive Order on AI, there is now a set of preliminary rules requiring the registration of models meeting a certain criteria of compute power. However, this threshold has currently been set beyond the compute power of any existing models, and as such is likely only to impact the next generation of LLMs.
Section 4.2.b specifies that the reporting requirements are enforced for models trained with greater than 1026 floating-point operations, or computing clusters with a theoretical maximum computing capacity of 1020 floating-point operations per second.
Section 4.2.b specifies that the reporting requirements are enforced for models trained with greater than 1026 floating-point operations, or computing clusters with a theoretical maximum computing capacity of 1020 floating-point operations per second.
Section 4.2.b specifies that the reporting requirements are enforced for models trained with greater than 1026 floating-point operations, or computing clusters with a theoretical maximum computing capacity of 1020 floating-point operations per second.
For comparison, GPT-4, one of today’s most advanced models, was likely trained with approximately 1025 floating-point operations.
For comparison, GPT-4, one of today’s most advanced models, was likely trained with approximately 1025 floating-point operations.
For comparison, GPT-4, one of today’s most advanced models, was likely trained with approximately 1025 floating-point operations.
Reporting requirements seem intentionally broad and extensive, specifying that qualifying companies must report on an ongoing basis:
Reporting requirements seem intentionally broad and extensive, specifying that qualifying companies must report on an ongoing basis:
Reporting requirements seem intentionally broad and extensive, specifying that qualifying companies must report on an ongoing basis:
Section 4.2.i.a: Any ongoing or planned activities related to training, developing, or producing dual-use foundation models, including the physical and cybersecurity protections taken to assure the integrity of that training process against sophisticated threats.
Section 4.2.i.a: Any ongoing or planned activities related to training, developing, or producing dual-use foundation models, including the physical and cybersecurity protections taken to assure the integrity of that training process against sophisticated threats.
Section 4.2.i.a: Any ongoing or planned activities related to training, developing, or producing dual-use foundation models, including the physical and cybersecurity protections taken to assure the integrity of that training process against sophisticated threats.
Section 4.2.i.b: The ownership and possession of the model weights of any dual-use foundation models, and the physical and cybersecurity measures taken to protect those model weights.
Section 4.2.i.b: The ownership and possession of the model weights of any dual-use foundation models, and the physical and cybersecurity measures taken to protect those model weights.
Section 4.2.i.b: The ownership and possession of the model weights of any dual-use foundation models, and the physical and cybersecurity measures taken to protect those model weights.
Section 4.2.i.c: The results of any developed dual-use foundation model’s performance in relevant AI red-team testing.
Section 4.2.i.c: The results of any developed dual-use foundation model’s performance in relevant AI red-team testing.
Section 4.2.i.c: The results of any developed dual-use foundation model’s performance in relevant AI red-team testing.
AI Incident Reporting
The US does not currently have any existing or proposed legislation regarding reporting databases for AI-related incidents. However, the Executive Order on AI contains some preliminary language directing the Secretary of Health and Human Services (HHS) and the Secretary of Homeland Security to establish new programs within their respective agencies. These directives essentially request the creation of domain-specific incident databases:
Section 5.2: The Secretary of Homeland Security…shall develop a training, analysis, and evaluation program to mitigate AI-related IP risks. Such a program shall: (i) include appropriate personnel dedicated to collecting and analyzing reports of AI-related IP theft, investigating such incidents with implications for national security, and, where appropriate and consistent with applicable law, pursuing related enforcement actions.
Section 5.2: The Secretary of Homeland Security…shall develop a training, analysis, and evaluation program to mitigate AI-related IP risks. Such a program shall: (i) include appropriate personnel dedicated to collecting and analyzing reports of AI-related IP theft, investigating such incidents with implications for national security, and, where appropriate and consistent with applicable law, pursuing related enforcement actions.
Section 5.2: The Secretary of Homeland Security…shall develop a training, analysis, and evaluation program to mitigate AI-related IP risks. Such a program shall: (i) include appropriate personnel dedicated to collecting and analyzing reports of AI-related IP theft, investigating such incidents with implications for national security, and, where appropriate and consistent with applicable law, pursuing related enforcement actions.
Section 8: The Secretary of HHS shall…consider appropriate actions [such as]...establish[ing] a common framework for approaches to identifying and capturing clinical errors resulting from AI deployed in healthcare settings as well as specifications for a central tracking repository for associated incidents that cause harm, including through bias or discrimination, to patients, caregivers, or other parties.
Section 8: The Secretary of HHS shall…consider appropriate actions [such as]...establish[ing] a common framework for approaches to identifying and capturing clinical errors resulting from AI deployed in healthcare settings as well as specifications for a central tracking repository for associated incidents that cause harm, including through bias or discrimination, to patients, caregivers, or other parties.
Section 8: The Secretary of HHS shall…consider appropriate actions [such as]...establish[ing] a common framework for approaches to identifying and capturing clinical errors resulting from AI deployed in healthcare settings as well as specifications for a central tracking repository for associated incidents that cause harm, including through bias or discrimination, to patients, caregivers, or other parties.
Open-Source AI Models
The US AI Bill of Rights doesn’t discuss open-source models, but the Executive Order on AI does initiate an investigation into the risk-reward tradeoff of open-sourcing. Section 4.6 calls for soliciting input on foundation models with “widely available model weights”, specifically targeting open-source models. Section 4.6 summarizes the risk-reward tradeoff of publicly sharing model weights, which offers “substantial benefits to innovation, but also substantial security risks, such as the removal of safeguards within the model”. In particular: 4.6 calls for the Secretary of Commerce to:
Section 4.6(a): Set up a public consultation with the private sector, academia, civil society, and other stakeholders on the impacts and appropriate policy related to dual-use foundation models with widely available weights (“such models” below), including:
Section 4.6(a): Set up a public consultation with the private sector, academia, civil society, and other stakeholders on the impacts and appropriate policy related to dual-use foundation models with widely available weights (“such models” below), including:
Section 4.6(a): Set up a public consultation with the private sector, academia, civil society, and other stakeholders on the impacts and appropriate policy related to dual-use foundation models with widely available weights (“such models” below), including:
4.6(a)(i): Risks associated with fine-tuning or removing the safeguards from such models;
4.6(a)(i): Risks associated with fine-tuning or removing the safeguards from such models;
4.6(a)(i): Risks associated with fine-tuning or removing the safeguards from such models;
4.6(a)(ii): Benefits to innovation, including research into AI safety and risk management, of such models;
4.6(a)(ii): Benefits to innovation, including research into AI safety and risk management, of such models;
4.6(a)(ii): Benefits to innovation, including research into AI safety and risk management, of such models;
4.6(a)(iii): Potential voluntary, regulatory, and international mechanisms to manage risk and maximize the benefits of such models;
4.6(a)(iii): Potential voluntary, regulatory, and international mechanisms to manage risk and maximize the benefits of such models;
4.6(a)(iii): Potential voluntary, regulatory, and international mechanisms to manage risk and maximize the benefits of such models;
4.6(b): Submit a report to the president based on the results of 4.6(a), on the impacts of such models, including policy and regulatory recommendations.
4.6(b): Submit a report to the president based on the results of 4.6(a), on the impacts of such models, including policy and regulatory recommendations.
4.6(b): Submit a report to the president based on the results of 4.6(a), on the impacts of such models, including policy and regulatory recommendations.
Cybersecurity of Frontier AI Models
Compared to the EU and China, the US Executive Order on AI places the greatest priority on the cybersecurity of frontier AI models (beyond data privacy requirements), in accordance with the US’ developing interest in limiting Chinese access to US technologies. It is developing specific reporting requirements regarding cybersecurity for companies developing dual-use foundation models, and has requests for reports out to various agencies to investigate AI model cybersecurity implications across a number of domains.
Specific regulatory text in the Executive Order includes:
Section 4.2: This section establishes reporting requirements to the Secretary of Commerce for measures taken to protect the model training process and weights of dual-use foundational models, including:
Section 4.2: This section establishes reporting requirements to the Secretary of Commerce for measures taken to protect the model training process and weights of dual-use foundational models, including:
Section 4.2: This section establishes reporting requirements to the Secretary of Commerce for measures taken to protect the model training process and weights of dual-use foundational models, including:
a
Companies developing dual-use foundation models must provide information on physical and cybersecurity protections for the model training process, model weights, and the result of any read-team testing for model security
a
Companies developing dual-use foundation models must provide information on physical and cybersecurity protections for the model training process, model weights, and the result of any read-team testing for model security
a
Companies developing dual-use foundation models must provide information on physical and cybersecurity protections for the model training process, model weights, and the result of any read-team testing for model security
b
Directs the Secretary of Commerce to define the technical conditions for which models would be subject to the reporting requirements in 4.2(a). Until defined, this applies to any model trained using
b
Directs the Secretary of Commerce to define the technical conditions for which models would be subject to the reporting requirements in 4.2(a). Until defined, this applies to any model trained using
b
Directs the Secretary of Commerce to define the technical conditions for which models would be subject to the reporting requirements in 4.2(a). Until defined, this applies to any model trained using
i
Over 10²⁶ integer/floating-point operations per second (FLOP/s)
i
Over 10²⁶ integer/floating-point operations per second (FLOP/s)
i
Over 10²⁶ integer/floating-point operations per second (FLOP/s)
ii
Over 10²³ FLOPs if using primarily biological sequence data
ii
Over 10²³ FLOPs if using primarily biological sequence data
ii
Over 10²³ FLOPs if using primarily biological sequence data
iii
Any computing cluster with data center networking of over 100 Gbit/s and a maximum computing capacity of 10²⁰ FLOPs for training AI.
iii
Any computing cluster with data center networking of over 100 Gbit/s and a maximum computing capacity of 10²⁰ FLOPs for training AI.
iii
Any computing cluster with data center networking of over 100 Gbit/s and a maximum computing capacity of 10²⁰ FLOPs for training AI.
Section 4.3: This section requires that a report is delivered to the Secretary of Homeland Security in 90 days on potential risks related to the use of AI in critical infrastructure sectors, including ways in which AI may make infrastructure more vulnerable to critical failures, physical attacks, and cyber attacks.
Section 4.3: This section requires that a report is delivered to the Secretary of Homeland Security in 90 days on potential risks related to the use of AI in critical infrastructure sectors, including ways in which AI may make infrastructure more vulnerable to critical failures, physical attacks, and cyber attacks.
Section 4.3: This section requires that a report is delivered to the Secretary of Homeland Security in 90 days on potential risks related to the use of AI in critical infrastructure sectors, including ways in which AI may make infrastructure more vulnerable to critical failures, physical attacks, and cyber attacks.
It also requests that the Secretary of the Treasury issue a public report on best practices for financial institutions to manage AI-specific cybersecurity risks.
It also requests that the Secretary of the Treasury issue a public report on best practices for financial institutions to manage AI-specific cybersecurity risks.
It also requests that the Secretary of the Treasury issue a public report on best practices for financial institutions to manage AI-specific cybersecurity risks.
Section 4.6: The Secretary of Commerce shall solicit input for a report evaluating the risks associated with open-sourced model weights of dual-use foundational models, including the fine-tuning of open-source models, potential benefits to innovation and research, and potential mechanisms to manage risks.
Section 4.6: The Secretary of Commerce shall solicit input for a report evaluating the risks associated with open-sourced model weights of dual-use foundational models, including the fine-tuning of open-source models, potential benefits to innovation and research, and potential mechanisms to manage risks.
Section 4.6: The Secretary of Commerce shall solicit input for a report evaluating the risks associated with open-sourced model weights of dual-use foundational models, including the fine-tuning of open-source models, potential benefits to innovation and research, and potential mechanisms to manage risks.
Section 7.3: The Secretary of HHS shall develop a plan [that includes the]... incorporation of safety, privacy, and security standards into the software-development lifecycle for protection of personally identifiable information, including measures to address AI-enhanced cybersecurity threats in the health and human services sector.
Section 7.3: The Secretary of HHS shall develop a plan [that includes the]... incorporation of safety, privacy, and security standards into the software-development lifecycle for protection of personally identifiable information, including measures to address AI-enhanced cybersecurity threats in the health and human services sector.
Section 7.3: The Secretary of HHS shall develop a plan [that includes the]... incorporation of safety, privacy, and security standards into the software-development lifecycle for protection of personally identifiable information, including measures to address AI-enhanced cybersecurity threats in the health and human services sector.
The US does not have a comprehensive data privacy law similar to the GDPR or the PRC Personal Information Protection Law, nor a comprehensive cybersecurity law similar to the PRC Cybersecurity Law.
AI Discrimination Requirements
The US government is actively addressing AI discrimination via two primary initiatives by the executive branch. However, both of these initiatives are non-binding and non-specific in nature: in particular, the Executive Order directs several agencies to publish guidelines, but doesn’t identify any specific requirements or enforcement mechanisms.Specific regulatory text in the Executive Order includes:
1
The AI Bill of Rights contains an entire section on Algorithmic Discrimination Protections. In particular, it emphasizes that consumers should be protected from discrimination based on their “race, color, ethnicity, sex (including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual orientation), religion, age, national origin, disability, veteran status, genetic information, or any other classification protected by law.” Though this bill is non-binding, it sets a general principle for enforcement by the US executive branch for more specific regulations.
1
The AI Bill of Rights contains an entire section on Algorithmic Discrimination Protections. In particular, it emphasizes that consumers should be protected from discrimination based on their “race, color, ethnicity, sex (including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual orientation), religion, age, national origin, disability, veteran status, genetic information, or any other classification protected by law.” Though this bill is non-binding, it sets a general principle for enforcement by the US executive branch for more specific regulations.
1
The AI Bill of Rights contains an entire section on Algorithmic Discrimination Protections. In particular, it emphasizes that consumers should be protected from discrimination based on their “race, color, ethnicity, sex (including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual orientation), religion, age, national origin, disability, veteran status, genetic information, or any other classification protected by law.” Though this bill is non-binding, it sets a general principle for enforcement by the US executive branch for more specific regulations.
2
The Executive Order on AI directs various executive agencies to publish reports or guidance on preventing discrimination within their respective domains within the 90–180 days after its publication. These include the following directly responsible parties:
2
The Executive Order on AI directs various executive agencies to publish reports or guidance on preventing discrimination within their respective domains within the 90–180 days after its publication. These include the following directly responsible parties:
2
The Executive Order on AI directs various executive agencies to publish reports or guidance on preventing discrimination within their respective domains within the 90–180 days after its publication. These include the following directly responsible parties:
a
Section 7.1: “The Attorney General of the Criminal Justice System, and the Assistant Attorney General in charge of the Civil Rights Division will publish guidance preventing discrimination in automated systems.”
a
Section 7.1: “The Attorney General of the Criminal Justice System, and the Assistant Attorney General in charge of the Civil Rights Division will publish guidance preventing discrimination in automated systems.”
a
Section 7.1: “The Attorney General of the Criminal Justice System, and the Assistant Attorney General in charge of the Civil Rights Division will publish guidance preventing discrimination in automated systems.”
b
Section 7.2.b.i: “The Secretary of HHS (The Department of Health and Human Services) will publish guidance regarding non-discrimination in allocating public benefits.”
b
Section 7.2.b.i: “The Secretary of HHS (The Department of Health and Human Services) will publish guidance regarding non-discrimination in allocating public benefits.”
b
Section 7.2.b.i: “The Secretary of HHS (The Department of Health and Human Services) will publish guidance regarding non-discrimination in allocating public benefits.”
c
Section 7.2.b.ii: “The Secretary of Agriculture will publish guidance regarding non-discrimination in allocating public benefits.”
c
Section 7.2.b.ii: “The Secretary of Agriculture will publish guidance regarding non-discrimination in allocating public benefits.”
c
Section 7.2.b.ii: “The Secretary of Agriculture will publish guidance regarding non-discrimination in allocating public benefits.”
d
Section 7.3: “The Secretary of Labor will publish guidance regarding non-discrimination in hiring involving AI.”
d
Section 7.3: “The Secretary of Labor will publish guidance regarding non-discrimination in hiring involving AI.”
d
Section 7.3: “The Secretary of Labor will publish guidance regarding non-discrimination in hiring involving AI.”
AI Disclosures
The Executive Order on AI states that Biden’s administration will “develop effective labeling and content provenance mechanisms, so that Americans are able to determine when content is generated using AI and when it is not.” In particular:
Section 4.5(a): Requires the Secretary of Commerce to submit a report identifying existing and developable standards and tools for authenticating content, tracking its provenance, and detecting and labeling AI-generated content.
Section 4.5(a): Requires the Secretary of Commerce to submit a report identifying existing and developable standards and tools for authenticating content, tracking its provenance, and detecting and labeling AI-generated content.
Section 10.1(b)(viii)(C): Requires the Director of OMB to issue guidance to government agencies that includes the specification of reasonable steps to watermark or otherwise label generative AI output.
Section 10.1(b)(viii)(C): Requires the Director of OMB to issue guidance to government agencies that includes the specification of reasonable steps to watermark or otherwise label generative AI output.
Section 8(a): Encourages independent regulatory agencies to emphasize requirements related to the transparency of AI models.
Section 8(a): Encourages independent regulatory agencies to emphasize requirements related to the transparency of AI models.
The AI Disclosure Act was proposed in 2023, though it has not passed the house or senate yet, instead being referred to the Subcommittee on Innovation, Data, and Commerce. If passed, the act would require any output generated by AI to include the text: ‘‘Disclaimer: this output has been generated by artificial intelligence.’’
AI and Chemical, Biological, Radiological, & Nuclear Hazards
The Executive Order on AI has several sections on CBRN hazards: several department secretaries are required to implement plans, reports, and proposals analyzing CBRN risks and how to mitigate them, and Section 4.4 specifically focuses on analyzing biological weapon risks and how to reduce them in the short-term. In full:
Section 3(k): The term “dual-use foundation model” is defined as AI that, among other criteria, exhibits or could be modified to exhibit high performance at tasks that pose serious risks, such as substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use CBRN weapons.
Section 3(k): The term “dual-use foundation model” is defined as AI that, among other criteria, exhibits or could be modified to exhibit high performance at tasks that pose serious risks, such as substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use CBRN weapons.
4.1(b): The Secretary of Energy must coordinate with Sector Risk Management Agencies to develop and implement a plan for developing AI model evaluation tools and testbeds, and at a minimum, to develop tools to evaluate AI capabilities to generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards and must develop model guardrails that reduce such risks.
4.1(b): The Secretary of Energy must coordinate with Sector Risk Management Agencies to develop and implement a plan for developing AI model evaluation tools and testbeds, and at a minimum, to develop tools to evaluate AI capabilities to generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards and must develop model guardrails that reduce such risks.
4.2(a)(i)(C): The Secretary of Commerce must require companies developing dual-use foundation models to provide continuous information and reports on the results of any red-team testing related to lowering the barrier to entry for the development, acquisition, and use of biological weapons by non-state actors.
4.2(a)(i)(C): The Secretary of Commerce must require companies developing dual-use foundation models to provide continuous information and reports on the results of any red-team testing related to lowering the barrier to entry for the development, acquisition, and use of biological weapons by non-state actors.
4.2(b)(i): Any model that primarily uses biological sequence data and that was trained using at least 1023 FLOPs must comply with 4.2(a) until proper technical conditions are developed.
4.2(b)(i): Any model that primarily uses biological sequence data and that was trained using at least 1023 FLOPs must comply with 4.2(a) until proper technical conditions are developed.
The following points are all part of 4.4, which is devoted to Reducing Risks at the Intersection of AI and CBRN Threats, with a particular focus on biological weapons:
4.4(a)(i): The Secretary of Homeland Security must evaluate the potential for AI to be misused to enable the development or production of CBRN threats, while also considering the benefits and application of AI to counter these threats.
4.4(a)(i): The Secretary of Homeland Security must evaluate the potential for AI to be misused to enable the development or production of CBRN threats, while also considering the benefits and application of AI to counter these threats.
(A) This will be done in consultation with experts in AI and CBRN issues from the DoE, private AI labs, academia, and third-party model evaluators, for the sole purpose of guarding against CBRN threats.
(A) This will be done in consultation with experts in AI and CBRN issues from the DoE, private AI labs, academia, and third-party model evaluators, for the sole purpose of guarding against CBRN threats.
(B) The Secretary of Homeland Security will submit a report to the president describing progress, including an assessment of the types of AI models that may present CBRN risks to the United States and recommendations for regulating their training and use, including requirements for safety evaluations and guardrails for mitigating potential threats to national security
(B) The Secretary of Homeland Security will submit a report to the president describing progress, including an assessment of the types of AI models that may present CBRN risks to the United States and recommendations for regulating their training and use, including requirements for safety evaluations and guardrails for mitigating potential threats to national security
4.4(a)(ii): The Secretary of Defense must enter a contract with the National Academies of Sciences, Engineering, and Medicine to conduct and submit a study that:
4.4(a)(ii): The Secretary of Defense must enter a contract with the National Academies of Sciences, Engineering, and Medicine to conduct and submit a study that:
(A) assesses how AI can increase biosecurity risks, and makes recommendations on mitigating such risks;
(A) assesses how AI can increase biosecurity risks, and makes recommendations on mitigating such risks;
(B) considers the national security implications of the use of data associated with pathogens and omics¹ studies that the government funds or owns for the training of generative AI, and makes recommendations on mitigating such risks;
(B) considers the national security implications of the use of data associated with pathogens and omics¹ studies that the government funds or owns for the training of generative AI, and makes recommendations on mitigating such risks;
(C) assesses how AI can be used to reduce biosecurity risks;
(C) assesses how AI can be used to reduce biosecurity risks;
(D) considers additional concerns and opportunities at the intersection of AI and synthetic biology.
(D) considers additional concerns and opportunities at the intersection of AI and synthetic biology.
4.4(b): To reduce the risk of misuse of synthetic nucleic acids²:
4.4(b): To reduce the risk of misuse of synthetic nucleic acids²:
(i) The director of OSTP, in consultation with several secretaries, shall establish a framework to encourage providers of synthetic nucleic acid sequences to implement comprehensive, scalable, and verifiable synthetic nucleic acid procurement screening mechanisms. As part of this framework, the director shall:
(i) The director of OSTP, in consultation with several secretaries, shall establish a framework to encourage providers of synthetic nucleic acid sequences to implement comprehensive, scalable, and verifiable synthetic nucleic acid procurement screening mechanisms. As part of this framework, the director shall:
(A) establish criteria for ongoing identification of biological sequences that could be pose a risk to national security; and
(A) establish criteria for ongoing identification of biological sequences that could be pose a risk to national security; and
(B) determine standard methodologies for conducting & verifying the performance of sequence synthesis procurement screening, including customer screening approaches to support due diligence with respect to managing security risks posed by purchasers of biological sequences identified in (A) and processes for the reporting of concerning activity.
(B) determine standard methodologies for conducting & verifying the performance of sequence synthesis procurement screening, including customer screening approaches to support due diligence with respect to managing security risks posed by purchasers of biological sequences identified in (A) and processes for the reporting of concerning activity.
(ii) The secretary of commerce, acting through NIST and in coordination with others, shall initiate an effort to engage with industry and relevant stakeholders, informed by the framework of 4.4(b)(i), to develop and refine:
(ii) The secretary of commerce, acting through NIST and in coordination with others, shall initiate an effort to engage with industry and relevant stakeholders, informed by the framework of 4.4(b)(i), to develop and refine:
(A) Specifications for effective nucleus synthesis procurement screening;
(A) Specifications for effective nucleus synthesis procurement screening;
(B) Best practices, including security and access controls, for managing sequence-of-concern databases to support screening
(B) Best practices, including security and access controls, for managing sequence-of-concern databases to support screening
(C) technical implementation guides for effective screening; and
(C) technical implementation guides for effective screening; and
(D) conformity-assessment best practices and mechanisms.
(D) conformity-assessment best practices and mechanisms.
(iii) All agencies that fund life-sciences research shall establish as a requirement of funding that synthetic nucleic acid procurement is conducted through providers or manufacturers that adhere to the framework of 4.4(b)(i). The Assistant to the President for National Security Affairs and Director of OSTP shall coordinate the process of reviewing such funding requirements to facilitate consistency in implementation.
(iii) All agencies that fund life-sciences research shall establish as a requirement of funding that synthetic nucleic acid procurement is conducted through providers or manufacturers that adhere to the framework of 4.4(b)(i). The Assistant to the President for National Security Affairs and Director of OSTP shall coordinate the process of reviewing such funding requirements to facilitate consistency in implementation.
(iv) To facilitate effective implementation of the measures of 4.4(b)(i)-(iii), the Secretary of Homeland Security shall, with consultation:
(iv) To facilitate effective implementation of the measures of 4.4(b)(i)-(iii), the Secretary of Homeland Security shall, with consultation:
(A) Develop a framework to conduct structured evaluation and stress testing of nucleic acid synthesis procurement screening [...];
(A) Develop a framework to conduct structured evaluation and stress testing of nucleic acid synthesis procurement screening [...];
(B) Submit an annual report [...] on any results of the activities conducted pursuant to 4.4(b)(iv)(A), including recommendations on how to strengthen procurement screening.
(B) Submit an annual report [...] on any results of the activities conducted pursuant to 4.4(b)(iv)(A), including recommendations on how to strengthen procurement screening.