Notable finds

Big news on the AI regulatory front out of California—Governor Newsom vetoed the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, which had been poised to be the most significant U.S. genAI regulation. Though the bill had sailed through the California legislature, it was highly controversial within the technology community, drawing high profile supporters and detractors.

Although the Act was not signed into law, it's valuable to examine its approach and the intense debate surrounding both this bill and genAI safety regulation in general.. The law presented a multi-pronged approach to genAI regulation, as generally summarized by WilmerHale as follows:

Another key feature of the Act was that its provisions only applied to “covered” AI models; defined according to the compute power and cost employed to train the model. At its inception, its regulations would have only applied to the largest frontier models.

Governor Newsom vetoed the bill on Sunday. In his veto message, the Governor acknowledged the need for AI regulation, but argued that SB 1047 was not the right approach. He expressed concern that the bill's focus on large, expensive AI models could create a false sense of security, while smaller models with equal or greater potential for harm could emerge unchecked. Essentially, the Governor argued that the bill's method of identifying potentially dangerous AI systems—based on the cost and computing power required for their development—was flawed and could stifle innovation without effectively mitigating risk. 

The Governor’s criticisms echo those raised by various technology industry stakeholders that had opposed the Act. A representative example is Anderson Horowitz partner Anjnay Midha, who had extensively criticized the proposed bill prior to the Governor’s veto. Midha, like other critics, argued that the Act's reliance on computing power as a measure of a model's potential danger was arbitrary. He noted that “as algorithmic efficiency improves, computing costs decline, and models that take far fewer resources . . . will match the capabilities of a [model exceeding the compute threshold to be regulated under the Act] of today within a fairly short time frame. So this threshold would quickly expand to cover many more models than just the largest, most cutting-edge ones being developed by tech giants.”

However, this concern had already been largely addressed by an amendment to the Act that limited its scope to models that both exceeded the compute threshold AND cost more than $100 million to train. Further, the bill also provided for the compute threshold for covered models to be redetermined after 2027.

Midha also argued that the Act overemphasized the hypothetical "existential risks" of AI while neglecting more pressing concerns related to AI security. Instead of focusing on preventing AI systems from autonomously causing harm, he argued for prioritizing efforts to combat malicious actors who could exploit AI for harmful purposes, such as cyberattacks and misinformation campaigns. In line with this focus, he argued, “the core issue is that [the Act is] trying to regulate the model layer itself, rather than focusing on the malicious applications or misuses of the models.”

The 'core issue' Midha identified – whether to regulate AI models themselves or their applications – is a matter of vigorous debate.. Midha, Governor Newsom, and others have suggested that regulation should not target the entities creating underlying AI models, but rather specific applications of such models, whether by intermediate vendors, end users, or the model creators themselves. Supporters of the Act disagree sharply with this redirection of regulatory focus.

As one commentator forcefully put it:

This strikes me as a powerful rejoinder. The alternative regulatory strategy proposed by Midha and Governor Newsom moves from regulating a small group of large, highly visible, organizations to a far more fragmented group that includes lawbreakers, ephemeral entities, and out of jurisdiction actors. It also seems that many of the harmful end uses are likely covered by existing law (e.g., it is already illegal to defraud someone, whether you are using AI or not), and to the extent that AI-specific regulation is needed, it could easily exist as a complement alongside model-level regulation.

Several other points were raised by observers disappointed by the veto. For example, the highly respected AI researcher Yoshua Bengio offered his defense of the Act. Bengio considered the Act to be a "light touch" approach to regulation, outlining the minimum requirements for safe development of powerful AI systems. He highlighted the Act's focus on self-assessment of risks and basic safety testing, arguing that these measures would not stifle innovation, particularly for startups and smaller companies, due to the bill's focus on large, expensive AI models. He also pointed out that AI model builders carrying some degree of regulatory burden would simply place them in the same position as organizations across industries with safety implications.

Some opponents of the Act had suggested instead that it forced model developers into impossible positions of having to “prove” that their models were safe, or to shut down models outside of their control. However, these critics often overstated or misstated the Act’s requirements. The Brookings Institute offered a mostly persuasive response to these arguments, focusing on how the Act’s multiple amendments addressed these concerns.

Following the Governor’s veto, the Act’s author issued also a responsive press. In addition to referencing some of these arguments, the state Senator pointed to the broad consensus that genAI regulation is needed:

With the keystone state AI Act vetoed, the path forward is unclear for regulatory solutions to these safety concerns. Even the Act’s detractors broadly agree that rapidly advancing AI technology creates powerful risks to be mitigated. But this does not translate into any consensus on the correct approach for governments to take.

In other news, OpenAI released a “preview” version of its new o1 model. Premium users can access the preview model on ChatGPT (with severe weekly usage caps, so pick your prompts carefully!). The article above provides an overview of the new model, how it differs from other GenAI models, and a few areas where it really shines.

As described by OpenAI itself, the “o1 series models are new large language models trained with reinforcement learning to perform complex reasoning. o1 models think before they answer, and can produce a long internal chain of thought before responding to the user.” This results in dramatic improvements in some areas, like mathematics, but for other tasks, like writing, the new model performs no better than the existing GPT-4o model. Accordingly, “o1 models offer significant advancements in reasoning, but they are not intended to replace GPT-4o in all use-cases.”

This raises the question: which legal tasks, if any, can o1 handle more effectively than previous models?

The graph below has been making the rounds, demonstrating the o1 models large advances in math, science and coding:

But another diagram from the same article includes some very intriguing legal benchmarks:

As you can see, this graph shows a dramatic increase on LSAT performance by the new model, as well as notable improvements in law (as well as global facts and formal logic) on a popular benchmark (The Massive Multitask Language Understanding or “MMLU” benchmark is a framework that evaluates how well AI models understand and generate language across a wide range of subjects).

Aside from these benchmarks, some legal vendors and commentators have suggested that the o1 models represent a significant step forward in genAI for legal services. Legal AI vendor Spellbook highlighted a couple areas in particular where o1 is expected to improve its products:

In my own initial experimentation with o1, I have observed a notable (but not massive) improvement in general legal analysis and brainstorming support. II haven't observed significant improvements in addressing hallucination issues or enhancing writing quality – two critical factors that currently limit genAI's usefulness in litigation-related research and writing tasks. However, it is very early days for the new model, and this is just a very quick first impression. I would recommend that anyone with ChatGPT Premium experiment with prompting o1-preview vs. GPT-4o with various tasks, and comparing the outcome.

Shameless self promotion

Recently, I had a great time serving on a panel at the State Capital Group’s annual meeting in Los Angeles. I absorbed a lot more knowledge than I delivered in that discussion, as I was joined by Netflix’s in house counsel for legal operations and technology, an accomplished Canadian cybersecurity attorney, and the Partner leading the technology sector at a large UK law firm.

Overall, I’ve really been enjoying a more varied slate of speaking opportunities over the past couple months, as I’ve developed presentations specific to litigators, government lawyers, ADR professionals, and public interest attorneys. I’m looking forward to continuing to dive into different practice areas! Next week, I’ll be giving a guest lecture at Professor Nelia Robbi’s excellent Beyond the Billable Hour: Board Service and Business Development class at my alma mater, UT Law. Over the next couple weeks I’m keeping that streak going. On October 15, I’ll be co-presenting a CLE on Using Generative AI in Family Law with Katie Valle—registration is open, come join us!