When AI Broke the Law

Authored by Alicia Solow-Niederman

AI is breaking the law. No, not in the literal sense. I’m not arguing that AI has volition or asserting that AI companies are acting illegally. AI development breaks the law in a more fundamental way. The legal structures and doctrines that govern AI systems are failing to operate in clear, consistent, and principled ways, with rule of law consequences.

Because AI today runs on data, how we regulate data affects AI regulation. And because more than one field of law applies to data, data regulation is complicated. Take copyright law and information privacy law: the fields of law overlap in that they both apply to data, yet they have different legal rules and different underlying goals. When the copyright-privacy boundaries blur and the discrete rules and rationales for each domain do not remain sufficiently distinct, the two regimes lose their independent structural integrity and collapse into one another. This is what I call “inter-regime doctrinal collapse” (or doctrinal collapse, for short).

Although “collapse” may sound catastrophic, doctrinal collapse can be a good thing if it enables beneficial innovation or constructive change. But it is bad, I think, when it disproportionately facilitates exploitation by already-established corporate players. In a forthcoming article, I distinguish this regulatory phenomenon from others, such as regulatory arbitrage or regulatory gaps, and detail specific tactics that companies use to acquire data. There, I focus on business-to-business licensing deals (“buy”) and business-to-user agreements in privacy policies and terms of service (“ask”) and how these tactics tend to let the “haves” get ahead.

Here, I focus on a complementary point: doctrinal collapse can also have negative consequences when it impedes law’s ability to constrain the arbitrary exercise of private power. That is exactly what is happening in AI today. When a leading AI company can contend that data is public enough to scrape—diffusing both privacy and copyright controversies—and then turn around and claim that it’s private enough to keep secret—contesting disclosure or impeding oversight of its training data—something has gone terribly awry. Formally, copyright law’s legal rules and incentive-based approach and privacy law’s legal rules and control-based approach are distinct; functionally, they blur.

Even for someone like me who tends to eschew formalism and embrace law’s inevitable ambiguity, there’s a problem: the current structure of law allows companies to switch between copyright law and privacy law arguments, depending on what advances their interests at a particular moment. Corporate opportunism like this of course isn’t new, particularly in the age of informational capitalism. But this sort of doctrinal switching is more than gamesmanship. Analyzing only the result (opportunistic behavior) overlooks the relationship between legal regimes (the inter-regime doctrinal collapse) that enables that result.

Doctrinal collapse comes with rule of law consequences. When sophisticated actors exploit collapse, they manipulate the instability of the two partially overlapping domains. In public rhetoric about AI development as well as generative AI litigation and regulation nationwide and worldwide, it’s increasingly difficult to trace which arguments are being advanced or to assess the legitimacy of those claims. This outcome poses a rule of law problem, because a rule-of-law system requires the rules to be publicly understandable and applied in a consistent, justifiable manner. To be sure, rule of law concerns traditionally focus on state actors and binding legal pronouncements. But when private companies strategically leverage overlapping legal regimes—even through litigation briefs and public rhetoric—law loses its capacity to constrain the arbitrary exercise of power. That’s a rule of law problem.

Pending generative AI litigation reveals how copyright law and privacy law lines become unclear and unpredictable as privacy is strategically minimized, then maximized, in ways that do not cohere over time. Recall that, at the data acquisition stage, AI developers emphasize that data they have scraped to train AI models is public. The public availability of the data is used to make copyright arguments, to maintain that there is no privacy interest in the data used to train an AI model, or both. The common refrain is that the data was voluntarily shared with the company, by making it publicly available online. Later, if the company is one of the dozens sued for alleged copyright infringement, it might again double down on these same arguments about the public nature of data. A developer might also highlight a user’s choice to accept the platform’s terms of service and privacy policy, seeking to rebut any privacy objections on the grounds that data was voluntarily provided and licensed to the company.

But the same company may simultaneously refuse to disclose the training data and claim that the dataset is proprietary, or even advance arguments about privacy to shield that data. Notably, although information privacy arguments don’t feature prominently in the company’s initial public statements or legal briefs, as soon as the plaintiffs demand access to that training data, the very same company may, in the context of mounting a copyright defense, suddenly claim that sharing data would compromise user privacy interests. Privacy didn’t matter before, when the company was training the model, during the deployment of the AI system, or in earlier court filings and public statements—but when it can be strategically invoked to resist discovery, it suddenly becomes a leading argument and even a PR opportunity.

There may well be valid privacy interests here, but my point is more basic: when private actors can choose which doctrines apply, at which times, depending on what serves them, controlling legal regimes lose their consistency and coherence. This outcome makes legal rules less publicly accessible and less easily understandable by members of the public. Put simply, private actors’ toggling makes the law increasingly illegible. If we blink that reality, we will let private actors manipulate copyright law and privacy law to acquire data, and we will miss how power flows in AI development. If we instead recognize these dynamics, then we can confront how doctrinal collapse is happening in AI and turn to the real question: not how to stop doctrinal collapse, but instead how to temper its pernicious consequences.

One auspicious possibility, drawn from conflict of laws, is to look to the courts as front-line managers of collapse. For instance, in a generative AI lawsuit that involves competing copyright and privacy interests and arguments, the adjudicating court might insist on a rebuttable “anti-switching” presumption. As a rough cut, the idea is that a party in litigation cannot assert mutually incompatible claims at different points in the lawsuit, absent a sufficiently compelling reason to defeat the presumption. Because it is a rebuttable presumption, this intervention would require the court to engage in case-by-case, fact specific analysis. This nuanced analysis may be a feature, because it doesn’t lock in any assumption that copyright law or privacy law should always take priority and requires evaluation of the interests as they are presented in a particular legal dispute. An open question, though, is whether it places too many demands on the court, by straining judicial resources or by encouraging improper judicial policymaking.

Whatever the answer, notice how the frame has shifted. That is the key. AI accountability must be an institutional design question that considers which actors matter, and which also accounts for multiple legal domains and the interactions between them—or else AI really will break the law.

Alicia Solow-Niederman is an Associate Professor of Law at The George Washington University Law School. This post is based on her forthcoming paper, AI and Doctrinal Collapse, 78 Stanford Law Review __ (forthcoming 2026).