Data Scraping for AI Training: Legal and Ethical Considerations in 2026

Scraping data to train AI models sits in genuinely unsettled legal territory in 2026. US courts have generally treated the training process itself as fair use, but how the data was acquired (bypassing paywalls, ignoring access controls, scraping personal data without a legal basis) is where companies are actually losing cases and paying settlements.

Why This Became a Boardroom Problem, Not Just a Legal One

Eighteen months ago, most conversations about AI training data were happening in engineering standups: which sources to crawl, how to structure the pipeline, how much data was enough. In 2026, the same conversation is happening in board meetings, because the financial exposure has stopped being theoretical.

Anthropic’s roughly $1.5 billion settlement with a class of authors over training data sourced from an unlicensed archive was the moment this became unavoidable. It wasn’t a ruling that training itself was illegal; it was a settlement over how the underlying works were acquired. That distinction is the single most important thing to understand about the current legal landscape, and it’s the one most casual coverage of this topic gets wrong.

Where US Courts Actually Stand Right Now

The pattern across the major US rulings so far is more specific than “AI training is legal” or “AI training is theft.” It breaks down into two separate questions that courts are answering differently.

Training on lawfully acquired data: trending toward fair use

Where a model was trained on data the company legitimately possessed, courts have generally accepted the argument that training is transformative and falls under fair use. Legal commentators tracking these cases describe this as the clearer, more settled half of the picture.

How the data was acquired: where companies are actually losing

The harder problem is acquisition. Courts have drawn a real line between training on data you obtained properly and training on data pulled from a shadow library or scraped in a way that got you sued separately. This is sometimes called the “scraping plus” theory: the practices AI companies use to acquire data often go further than what a court would call fair use, even if the training step downstream is fine.

A newer and faster-growing version of this fight uses the DMCA’s anti-circumvention provisions instead of copyright infringement directly, with plaintiffs arguing that a company circumvented access controls (like YouTube’s systems) to scrape content, regardless of whether the underlying copyright claim would succeed. Reddit’s cases against Anthropic and Perplexity opened this door in 2025; YouTube creators have since brought near-identical claims against Nvidia, Snap, Meta, and Apple. It’s quickly become the preferred legal theory precisely because it sidesteps the messier fair-use debate entirely.

One clarifying ruling worth knowing: a federal court held that robots.txt files are requests, not technological access controls, so ignoring them isn’t a DMCA circumvention violation on its own. That doesn’t make ignoring robots.txt a good idea; it just means the legal exposure for doing so runs through other theories, including breach of contract and the DMCA claims above, not this particular one.

The EU Adds a Compliance Layer, Not Just a Legal Risk

If US litigation is about what already happened, the EU AI Act is about what you’re required to document going forward. General-purpose AI model providers have been under transparency obligations since August 2025, including publishing a summary of training content and demonstrating a copyright compliance policy. The next major deadline, August 2, 2026, brings full conformity assessment requirements for high-risk AI systems, with fines that can reach 7% of global annual turnover for the most serious violations.

The part that catches companies off guard is that the AI Act doesn’t replace GDPR; it sits on top of it. Any training process that touches personal data still needs a valid GDPR legal basis, and high-risk systems processing personal data typically need a documented data protection impact assessment before deployment, separate from AI Act obligations. Treating these as one compliance project instead of two is where most gaps show up.

India’s Own Test Case Is Still Being Decided

For businesses sourcing or providing data in India, ANI Media v. OpenAI at the Delhi High Court is the case to watch: the first generative-AI copyright dispute heard in an Indian court. After 32 hearings between late 2024 and March 2026, Justice Amit Bansal reserved judgment in April 2026. The core issue is whether training ChatGPT on ANI’s news content falls under Section 52’s fair dealing exception, which was written for private study, criticism, and news reporting, not for training a commercial AI system.

The gap that makes this case matter beyond ANI itself: unlike the EU, India has no text-and-data-mining exception in its copyright law. That means Indian courts don’t have a ready-made carve-out for AI training the way European courts do, and the outcome here will likely shape how Indian companies, and any company processing Indian-sourced data, approach data provisioning for AI going forward. Separately, India’s IT Act, 2000 already treats scraping of publicly available user data by intermediaries as a regulated activity, which matters for any pipeline touching user-generated content.

Legal Isn’t the Same as Defensible: the Ethics Layer

Even where a scraping practice would likely survive a court challenge, that’s a low bar for a business decision. Three patterns show up repeatedly in the cases above, and all three are avoidable regardless of how the underlying legal question eventually gets resolved:

• Circumventing paywalls or access controls to reach content that was deliberately restricted, which is the fact pattern driving most of the DMCA claims, independent of copyright merits.
• Collecting personal data without a documented legal basis, which is a GDPR problem even when the copyright question is a non-issue.
• Ignoring a site’s stated terms of service or robots.txt directives as a matter of routine, rather than as a deliberate, reviewed exception. This is a pattern plaintiffs increasingly cite as evidence of intent, even in claims where robots.txt itself isn’t the legal hook.

What This Means for How You Source Training Data Today

None of the above is legal advice, and the law in this area is still moving, but a few practices hold up regardless of how individual cases resolve:

• Keep a documented record of what was collected, from where, and under what terms. This is now a legal requirement under the EU AI Act’s data governance rules for high-risk systems, not just good hygiene.
• Treat robots.txt and terms of service as signals to review, not obstacles to route around by default.
• Screen for personal data before it enters a training set, and have a documented legal basis for anything that isn’t filtered out.
• Prefer licensed, consented, or first-party data sources where the use case allows it, and reserve broad scraping for genuinely public, non-personal data.
• Build provenance into the pipeline itself rather than trying to reconstruct it after a regulator or plaintiff asks.

How DataSOS Approaches This

This is the reasoning behind our compliance-first approach to web scraping and data extraction, respecting robots.txt where required, screening for personal data, and building an audit trail into every pipeline rather than treating compliance as a separate step. For teams evaluating a data partner for AI training use cases specifically, the same questions we’ve written about for choosing the right web scraping provider apply, with legal defensibility now weighted even more heavily than it was two years ago.

We’ve built data pipelines across 30+ industries that hold up under exactly this kind of scrutiny, including regulatory-facing work like automated certificate retrieval under NYC Local Law 33 and commission-tracking systems for energy retail compliance. See the full breakdown in our case studies, or talk to our team about a data sourcing strategy built for AI training use cases from the start.