What is AI distillation? How Claude was allegedly copied

What is AI distillation? How Claude was allegedly copied

Marketing Sideways · AI, explained

What is AI distillation?

Anthropic says Chinese AI companies used distillation to copy its Claude chatbot, secretly passing their own users' questions to it. Here is what the term means, explained with a takeaway shop, and where marketers do the same thing.

A takeaway counter phoning its customers' orders through to a bigger kitchen down the road, the picture this article uses to explain AI distillation.

Anthropic, the US company that makes the Claude AI assistant, says Chinese AI companies used a technique called distillation to copy Claude. One of them, Moonshot AI, the company behind the Kimi chatbot, had more than 23 million exchanges with Claude between May and July this year. An exchange is one question and its answer. AI distillation means copying an AI model by studying its answers, and Anthropic says Moonshot used thousands of fake accounts to do it.

Those figures come from a report Anthropic published on 10 September about how its AI is being misused. Before I explain the technique, a note on what this is. These are allegations. They come from Anthropic's own report, and the Wall Street Journal reported them. Moonshot declined to comment. DeepSeek, another Chinese AI company named in the report, did not respond. China's Foreign Ministry said the US should stop making false accusations and smearing China. The Journal reports that Chinese companies have not denied using distillation in general. Earlier this year, though, a Moonshot executive told local media that its latest model, K3, performed well because of the company's own innovations, and denied it came from distillation or copying. So read what follows as Anthropic's account. My job here is to explain the technique, because you are going to hear the word a lot.

The copier learns from the answers and skips the cost.

How AI distillation works

An AI model is the program behind a chatbot like Claude. A company builds one by training it on a huge amount of text, which costs a great deal of money and time. The training produces a set of internal settings that decide how the model answers. Think of those settings as a secret recipe. The company keeps the recipe locked in the kitchen. Customers only ever see the plates, which are the answers that come back when they type a question.

Now picture a rival chef who wants to copy a famous dish. He cannot get into the kitchen to read the recipe. So he orders the dish thousands of times and asks for a change each time: extra hot, half size, no onions. Each plate shows him how the kitchen handles a different request. He studies every plate and trains his own cooks to make something very close. Everything he learns comes from the plates.

Distillation works the same way. One company sends another company's AI model millions of questions and records every answer. It then trains its own model on those questions and answers until its model responds the same way. The name comes from cooking and brewing. To distil something is to boil it down to its strongest part. Here, the skill of a big model gets boiled down into a new one.

AI companies distil their own models all the time, and that is allowed. They boil a large, expensive model down into a smaller one that is faster and cheaper to run. What US AI companies generally ban is an outsider doing it to their models. The reason is cost. The original company paid for the training. The copier skips that cost and only pays to ask questions. US officials compare it to theft on an industrial scale and say rivals use it as a shortcut to catch up.

How the middlemen passed questions to Claude

Because US companies ban this, the copier has to hide who is asking. Anthropic says the Chinese companies did that through go-between services it calls transfer stations. A transfer station takes a question from one company and sends it on to Claude. Anthropic says these services operate outside China and often use stolen or fraudulently obtained accounts, which hides who is really asking.

In kitchen terms, you walk into a takeaway and order. The takeaway says it cooks everything itself. Behind the counter, it phones your order through to the famous restaurant down the road, collects the food, and serves it to you in its own packaging. Then it writes down what you ordered and what came back. Every order teaches it a little more about how the famous restaurant cooks.

Anthropic says this is what happened with Kimi. A person typed a question into Kimi. Kimi sent it through a transfer station to Claude. Claude answered. Kimi gave that answer to its user and kept a copy of the question and answer to train its own model. As far as the user could tell, Kimi had answered. According to the report, the user had "no way of knowing" their question had gone to Claude.

Jacob Klein, Anthropic's head of threat intelligence, said it would be a large scandal if Anthropic or another US AI company treated its customers this way.

Every order slip carried something personal, and the customer had no view of where it went next.
US lawmakers have introduced a bill that would make it easier for AI companies to act together against distillation, with less risk under competition law.

Users' private information went to Claude too

This is the part that matters most for anyone running a business. The questions passed along were real questions from real people. People put real information into chatbots, so that information travelled too.

Anthropic's report gives examples. One Kimi user asked it to analyse surveillance data on a single person, taken from hundreds of CCTV cameras in the Chinese city of Chengdu. Moonshot sent that data to Claude. An engineer using Kimi to build software for a Chinese company shared login details for several companies. Those logins went to Claude as well. Anthropic says DeepSeek passed on sensitive customer information in the same way, and that the questions passed along included people's locations and passwords.

In kitchen terms, your order slip had your home address and the code to your front door written on it, and the takeaway handed it to someone you had never met.

7 China-based AI companies that tried to learn Claude's abilities through distillation over the past seven months, according to Anthropic's report.

So Anthropic is describing a pattern across several companies.

Marketing does a version of this every day

Distillation sounds like an AI problem. Marketers have always copied each other, and most of it is fair.

Copywriters keep swipe files, which are folders of other people's best ads, emails and headlines, kept to learn from. Agencies sign up to a competitor's mailing list to study how its emails are written and timed. Brands buy a rival's product to see how it is packed and priced. Anyone can open Meta's Ad Library and see the ads a competitor is running on Facebook and Instagram right now. All of this is studying what a competitor has made public. It is the marketing version of ordering the plates, and it is how most of us learned the job.

Anthropic's account describes three things that go past fair study. Marketing has its own version of each.

Line one is how you get the material. Anthropic says Moonshot used thousands of fake accounts. The marketing version is setting up fake accounts, using software to copy a website when its terms forbid it, or pretending to be a customer to get a competitor's price list.

Line two is passing someone else's work off as your own. Kimi gave its users Claude's answers and presented them as Kimi's. The marketing version is the landing page copied line for line, or a competitor's campaign lifted and rebranded. Plenty of marketers have done it.

Line three matters most: passing on someone's information without telling them. The marketing version is the agency that sells you "our in-house team" and then sends your brief, your customer list and your logins to another business you have never heard of. Handing work to another business in this way is called white-labelling. It is common, and it is fair when the client knows. When the client is kept in the dark, their information ends up with a business they never agreed to share it with. That is the same problem Anthropic describes with Kimi's users.

The same applies to AI tools. Many AI writing apps and chat assistants run on a model made by a bigger company. That is normal when the app tells you which company it uses. Kimi's users, according to Anthropic, were never told.

Three things to check this week

Find out which AI sits behind each tool you use. Before your team pastes client material into a chatbot or writing app, check which company's model it runs on and where your information is sent. Look in the privacy policy, or in the terms that explain how the app handles data. If you cannot find it, email the provider and ask. If the answer is vague, keep sensitive work out of that tool.

Keep passwords and client details out of chatbots. In Anthropic's example, an engineer typed company logins into Kimi, and those logins ended up with Claude. Share access to accounts through a password manager instead. Before you paste anything into a chatbot, remove names, addresses and account details. Assume anything you type could be passed to another company.

Get your agency's subcontracting in writing. If you hire an agency or freelancer, ask them in writing whether any other business will see your brief or your data, and require them to tell you before they bring anyone new in. If you run an agency, give your clients that answer before they ask. Check your current contracts for that clause this week.

If you cannot say where your data goes, assume someone else is reading it.

Sources. The Wall Street Journal, "How Chinese AI Firms Tried to Clone U.S. AI Models", Robert McMillan and Amrith Ramkumar, 10 September 2026. Figures and examples from Anthropic's threat report, published 10 September 2026. Statements from Michael Kratsios, Scott Bessent, Moonshot AI, DeepSeek and China's Foreign Ministry as reported by the Wall Street Journal. All claims about Moonshot AI and DeepSeek are allegations made by Anthropic.

Mashed Avocado · Marketing Sideways · MashedAvocado.com

Back to blog

Leave a comment

Please note, comments need to be approved before they are published.