US — TechCrunch
Anthropic is working on a system to prove AI models’ identity
AI2 min read
Anthropic, an AI company, is building an open source system called FlashProve. It aims to prove which AI model made a given answer. The idea is to let users check that an output really came from the model they think they are using.
What happened
Anthropic published a blog post on Tuesday about a new system called FlashProve. It is built to give what researchers call "provable inference." This means a cryptographic proof, which is a kind of mathematical guarantee, that an output came from a specific AI model. The system works like an email signature. It lets a receiver confirm that a message really came from the sender it claims. The researchers say it should still work even if the company running the model is the one trying to fool users.
Key facts
- The system is open source, so anyone can read and reuse the code on GitHub.
- AI labs would publish a "fingerprint" for each model they release.
- The fingerprints do not reveal a model's weights, which are the internal numbers that define how the model behaves.
- The system is designed to be lightweight, so it could run on a local device.
Why proof is needed
Users often cannot tell which AI model gave them an answer. A company could secretly swap in a cheaper model to save money. It could also swap in a more expensive model to make its results look better. FlashProve could also check claims by third parties, such as a claim that a model reviewed a piece of software for security problems.
Why it matters
If the system works, people could check AI answers instead of just trusting the company. This could make AI services more honest and easier to audit. It also lets companies keep their model secrets while still proving what they ran.
What to watch
- The research is still in progress and is not a finished product.
- Anthropic says it has no plans to use FlashProve itself or to launch it at any other lab yet.
- Anthropic shared the code so that other researchers can improve it.
Glossary
- FlashProve
- A new open source system that aims to prove which AI model made a given output.
- Anthropic
- An AI company that builds AI models and published the research about FlashProve.