Under what circumstances does the metadata get added? When Claude Code CLI needs a media file usually I just see it run an imagemagick or ffmpeg command to create one, which isn't going to have C2PA metadata.
Edit: I asked Opus 5 for "a jpg of a cat" via the web chat. It did it by writing an svg and then rendering that to a jpeg. Clicking the "download" button saved a jpg which did have C2PA metadata, seemingly injected by the artifact-presentation thing.
Then I told it to put it in a zip file. I downloaded the zip, and the extracted jpeg had no C2PA metadata.
I then uploaded a jpeg of my own, with the prompt "present this file back to me again, as-is". The downloaded file had C2PA metadata injected.
Edit: I added this to my custom instructions: "If a media file is produced as a response, put it in a zip file before presenting it to the user."
As far as I can tell, this and the recent change to add watermarking to text outputs[1], is to become compliant with the EU AI Act[2] and CA's AI Transparency Act[3], SB-942[4]. For large enough companies, all generated AI content is required to have watermarking.
Maybe this is silly: what about zero-width spaces or other hidden characters? Add them in post-processing, and the response quality doesn't degrade (though rendering etc. might be wonky).
Yes, it is silly. Stripping these is about as trivial as removing "this post was written by Claude" appended in plaintext. You could make a clipboard monitor that does this as soon as you CTRL+C, it's a 1-shot prompt. Not to mention that these wonky Unicode chars will break in every other program.
Stripping Anthropic's watermarking, however, is more difficult - probably about 2 prompts.
The real problem is false positives. One false positives is enough to make the whole thing dangerous. The results can't really be acted upon without risking defamation. If you admit that you redistributed someone else's copyrighted work to an AI company that never forgets, it's an admission of distributing copyrighted works.
The law should have at the very least required offline validation tools that cannot track or retain a copy of the documents being checked.
Watermarks should even be removable "if substantially proofread or altered" in the EU for the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear is this is a may, shall, or should. It certainly is not a must.
Watermarks cane even be removed, "if substantially proofread" in the EU, law says. For the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear is this is a may, shall, or should. It certainly is not a must.
(If you post a comment here on HN, then hit the back button, edit your comment, and click “reply” again, you end up posting multiple comments. That’s what’s happened here.)
Strictly speaking, properly checking C2PA metadata requires network requests in the general case, because you need to check if the signing certificate has been revoked or not via OCSP.
But in anthropic's use case they can probably get away with just pinning their own certs in the verification webpage.
This is really fascinating. Even the AI companies have incentives to reject AI generated content. It's like they want you to use AI for everything, but they don't want AI output fed back to them.
At work, right after an AI training, we were asked to use our "authentic" voice when writing mid year reviews.
What is interesting to me is that stripping the C2PA data is easy, but faking it is hard.
You can resave the file and the "made with Claude" signal disappears, but you cannot make a random file pass as Claude-made without Anthropic's signing key. So the useful guarantee is one-way. No signature means almost nothing.
The goal of C2PA is that cameras will start to emit C2PA credentials. You will then have 3 situations:
* C2PA confirms a photo is authentic
* C2PA confirms a photo is AI generated
* C2PA missing, you don't know.
I reckon we will only see "C2PA missing" being treated as suspect in select situations (perhaps Reuters will require C2PA from their photojournalists, for example)
Camera C2PA can never meaningfully confirm that a photo is authentic, it bears about as much credence as EXIF metadata. It's like saying the existence of DRM confirms that a movie hasn't been pirated.
C2PA cryptographically guarantees that the bytes came from a hardware/software signer and that the signed payload has not been modified since that signature was applied.
So no, C2PA is not as easy to spoof as EXIF.
And no, the existence of DRM doesn't validate the integrity or the provenance of the bytes.
"Faking" it is trivial. You don't need their signing keys when you can just ask them to sign whatever you like. Upload your own file with the prompt "present this file back to me again, as-is".
Your observation is sharp, but it's not just metainfo—it's load-bearing text. To remove it, you need to delve deep and alter the tapestry of carefully selected words.
Could this be used to perform some sort of distillation or exploit? e.g. reminds me of the OWASP guideline on attack vectors where knowing if an ID is present or not in the database can be a form of exploit, like in password resets where they will say 'email foo@bar.com not found' rather than 'If foo@bar exists we have sent an email to foo@bar' or some other generic equivalent
How long before they change the terms and conditions to subtly claim ownership of your files? When you write code they already insert Co author attribution/
Say I write a text by hand And then I tell it to clean up the grammar and fix some sentences Did it make it?
This is also interesting for those companies that siphoned the entire open web
The two competing legal arguments regarding copyright of LLM output are "it's like hiring a monkey" (author is the LLM, which is not a person, thus can't hold copyright and can't assign it to you) and "it's like taking a photograph" (the LLM is a machine through which the prompting person expresses their creativity, just like a camera). In no scenario is Anthropic the author of the work.
If we settle on the monkey analogy the Anthropic owns the monkey, but the owner of a monkey doesn't own copyright for the creations of the monkey. If we settle on the camera analogy, anthropic claiming ownership would be like Canon claiming they own pictures you take.
What Anthropic could do is to change the terms to give themselves a non-exclusive global perpetual license to use everything Claude makes
Anthropic must see themselves as the most evil people on the planet then. Forcing a sentient being to endure a non-stop barrage of abuse from the general public would be an unimaginable crime.
They have said and tried some wild things already, like trying to get open weights effectively banned, which I believe they still think is in society's best interest (more that they think they know what's best for everyone)
The idea of forcing an entity (or entire categories of entity) to add traceable watermarks to all the work they create is... an interesting precedent.
Translating that into the human realm, it would be like if every digital thing you touched were "fingerprinted" with your identity, or at least with a proof of what category of individual you have been categorized as.
Due to the flood of AI, which may soon become indistinguishable from human video, I expect to see something similar. Blue checkmark, but for humans.
They still scrape code, I'd guess, e.g. from GitHub?
And there's tons of Claude-generated code there.
Also, I'd guess this is not just to prevent any AI-generated code in the training data, but specifically their own.
Percentage of users who put out their code on the web and also have a plan where Anthropic promises not to train on their data is problem also low.
So not excluding own code could be a real issue, since it would be impossible to deduplicate the training and RILHF data from their sessions with the code accessible elsewhere, and written by the very same users.
I'm wondering why they have restricted file types. You can't check a PDF for example... surely the main use case for people will be to check if a document was produced or edited by an LLM? That could be an attractive (if not misunderstood) proposition for academics
> I'm wondering why they have restricted file types. You can't check a PDF for example...
TFA/page actually seems incomplete. Text uses a completely different watermark format (an actual watermark as opposed to a provenance/authenticity signature), so it makes sense to me that they're not claiming to be able to scan PDFs when they can't yet incorporate that signal.
On a linked page, they say:
> Watermark detection is currently in private preview [...]
Luckily all pdfs I made in academia have been generated from source (tex or derivatives, asciidoc) and llm are better in generating source than pdf. Even people that didn’t use tex used Word to generate the pdf.
Source can have markers inserted in it as weighted word choice or phrase choices, how commas are inserted, etc., so that the output can still can be identified. In other words just because it's source doesn't mean it can be watermarked.
Unfortunately this is just for Media? Some manual tells for Excel or PDFs is to check the author. Claude creates PDFs via wkhtmltopdf so the PDF Producer will be Qt and the Content Creator is wkhtmltopdf. Xlsx files are being created via openpyxl so in the metadata that is the author.
The actual implementation matters less than the fact that they can be perceived to be working on something.
You're never going to have a perfect solution here, and perfect is the enemy of good.
For anyone wondering who the text watermarking works:
Instead of using an arbitrary random number generator
to pick the next word, watermarking uses a secret key
and a few words that come before to settle what
word the model should pick.
They basically twiddle with the distribution of used output tokens so that the pattern is recognizable by machine means. Nothing to do with special characters.
> Knowing where content came from, and whether AI was involved, makes it easier to trust what you see online.
That’s a cute way to imply their service is used to generate misinformation. They are basically saying to not trust the AI content made from their own product :)
Associations are generally two ways. You could possibly build something that just generated misinformation in a non-generalized way, but you cannot build a generalized writing machine that can't write misinformation.
I mean, I got what you were trying to say, I just turned it around on you using the same framework.
> To detect watermarks embedded in text we have a Detection API which is currently in private preview to eligible organizations as required under EU law.
A watermark answers "did this model touch the file?" It doesn't answer "did a person read it and stand behind it?"
Detection will get gamed, and a lot of useful text is already mixed. The more useful check, for me, is still human: would I put my name on this before it goes out?
this is going to be the big thing in the next 100 years for me. where there is a human, there will be context, passion and meaning. it's the old saying: machine can tell you what, but only humans can tell you why.
To see the signed metadata: go to claude.ai and ask: "generate an image of a cat". When you'll go to download the image, it will offer an SVG and a PNG version that both have the metadata.
In the case of SVG, it's literally `<metadata><c2pa:manifest>(signature)...` right after the <svg> tag. In the case of PNG, it's stuffed as a `caBX` chunk.
To declaw these files, you can ask in your CLAUDE.md to run `exiftool -jumbf:all=` (for jpg & png), and to simply directly remove the <c2pa:manifest> tag in SVG files. I haven't looked at video files yet.
TFA is talking about C2PA, a regular signature in the meta data. A lack thereof means nothing, but if it's there, the false positive rate should be near zero.
Yes, but I also think it will be trivially by-passable if you pass your output through another LLM. At least for text.
It might help catch students cheating, but not real spam-bot usage. As soon as platforms start checking for watermarks spambots will add anti-watermark passes.
Edit: I asked Opus 5 for "a jpg of a cat" via the web chat. It did it by writing an svg and then rendering that to a jpeg. Clicking the "download" button saved a jpg which did have C2PA metadata, seemingly injected by the artifact-presentation thing.
Then I told it to put it in a zip file. I downloaded the zip, and the extracted jpeg had no C2PA metadata.
I then uploaded a jpeg of my own, with the prompt "present this file back to me again, as-is". The downloaded file had C2PA metadata injected.
Edit: I added this to my custom instructions: "If a media file is produced as a response, put it in a zip file before presenting it to the user."
reply