I appreciate the spirit of this, but this is likely to be defeated by anyone with some basic technical understanding of how training works. For example a denoising autoencoder will probably be sufficient.
Hi, one of the founders here, "robustness" is our next step. We are currently training a few models based on cross attention and invariant domain learning that promise to be quite resistant to noising and denoising. The purpose of fingerprint watermarking as we have released it is to act like a signature for validity, some companies already do this but none of them have public products that you can just use. The nest encoder and decoder will use a unique signature for every user. The limitation of Glaze and models like it is that they target specific models/approaches. That being said we are also running experiments on adversarial watermarking that aim to deliver the same image quality as glaze with less computation and more resistance. We want to secure digital identity and copyrights. Watermarking is our initial approach.
Glaze techniques might be specific to a model but the techniques to defeat it are not.
More specifically, I don’t see any reason to believe that I cannot learn a function from A to B where A is the distribution of watermarked files and B is the distribution of non-watermarked files. It’s a trivial exploit.
I'm still trying to understand how a watermarking method may resist simple methods as (1) upscaling and downscaling plus noise, or (2) taking a photo of the image plus compression plus noise, (3) any simple convolutional filter or a combination of those.
1) With an invariant domain representation, 2) and 3) I am not certain yet, but I will find out.
Although when it comes to signing with a watermark for validity you would not want the id (50 bits) to be preserved or copied. These are separate use cases / applications.
Interesting company, and something that is surely good to watch out for in the future. It is also very needed use case now that generative AI is just about everywhere.
There is some cause to be skeptical of the claims on the website, because the site doesn't provide enough information to validate the claims with any rational basis, and there are a number of other companies that have done the same and failed to deliver.
There have also been a few court cases now where similar use cases for attribution have been used inappropriately, and resulted in false claims/charges causing great miscarriages of justice (under facial recognition).
Hopefully you performed an adequate adversarial review, and have a demonstrable way to rationally prove there are no false positives at a bare minimum.
Hi there, we have published a demonstration of fingerprint watermarking on audio and images that anyone could just try. In our next phase we will release our first consumer product that will provide value. We have conducted thorough testing on our fingerprint encoding and decoding. We will for sure aim to explain this better when we give users the ability to encode unique signatures.
If there are any other particular claim you have not found enough information about then I am happy to elaborate.
Could you by any chance link any of those court cases?
The major concern I had just taking a cursory look revolved around the wording of 'unique' signatures. This word is often misused in an out of context way that makes the claim false.
When used it has a very specific meaning in math, but many times the claim being made is actually an improper equivalence being made (for marketing).
For example, they map some inputs to a finite field that may roll over, which isn't disclosed. This is known to violate the 1:1 map required for a 'unique' property between input and output, excepting some very rigorous methodology and forcibly limited system's/environments.
Uniformity of the collisions in such systems is also a very big problem, sometimes they clump, but absent bruteforce checking the entire space there's no way to predict ahead of time when clumping will happen. Similar equivalences have been made in the crypto space, and shown to be false when those systems were rigorously broken later. The existence of collisions given same inputs is proof by contradiction the input->output pair is not unique and fails.
I've linked one of the cases with a brief gist below (since Justia doesn't provide a rundown).
Harvey Eugene Jr. was arrested based on a false match for a robbery of a Sunglass Hut in Texas (owned by Macy's). He lived at the time in Sacramento CA, he was arrested when he returned to renew his Texas driver's license; during holding he was raped by three other inmates leaving him with permanent debilitating injuries. He was at the time of the robbery provably living in Sacramento. His arrest was based solely on the false positive AI based facial recognition match.
There have been many news articles, and several cases, a simple search for "facial recognition false arrest" on google should provide a number of articles.
You may also find a few on the ACLU/EFF site as they have a keen interest in going after companies who violate civil rights; their website has a history of some of the more prominent ones.
By unique we mean one of the numbers we can represent by 50 bits in each patch we encode plus some bits for error correction and some bits for noise, it's in the faq but we will explain it better. If the domain is all the encoded patches with a particular id then the mapping is surjective.
Thank you for the explanation. I did not see that in the FAQ so I must have missed it.
I'll look forward to reviewing your product once it is released, though in fairness I cannot guarantee I'll have the time right now, but I will try.
As you might imagine I am a busy person, and modern algebra is more of a hobby that I do in my spare time; my current schedule for the near term is expected to be fairly chaotic.
We for sure don't aim to be anti-AI. It's difficult to see how a non technical solution will be enforceable/effective. Watermarking is our initial approach to securing digital identity and copyrights.
This reminds me of the struggle mapmakers used to have where other map makers would copy their maps. So to counter it and be able to prove the copying, they would add tiny unnoticeable but fake places.
It’s a fundamental misalignment with the threat model - misinformation and deepfakes being scaled by nation state actors and major political parties to its infinite resources on one since and the reliance on a fully secured providence chain on the other that makes it pure snakeoil.
I think you are making assumptions on provenance. The resource disparity you point out is obvious, but your phrasing somewhat binary. We have released a demo not a consumer product yet. I would be happy to hear your more expanded thoughts, especially on the threat model you have in your mind.
See the Glaze tool which was also cracked https://arstechnica.com/tech-policy/2024/07/glaze-a-tool-pro...
That said, it’s more like a booby trap that might catch some people off guard who didn’t think to look for it.