• Kaligalis@lemmy.world
    link
    fedilink
    English
    arrow-up
    18
    ·
    5 hours ago

    Those hidden markers will be very hard to hide in code review in an IDE which highlights invisible Unicode code points. For code, adding watermarks is likely to cause bugs. For text, Chinese models are as good - so just switching to them solves the issue.
    I think, Anthropic doesn’t want to perform economic seppuku.

    • toph@feddit.uk
      link
      fedilink
      English
      arrow-up
      5
      ·
      edit-2
      60 minutes ago

      These are not watermarks using hidden characters. Their approach is undetectable even with an IDE. For a sequence of tokens, an LLM predicts the most likely next token, with some amount of randomness between equally likely candidates. The “watermark” is to introduce a statistical bias to this randomness, by altering the probability distribution of generated text according some hash function with a secret key, thereby embedding a statistical signature into the text itself.

      So if the text is “I like to eat __” the model might have 3 top candidates for the next word (apple/orange/banana) that would be chosen at random. Instead that choice will be biased towards one option according to their hash function. And then again “I like to eat banana __” (cake/pie/tart).

      To verify a text, they look for the “watermark” by scanning the text and looking at whether sequences of tokens chocies fits their biased probability distribution or are truly random. Just one match doesn’t tell you anything, but if they see a consistent pattern over a 1000 word document, they can give a very high confidence that their model generated the text.

      To a human it looks like nornal generated text, and the output quality isn’t affected much (or ar all). It’s much more effective on generated prose, and not very effective on computer code.

      • orclev@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 hour ago

        I mean it kind of already does this accidentally, just look for em dash and/or emojis sprinkled all over the document and you can be pretty certain that it was AI generated.

        The other thing I’m seeing here is that this will only be effective for large chunks of text, if you’re dealing with small snippets interspersed with human generated content there will be enough statistical noise to make classification hard without introducing a bunch of false positives and negatives.

  • Wildmimic@anarchist.nexus
    link
    fedilink
    English
    arrow-up
    7
    ·
    5 hours ago

    This is a futile approach. It has been shown again and again that watermarks can easily be removed by AI models that are widely and openly available. It is like a bandaid for a severed artery and only is useful to shield Anthropic from the wrath of the courts.

  • PotatoesFall@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    48
    ·
    8 hours ago

    The year is 2027. A beginner programmer searches how to access the last array element in a programming language she is unfamiliar with. The results come in. “Top 10 array accesses that will make you want to buy our sponsors product”. She sighs and asks Claude instead. She pastes the one-liner into her code. Finishes the remaining code. Tests. Double-check. Commit. Push. A notification appears. “Your account has been temporarily suspended for violating GitHub’s terms of service”. Forgot to mark claude as a co-author. She sighs again

    • joe@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      4 hours ago

      These watermarks require the text to be long-ish and not too edited from the original. I didn’t see this explained in the article, but I only skimmed it. The press release from Anthropic goes into it, without giving many details.

      I’m guessing it’s going to embed a pattern with punctuation and word/letter choices, so there will need to be a sufficient amount of text to ensure its actually the watermark and not just dumb luck.

      • toph@feddit.uk
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 hour ago

        Yeah it’s exactly a pattern with token choices, by introducing a statistical bias to the randomness in token generation. Quality isn’t affected because the model is still choosing from the best candidates for the next token. You’re right that to be effective they need a long enough input to get enough matches to be.

        Computer code is also going to be less effective because code has a much more rigid structure dictated by language syntax rules, coding conventions, linters and formatters, etc.

  • neuracnu@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    76
    ·
    edit-2
    1 hour ago

    Correction: consumer grade machine-extruded text will be watermarked.

    Machine-extruded text generation controlled by government entities, fascist regime propaganda outlets, capitalist interests and basically anyone else with a ton of money will be welcome to bulldoze venues of public debate with as much un-watermarked thought-slop as they care to generate that will be virtually indistinguishable from the real thing.

    Imagine layers of public influence campaigns, all conjured by LLMs, with agents A/B testing rhetoric constantly for success and efficacy.

  • mrmaplebar@fedia.io
    link
    fedilink
    arrow-up
    25
    ·
    8 hours ago

    Call me cynical, but I think this is an attempt to do the bare minimum in an effort to preempt laws being put in place.

  • Casuls_Die_Thrice@lemmy.zip
    link
    fedilink
    English
    arrow-up
    17
    ·
    edit-2
    9 hours ago

    As I’ve said in response to a number of articles about this, all this means is that when someone wants to use Claude’s output for something, they just need to remember to manually type it in now, rather than just copy-paste it. Practically speaking, this changes nothing.

    • jagermo@feddit.org
      link
      fedilink
      English
      arrow-up
      19
      ·
      9 hours ago

      It’s some kind of statistical thing about what word is where, what character is there how often. Typing won’t help, but a rewrite will.

      • evilcultist@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        7 hours ago

        That was the author’s guess. Seems like this would only be detectable if it’s being used in enough text (possibly across multiple files).

      • Casuls_Die_Thrice@lemmy.zip
        link
        fedilink
        English
        arrow-up
        2
        ·
        9 hours ago

        What I’m saying is that if someone just types verbatim what Claude says, instead of just going “Ctrl-C, Ctrl-V”, that completely undermines this whole thing.

    • SamuraiBeandog@lemmy.world
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      9 hours ago

      The difference between auto-generating hundreds to thousands of words, versus having to manually read and type all of those words out yourself is very obviously a lot more than “nothing”.

      And that’s not even taking into account the fact that we don’t know what their method of watermarking is. From how they’re talking it sounds like it will be algorithmically embedded in the text itself somehow, through specific choices of words or something similar. If that’s the case, typing it out word for word will retain the watermark.

      • Casuls_Die_Thrice@lemmy.zip
        link
        fedilink
        English
        arrow-up
        5
        ·
        edit-2
        9 hours ago

        How tho? Will Claude be looking over our proverbial shoulders now, and if whatever we type just so happens to be word-for-word what it said, it can magically flag it?

        • scrion@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          7 hours ago

          You have a fundamental misunderstanding of how the watermarking works. There is no hidden metadata, but most likely, the statistical properties of the generated text are being changed.

          If you met someone on Halloween with a face mask, and they’d always use the word cromulent in every sentence, you’d probably assume it’s your buddy Mark, who is about the only guy in your circle of friends who does that.

          The model will produce a text where individual words at certain positions, or various n-grams encode a kind of fingerprint that will be an indicator for the text being processed by Claude.

          Like, when people have, like, a specific accent or talk in a certain way, you can totally, like, figure out where they’re from, for sure.

          • AvocadoSandwich@eviltoast.org
            link
            fedilink
            English
            arrow-up
            2
            ·
            7 hours ago

            So isn’t the solution then to just feed the Claude output through a second tiny LLM with the prompt to slightly rewrite the input? I mean you can probably use a 1b model to do that and that can run on practically anything nowadays

            • joe@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              ·
              4 hours ago

              Yes, if the watermarking works like we all assume, significantly editing the output will likely break the watermark. They say this in their press release, as well as that the text must be a certain length to be watermarked. (Probably as a function of how the watermark works.)

              I don’t think this is expected to be 100% foolproof. They certainly don’t make that claim; they explicitly say that the lack of a watermark isn’t conclusive evidence that the text wasn’t generated by their models.

    • Wildmimic@anarchist.nexus
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      6 hours ago

      I don’t think you will even need to type yourself. Tools removing the watermark will be available 3-4 days after this hits the market, fully able to be automated. Watermarks are not a solution that will prevent someone who wants to deceive you from doing so - only awareness that everything on the web is sus will.

      Edit: The only alternative is an 100% digitally signature authenticated web, from the web login down to the posting. I am not sure this is a better way because it also means 100% transparency of everything someone does.

    • Pennywise@quokk.au
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      7 hours ago

      There’s typically a keyboard shortcut to copy things sans formatting (plain text.)

  • Sims@lemmy.ml
    link
    fedilink
    English
    arrow-up
    5
    ·
    7 hours ago

    They want to check and control what the output is used for, IE who train on their output… The big US scum-corps are just tightening their ps*cho control regime (although most of us escapes them via Chinese open models)…