These companies should be criminally liable for training their models on child porn.

  • Glitchvid@lemmy.world
    link
    fedilink
    English
    arrow-up
    59
    ·
    17 hours ago

    One concerning thing is that when these AI companies are pushed to crack down on this, they’ll retrain and bias their models to only recognize traditional “pornstar” body types as ‘adult’.

    This is a problem when they then get used as a mass content flagging and moderation tool — it’s not hard to imagine some adult woman taking nudes (as people do) and just because they don’t have a huge bust, getting auto flagged as having illegal material. Not to mention body types that aren’t even real, like the art made in furry communities.

    It’s not like there isn’t precedent, recently Discord banned a ton of people because its scanning system started throwing false positives, and that was only corrected because it made such a stink.

    • square@lemmy.zip
      link
      fedilink
      English
      arrow-up
      8
      ·
      8 hours ago

      There was a story a while ago where some dude was about to be convicted of possession for a few videos of a young looking woman. The prosecution even had an expert witness that repeatedly testified that it was impossible for an adult woman to look this way and wouldn’t be budged at all on cross examination. Luckily, at the 11th hour, the defense found the porn star, who was actually somewhat famous but the videos didn’t contain her stage name. She took the stand, a 20-something looking like a 13yo, and the case was dismissed.

      If this shit is automated there will definitely be a lot of false positives. And it will be automated, and it will be a “wide net” pulling in all kinds of stuff like you mentioned and more. And no one will really care to make it work better because “the children”. Meanwhile, the will be no additional effort to stop harm to actual children because…well, I think we all know why. Sloppy, performative, bullshit is what we’ll get instead of measures to prevent and help victims.

      • ayyy@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 hours ago

        This sounds like an urban legend, do you remember the time/location or any names for this case?

        • square@lemmy.zip
          link
          fedilink
          English
          arrow-up
          11
          ·
          edit-2
          5 hours ago

          Yup, took a little digging though, I couldn’t remember the name off the top of my head. Back when this happened she would be a top result for “young Latina” on all the big sites, but I think the algorithms favour newer videos so I had to scroll several pages before I recognised her. If you want to read more about it Google “case dismissed Little Lupe.”

          I remember reading about this myself in a major news outlet specifically because I recognised the name at the time. Although she isn’t my type, absolutely no breasts and hips isn’t for me, if you use search terms for youthful actors, ones like this will also come up. Like I said, she was somewhat big at the time and I saw her name often.

          Here is another link with some more info. I did forget some details. The biggest one is the video was labelled with her name but the prosecutor refused to do a simple record search (all of her movies were compliant with the applicable laws regarding this), or accept all the evidence the defendant’s lawyer presented them with, and the defense had to fly her in to testify.

          • ayyy@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            5
            ·
            5 hours ago

            Oh wow thats fascinating, thanks for your detective work!

            Our whole system of “expert witnesses” in courtrooms is woefully broken and under-regulated.

            • square@lemmy.zip
              link
              fedilink
              English
              arrow-up
              5
              ·
              4 hours ago

              Yeah, the whole “expert witness” thing really needs some sort of regulation and consequences for spouting bullshit in a courtroom. I don’t know how old you are, but I’m old enough to have loved The Forensics Files, and I don’t even know how many times I’ve read an article or watched a documentary about someone who got their conviction overturned, or about ongoing efforts to do so, that seem very familiar and I realize I saw it on that show or another one years ago and the cutting edge forensics that were used are now considered complete bullshit.

    • Virtvirt588@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      ·
      15 hours ago

      This is a problem when they then get used as a mass content flagging and moderation tool — it’s not hard to imagine some adult woman taking nudes (as people do).

      The problem is with sexual abuse material not nudity on its own, there is a clear distinction between those two. It seems like this is the major problem when it comes to “crack-downs” on this.

    • outlawcarl@fedinsfw.app
      link
      fedilink
      English
      arrow-up
      15
      ·
      16 hours ago

      Yeah we will drown in flase postives, my other worry is they will spend on this auto scaning of media call it problem solved slash bugets for other things and predators move to dumb tech or really encoded shit.

      Thats if there even is enouth money/compute to scan everything…

    • abadidonque@feddit.uk
      link
      fedilink
      English
      arrow-up
      5
      ·
      16 hours ago

      They should be made to retrain their models using only images of those palaeolithic venus figurines.

    • howdy@lemmy.ml
      link
      fedilink
      English
      arrow-up
      4
      ·
      16 hours ago

      AI shouldn’t train on porn at all. Too many ethical issues.