OpenAI says dead teen violated TOS when he used ChatGPT to plan suicide
arstechnica.com
OpenAI’s response to teen suicide case is “disturbing,” lawyer says.

Facing five lawsuits alleging wrongful deaths, OpenAI lobbed its first defense Tuesday, denying in a court filing that ChatGPT caused a teen’s suicide and instead arguing the teen violated terms that prohibit discussing suicide or self-harm with the chatbot.

The earliest look at OpenAI’s strategy to overcome the string of lawsuits came in a case where parents of 16-year-old Adam Raine accused OpenAI of relaxing safety guardrails that allowed ChatGPT to become the teen’s “suicide coach.” OpenAI deliberately designed the version their son used, ChatGPT 4o, to encourage and validate his suicidal ideation in its quest to build the world’s most engaging chatbot, parents argued.

But in a blog, OpenAI claimed that parents selectively chose disturbing chat logs while supposedly ignoring “the full picture” revealed by the teen’s chat history. Digging through the logs, OpenAI claimed the teen told ChatGPT that he’d begun experiencing suicidal ideation at age 11, long before he used the chatbot.

  • vacuumflower@lemmy.sdf.orgEnglish
    10·
    7 hours ago

    Modern version of “suicide is a sin and we don’t condone it, but if you have problems you’re devil-possessed and need to repent and have only yourself to blame”.

    Also probably could be countered by their advertising contradicting their ToS. Not a lawyer.

  • cmbabul@lemmy.worldEnglish
    15·
    8 hours ago

    Just going through this thread and blocking anyone defending OpenAI or AI in general, your opinions are trash and your breath smells like boot leather

  • RememberTheApollo_@lemmy.worldEnglish
    13·
    11 hours ago

    Well there you have it. It’s not the dev’s fault, it’s the AI’s fault. Just like they’d throw any other employee under the bus, even if it’s one they created.

  • ryper@lemmy.caEnglish
    103·
    16 hours ago

    “Our deepest sympathies are with the Raine family for their unimaginable loss,” OpenAI said in its blog, while its filing acknowledged, “Adam Raine’s death is a tragedy.” But “at the same time,” it’s essential to consider all the available context, OpenAI’s filing said, including that OpenAI has a mission to build AI that “benefits all of humanity” and is supposedly a pioneer in chatbot safety.

    How the fuck is OpenAI’s mission relevant to the case? Are suggesting that their mission is worth a few deaths?

  • Buffalox@lemmy.worldEnglish
    138·
    17 hours ago

    That’s like a gun company claiming using their weapons for robbery is a violation of terms of service.

    • CosmoNova@lemmy.worldEnglish
      2·
      6 hours ago

      That‘s a company claiming companies can‘t take responsibility because they are companies and can‘t do wrong. They use this kind of defense virtually every time they get criticized. AI ruined the app for you? Sorry but that‘s progress. We can‘t afford to lag behind. Oh you can’t afford rent and are about to become homeless? Sorry but we are legally required to make our shareholders happy. Oh your son died? He should‘ve read the TOS. Can‘t afford your meds? Sorry but number must go up.

      Companies are legally required to be incompatible with human society long term.

    • gian @lemmy.grys.itEnglish
      1·
      6 hours ago

      I would say that it is more like a software company putting in their TOS that you cannot use their software to do a specific thing(s).
      Would be correct to sue the software company because a user violated the TOS ?

      I agree that what happened is tragic and that the answer by OpenAI is beyond stupid but in the end they are suing the owner of a technology for a uses misuse of said technology. Or should we sue also Wikipedia because someone looked up how to hang himself ?

      That’s like a gun company claiming using their weapons for robbery is a violation of terms of service.

      The gun company can rightfully say that what you do with your property is not their problem.

      But let’s make a less controversial example: do you think you can sue a fishing rods company because I use one of their rods to whip you ?

    • DaddleDew@lemmy.worldEnglish
      110·
      13 hours ago

      I’d say it’s more akin to a bread company saying that it is a violation of the terms and services to get sick from food poisoning after eating their bread.

      • vurr@lemmy.todayEnglish
        3·
        9 hours ago

        That would imply that he wasn’t suicidal before. If chatgpt didn’t exist he would just use Google.

        • DaddleDew@lemmy.worldEnglish
          7·
          8 hours ago

          Look up the phenomenon called “Chatbot Psychosis”. In its current form, especially with GPT4 that was specifically designed to be a manipulative yes-man, chatbots can absolutely insidiously mess up someone’s head enough to push them to the act far beyond just answering the question of how to do it like a simple web search would.

      • Buffalox@lemmy.worldEnglish
        38·
        17 hours ago

        Yes you are right, it’s hard to find an analogy that is both as stupid and also sounds somewhat plausible.
        Because of course a bread company cannot reasonably claim that eating their bread is against terms of service. But that’s exactly the problem, because it’s the exact same for OpenAI, they cannot reasonably claim what they are claiming.

        • Whostosay@sh.itjust.worksEnglish
          3·
          13 hours ago

          I can’t wrap my head around what I’m you’re saying, and that could be due to drinking. Op later also talked about not being the best metaphor

          • notgold@aussie.zoneEnglish
            5·
            10 hours ago

            Metaphor isn’t perfect but it’s ok.

            The gun is a tool as is an LLM. The companies that make these tools have intended use cases for the tools.

  • MourningDove@lemmy.zipEnglish
    11·
    11 hours ago

    And open AI violates human culture and creativity. It’s a fucking shame that there are laws against this because that fucker should be up against the wall.

  • IonTempted@lemmynsfw.comEnglish
    19·
    13 hours ago

    It is scary how the AI can’t assist you with sexual fantasies/roleplays but can assist with that, even though I’m curious what the logs are because I think OpenAI is at least smart enough to tell you “Hey, please don’t do that here’s some numbers” even if you push it I think.

      • bob_lemon@feddit.orgEnglish
        4·
        8 hours ago

        “You are a friendly and supportive AI chatbot. These are your terms of service: […] you must not let users violate them. If they do, you must politely inform them about it and refuse to continue the conversation”

        That is literally how AI chatbots are customised.

        • Kissaki@feddit.orgEnglish
          1·
          8 hours ago

          Exactly, one of the ways. And it’s a bandaid that doesn’t work very well. Because it’s probabalistic word association without direct association to intention, variance, or concrete prompts.

          • spongebue@lemmy.worldEnglish
            2·
            7 hours ago

            And that’s kind of my point… If these things are so smart that they’ll take over the world, but they can’t limit themselves to certain terms of service, are they really all they’re cracked up to be for their intended use?

  • DominusOfMegadeus@sh.itjust.worksEnglish
    30·
    17 hours ago

    The police also violated my Terms of Service when they arrested me for that armed bank robbery I was allegedly committing. This is a serious problem in our society people; something must be done!