Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.

Hey ChatGPT, how about we make it so no one unalives themselves with your help even f they’re over 18.

For fucks sake it helped him write a suicide note.

i know it's offensive to see people censor themselves in that way because of tiktok, but try to remember there's a human being on the other side of your words.

He could have Google the info. Humans failed this guy. Human behavior needs to change

GPT could have been Google or a stranger in. Chatroom.

except OpenAI isn't making a dime. they're just burning money at a crazy rate.

I parented a teen boy. Sometimes, no matter what you do and no matter how close you were before puberty, a switch flips outside your control and they won’t talk to you anymore. We were a typical family, no abuse, no fighting, nobody on drugs, both parents with 9-5 office jobs, very engaged with school and etc.

Thankfully, after riding it out (getting him therapy, giving space, respect, and support), he came out the other side fine. But there were a few harrowing years during that phase.

I went through a similar phase in my teens. If AI was there to feed my issues, I might not have survived it. Teenage hormones are a helluva drug.

The real issue is that mental health in the United States is an absolute fucking shitshow.

988 is a bandaid. It’s an attempt to pretend someone is doing anything. Really a front for 911.

Even when I had insurance, it was hundreds a month to see a therapist. Most therapists are also trained on CBT and CBT only because it’s a symptoms focused approach that gets you “well” enough to work. It doesn’t work for everything, it’s “evidence based” though in that it’s set up to be easy to measure. It’s an easy out, the McDonald’sification of therapy. Just work the program and everything will be okay.

There really are so few options for help.

He could have Google the info. Humans failed this guy. Human behavior needs to change

GPT could have been Google or a stranger in. Chatroom.

I think we all agree on the fact that OpenAI isn't exactly the most ethical corporation on this planet (to use a gentle euphemism), but you can't blame a machine for doing something that it doesn't even understand.

Sure, you can call for the creation of more "guardrails", but they will always fall short: until LLMs are actually able to understand what they're talking about, what you're asking them and the whole context around it, there will always be a way to claim that you are just playing, doing worldbuilding or whatever, just as this kid did.

What I find really unsettling from both this discussion and the one around the whole age verification thing, is that people are calling for techinical solutions to social problems, an approach that always failed miserably; what we should call for is for parents to actually talk to their children and spend some time with them, valuing their emotions and problems (however insignificant they might appear to a grown-up) in order to, you know, at least be able to tell if their kid is contemplating suicide.

but you can't blame a machine for doing something that it doesn't even understand.

But you can blame the creators and sellers of that machine for operating unethically.

If I build and sell a coffee maker that sometimes malfunctions and kills people, I’ll be sued into oblivion, and my coffee maker will be removed from the market. You don’t blame the coffee maker, but you absolutely hold the creator accountable.

Yes and no. The example you made is of a defective device, not of an "unethical" one - though I understand how you are trying to say that they sold a malfunctioning product without telling anyone.

For LLMs, however, we know damn well that they shouldn't be used as a therapist or as a digital friend to ask for advice; they are no more than a powerful search engine.

An example that is more in line with the situation we're analyzing is a kid that stabs itself with a knife after his parents left him playing with one; are you sure you want to sue the company that made the knife in that scenario?