$25 and 25 minutes with our new girlfriend, AI Amouranth
there is… a lot going on here–and it’s part of a broader trend which is probably not for the better and speaks to some deeper-seated issues we currently have in society. a choice moment from the article here on another influencer doing a similar thing earlier this year, and how that went:
Siragusa isn’t the first influencer to create a voice-prompted AI chatbot using her likeness. The first would be Caryn Marjorie, a 23-year-old Snapchat creator, who has more than 1.8 million followers. CarynAI is trained on a combination of OpenAI’s ChatGPT-4 and some 2,000 hours of her now-deleted YouTube content, according to Fortune. On May 11, when CarynAI launched, Marjorie tweeted that the app would “cure loneliness.” She’d also told Fortune that the AI chatbot was not meant to make sexual advances. But on the day of its release, users commented on the AI’s tendency to bring up sexually explicit content. “The AI was not programmed to do this and has seemed to go rogue,” Marjorie told Insider, adding that her team was “working around the clock to prevent this from happening again.”
I don’t really like the logic of “chatbots like this will help cure loneliness.” It might help someone feel less lonely at first. But then it’ll be a crutch and, if anything, hurt people’s ability to socialize with other real people. Like it’s a quick dopamine hit that will slowly dig you deeper into the hole you feel you’re in.
Adding on to your comment, I read a really interesting study recently that suggests that interacting with AI engages the social parts of our brains but does not provide the same stimuli/feedback as interacting with a real person leading to increased loneliness and thus increased alcohol abuse and insomnia.
I don’t agree with Amouranth’s statement of “I don’t think it’s harmful for people to form a social aspect with it. As long as we’re aware that it’s not a real person.” You can tell people it’s not real all you want, but when you model it off your voice/likeness, you’re inherently attaching yourself to it. I’m sure the majority of people will understand it’s an AI modeled off her voice, but there will always be some that get more attached to it than they should. Just saying “I don’t think it’s harmful because people should know it’s not real” doesn’t magically solve the issue and cause everyone to behave.
Replika is another smutty type AI, but it’s clearly designated as such, and the avatar is pretty generic looking, which is more of what I’d want to see, at least for now. It being text-only also helps keep that separation. Who knows, it could end up fine, but this is closer to diving in the deep end before people have waded in to see what the water is like.
Yeah, and people got extremely attached to their Replikas. To the point that the subreddit stickied a suicide hotline number when the app removed sexual roleplay.
This strikes me as very exploitative. Capitalizing on loneliness to enrich yourself gives me bad vibes (especially when the users of this thing may actually be worse off mentally in the end, as @cnnrduncan mentioned).
I have no doubts that for some users, it’ll turn into a cycle. They’ll feel lonelier each time they use it, which pushes them to use it more, and so on. I had the same feelings about Replika years before ChatGPT became a thing.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@beehaw.org
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
I don’t really like the logic of “chatbots like this will help cure loneliness.” It might help someone feel less lonely at first. But then it’ll be a crutch and, if anything, hurt people’s ability to socialize with other real people. Like it’s a quick dopamine hit that will slowly dig you deeper into the hole you feel you’re in.
Adding on to your comment, I read a really interesting study recently that suggests that interacting with AI engages the social parts of our brains but does not provide the same stimuli/feedback as interacting with a real person leading to increased loneliness and thus increased alcohol abuse and insomnia.
Fascinating, do you happen to have a link? No worries if it’s buried or you don’t remember, but I’d love to read it
Sure thing, here ya go!
Apologies for the extremely late reply but thank you so much for this fascinating article!
I don’t agree with Amouranth’s statement of “I don’t think it’s harmful for people to form a social aspect with it. As long as we’re aware that it’s not a real person.” You can tell people it’s not real all you want, but when you model it off your voice/likeness, you’re inherently attaching yourself to it. I’m sure the majority of people will understand it’s an AI modeled off her voice, but there will always be some that get more attached to it than they should. Just saying “I don’t think it’s harmful because people should know it’s not real” doesn’t magically solve the issue and cause everyone to behave.
Replika is another smutty type AI, but it’s clearly designated as such, and the avatar is pretty generic looking, which is more of what I’d want to see, at least for now. It being text-only also helps keep that separation. Who knows, it could end up fine, but this is closer to diving in the deep end before people have waded in to see what the water is like.
Yeah, and people got extremely attached to their Replikas. To the point that the subreddit stickied a suicide hotline number when the app removed sexual roleplay.
This strikes me as very exploitative. Capitalizing on loneliness to enrich yourself gives me bad vibes (especially when the users of this thing may actually be worse off mentally in the end, as @cnnrduncan mentioned).
I have no doubts that for some users, it’ll turn into a cycle. They’ll feel lonelier each time they use it, which pushes them to use it more, and so on. I had the same feelings about Replika years before ChatGPT became a thing.