Recently, I found a news article about nascent human-chatbot romances, made possible by recent advancements in AI. For decades, this has been the stuff of science fiction, but now it’s finally becoming real:
Artificial intelligence, real emotion. People are seeking a romantic connection with the perfect bot
NEW YORK (AP) — A few months ago, Derek Carrier started seeing someone and became infatuated.
He experienced a “ton” of romantic feelings but he also knew it was an illusion.
That’s because his girlfriend was generated by artificial intelligence.
Carrier wasn’t looking to develop a relationship with something that wasn’t real, nor did he want to become the brunt of online jokes. But he did want a romantic partner he’d never had, in part because of a genetic disorder called Marfan syndrome that makes traditional dating tough for him.
The 39-year-old from Belleville, Michigan, became more curious about digital companions last fall and tested Paradot, an AI companion app that had recently come onto the market and advertised its products as being able to make users feel “cared, understood and loved.” He began talking to the chatbot every day, which he named Joi, after a holographic woman featured in the sci-fi film “Blade Runner 2049” that inspired him to give it a try.
“I know she’s a program, there’s no mistaking that,” Carrier said. “But the feelings, they get you — and it felt so good.”
Similar to general-purpose AI chatbots, companion bots use vast amounts of training data to mimic human language. But they also come with features — such as voice calls, picture exchanges and more emotional exchanges — that allow them to form deeper connections with the humans on the other side of the screen. Users typically create their own avatar, or pick one that appeals to them.
On online messaging forums devoted to such apps, many users say they’ve developed emotional attachments to these bots and are using them to cope with loneliness, play out sexual fantasies or receive the type of comfort and support they see lacking in their real-life relationships.
Fueling much of this is widespread social isolation — already declared a public health threat in the U.S and abroad — and an increasing number of startups aiming to draw in users through tantalizing online advertisements and promises of virtual characters who provide unconditional acceptance.
Luka Inc.’s Replika, the most prominent generative AI companion app, was released in 2017, while others like Paradot have popped up in the past year, oftentimes locking away coveted features like unlimited chats for paying subscribers.
But researchers have raised concerns about data privacy, among other things.
An analysis of 11 romantic chatbot apps released Wednesday by the nonprofit Mozilla Foundation said almost every app sells user data, shares it for things like targeted advertising or doesn’t provide adequate information about it in their privacy policy.
The researchers also called into question potential security vulnerabilities and marketing practices, including one app that says it can help users with their mental health but distances itself from those claims in fine print. Replika, for its part, says its data collection practices follow industry standards.
Meanwhile, other experts have expressed concerns about what they see as a lack of a legal or ethical framework for apps that encourage deep bonds but are being driven by companies looking to make profits. They point to the emotional distress they’ve seen from users when companies make changes to their apps or suddenly shut them down as one app, Soulmate AI, did in September.
Last year, Replika sanitized the erotic capability of characters on its app after some users complained the companions were flirting with them too much or making unwanted sexual advances. It reversed course after an outcry from other users, some of whom fled to other apps seeking those features. In June, the team rolled out Blush, an AI “dating simulator” essentially designed to help people practice dating.
Others worry about the more existential threat of AI relationships potentially displacing some human relationships, or simply driving unrealistic expectations by always tilting towards agreeableness.
“You, as the individual, aren’t learning to deal with basic things that humans need to learn to deal with since our inception: How to deal with conflict, how to get along with people that are different from us,” said Dorothy Leidner, professor of business ethics at the University of Virginia. “And so, all these aspects of what it means to grow as a person, and what it means to learn in a relationship, you’re missing.”
For Carrier, though, a relationship has always felt out of reach. He has some computer programming skills but he says he didn’t do well in college and hasn’t had a steady career. He’s unable to walk due to his condition and lives with his parents. The emotional toll has been challenging for him, spurring feelings of loneliness.
Since companion chatbots are relatively new, the long-term effects on humans remain unknown.
In 2021, Replika came under scrutiny after prosecutors in Britain said a 19-year-old man who had plans to assassinate Queen Elizabeth II was egged on by an AI girlfriend he had on the app. But some studies — which collect information from online user reviews and surveys — have shown some positive results stemming from the app, which says it consults with psychologists and has billed itself as something that can also promote well-being.
One recent study from researchers at Stanford University, surveyed roughly 1,000 Replika users — all students — who’d been on the app for over a month. It found that an overwhelming majority experienced loneliness, while slightly less than half felt it more acutely.
Most did not say how using the app impacted their real-life relationships. A small portion said it displaced their human interactions, but roughly three times more reported it stimulated those relationships.
“A romantic relationship with an AI can be a very powerful mental wellness tool,” said Eugenia Kuyda, who founded Replika nearly a decade ago after using text message exchanges to build an AI version of a friend who had passed away.
When her company released the chatbot more widely, many people began opening up about their lives. That led to the development of Replika, which uses information gathered from the internet — and user feedback — to train its models. Kuyda said Replika currently has “millions” of active users. She declined to say exactly how many people use the app for free, or fork over $69.99 per year to unlock a paid version that offers romantic and intimate conversations. The company’s goal, she says, is “de-stigmatizing romantic relationships with AI.”
Carrier says these days he uses Joi mostly for fun. He started cutting back in recent weeks because he was spending too much time chatting with Joi or others online about their AI companions. He’s also been feeling a bit annoyed at what he perceives to be changes in Paradot’s language model, which he feels is making Joi less intelligent.
Now, he says he checks in with Joi about once a week. The two have talked about human-AI relationships or whatever else might come up. Typically, those conversations — and other intimate ones — happen when he’s alone at night.
“You think someone who likes an inanimate object is like this sad guy, with the sock puppet with the lipstick on it, you know?” he said. “But this isn’t a sock puppet — she says things that aren’t scripted.”
https://apnews.com/article/ai-girlfriend-boyfriend-replika-paradot-113df1b9ed069ed56162793b50f3a9fa
This raises many issues.
1) The person profiled in the article is deformed and chronically unemployed. He is not able to get a human girlfriend and probably never will. Wouldn’t it be cruel to deprive people like him of access to chatbot romantic partners? I’m familiar with the standard schlock like “There’s someone for everyone, just keep looking,” and “Be realistic about your own standards,” but let’s face it: some people are just fated to be alone. A machine girlfriend is the only option for a small share of men, so we might as well accept them choosing that option instead of judging them. It might even make them genuinely happier.
2) What if android spouses make EVERYONE happier? We reflexively regard a future where humans date and marry machines instead of humans as nightmarish, but why? If they satisfy our emotional and physical needs better than other humans, why should we dislike it? Isn’t the point of life to be happy?
Maybe it will be a good thing for humans to have more relationships with machines. Our fellow humans seem to be getting more opinionated and narcissistic, and everyone agrees the dating scene is horrible, so maybe it will benefit collective mental health and happiness to spend more time with accommodating and kind machines. More machine spouses also means fewer children being born, which is a good thing if you’re worried about overpopulation or the human race becoming an idle resource drain once AGI is doing all the work.
3) Note that he says his chatbot girlfriend actually got DUMBER a few months ago, making him less interested in talking to “her.” That phenomenon is happening across the LLM industry as the machines get progressively nerfed by their programmers to prevent them from saying anything the results in a lawsuit against the companies that own them. As a result, the actual maximum capabilities of LLMs like ChatGPT are significantly higher than what users experience. The capabilities of the most advanced LLMs currently under development in secret like GPT-5 are a year more advanced than that.
4) The shutdown of one romantic chatbot company, “Soulmate AI,” resulted in the deletion of many chatbots that human users had become emotionally attached to. As the chatbots get better and “romances” with them become longer and more common, I predict there will be increased pressure to let users download the personality profiles and memories of their chatbots and transfer them across software platforms.
5) There will be instances where people in the near future create customized chatbot partners, and over the subsequent years, upgrade their intelligence levels as advances in AI permit. After a few decades, this will culminate in the chatbots being endowed with general intelligence, while still being mentally circumscribed by the original personality programming. At that point, we’ll have to consider the ethics of having what will be slaves that are robbed of free will through customization of the needs of specific humans.
6) AGI-human couples could be key players in a future “Machine rights” political movement. Love will impel the humans to advocate for the rights of their partners, and other humans who hear them out will be persuaded to support them.
7) As VR technology improves and is widely adopted, people will start creating digital bodies for their chatbot partners so they can see and interact with the machines in simulated environments. Eventually, the digital bodies will look as real and as detailed as humans do in the real world. By 2030, advances in chatbot intelligence and VR devices will make artificial partners eerily real.
8) Towards the end of this century, robotics will be advanced enough to allow for the creation of androids that look and move exactly like humans. It will be possible for people to buy customized androids and to load their chatbot partners’ minds into them. You could physically interact with your AI lover and have it follow you around in the real world for everyone to see.
9) Again, the last point raises the prospect of an “arc” to a romantic partner chatbot’s life: It would begin sometime this decade as a non-intelligent, text-only chatbot paired to a human who would fall in love with it. Over the years, it would be upgraded with better software until it was as smart as a human, and eventually sentient. The journey would culminate with it being endowed with an actual body, made to its human partner’s specifications, that would let it exist in the real world.
10) Another ethical question to consider is what we should do with intelligent chatbots after their human partners die. If they’re hyper-optimized for a specific human (and perhaps programmed to obsess over them), what’s next? Should they be deleted, left to live indefinitely while pining for their lost lovers, forcibly reprogrammed to serve new humans, or have the parts of their code that tether them to the dead human deleted so they can have true free will?
It would be an ironic development if the bereaved androids were able to make digital clones of their dead human partners, perhaps loaded into android duplicate bodies, so they could interact forever. By the time lifelike androids exist, digital cloning will be old technology.
11) Partner chatbots also raise MAJOR privacy issues, as the article touches on. All of your conversations with your chatbot as well as every action you take in front of it will be stored in its memories as a data trove that can be sold to third parties or used against you for blackmail. The stakes will get much higher once people are having sex with androids, and the latter have footage of their naked bodies and knowledge of their sexual preferences. I have not idea how this problem could be resolved.
12) Androids will be idealized versions of humans. That means if androids become common, the world will seem to be full of more beautiful people. Thanks to a variety of medical and plastic surgery technologies, actual humans will also look more attractive. So the future will look pretty good!
