3.5 million times Germans have said ‘I love you’ in the first half of 2021 to Alexa, Amazon’s digital assistant. Many such stories are brought to the public by the media, for instance, reporting not-so-serious studies that found that 14% of male Alexa users in the UK desire a sexual relationship with ‘her’, fueling the public discourse about how some actually ‘feel’ about these talking devices, which are clearly nothing beyond nuts and bolts. A large amount of empirical research has shown how people anthropomorphize machines and treat them as social actors. But just because they say they love Alexa, it does not necessarily mean that they are actually in love as they would be with other humans. So, if people don’t actually ‘love’ Alexa but also don’t see Alexa as ‘just’ a tool – how do they actually perceive Alexa in relation to themselves? And what can we learn from better understanding how humans perceive their relationship with conversational AI?

To investigate perceived human-AI relationships, we took the same approach Nass & colleagues have proposed years ago, based on the CASA paradigm , the tendency of humans to treat computers as social actors. In essence, you look at a theory of human-to-human interaction, in our case, how humans think about their relationship with other humans. You scribble out one human, insert conversational AI, and you can start working on your theory and gather data. We chose a theory that has received mighty empirical support in the past, about how humans think about their relationship with other humans, namely the relational models theory by Alan P. Fiske 1992. At the core, the theory states that ‘motivation, planning, production, comprehension, coordination, and evaluation of human social life may be based largely on combinations of four psychological models’ (Fiske, 1992).

Based on this rationale, we ask the following research questions:

1. Can we apply the relational models theory to the (perceived) relationships between humans and AI?
2. How do users perceive the human-AI relationship?
3. How do the human-AI relationship modes relate to system perception variables (such as trust, perceived warmth or competence) and user characteristics (such as frequency or experience of usage)?

Method: We conducted an online questionnaire study on Prolific with 367 participants and performed factor analyses (PCA, CFA) and analyzed how the participants perceived the human-AI relationship. We conducted a correlational analysis, including variables of system perception and user characteristics, where we looked at bivariate and partial correlations to better understand relational modes in the broader context of human-machine interaction.

How do Humans Think about their Relationship with other Human Beings?

Psychologists have looked at how humans form, build or end relationships with other humans from different perspectives. For instance, the role of proximity and frequency of contact, i.e. how close you live to each other or how often you see each other: Even the direction in which you open the door towards a neighbor may have an impact on how close your relationship will be to this neighbor. Others have looked at how much you integrate the other person into your understanding of yourself in order to define the relationship or measure psychological closeness. Psychological closeness is important for healthy relationships, no matter if we are talking about friendship or a romantic relationship.

Alan P. Fiske (1992) has proposed and received mighty empirical support for the relational models theory (Haslam & Fiske, 1996), even across cultures. In their theory, four modes of how humans think about their relationship with other humans are explained and can be measured quantitatively in a questionnaire: Communal Sharing, Equality Matching, Authority Ranking, and Market Pricing. The questionnaire contains various questions, referring to each of these dimensions to find out how participants think about a specific relationship (which they have to choose before answering the questions) and then rate their level of agreement on a scale from ‘not at all true’ to ‘very true’ for this particular relationship.

The four elementary forms of sociality

To be able to investigate the perceived human-AI relationships through the relational models theory, we took the original questionnaire and adapted it to fit the context of conversational AI. We changed the wording and asked 15 people if they think the questions apply to the perceived human-AI relationships.

How do Humans perceive their Relationship with Conversational AI?

Much attention is brought to variables such as trust, to better understand its impact on how humans use (arguably complex) technology, such as AI or automated systems. Some even argue that trust is in fact a proxy for measuring relationships. Which makes sense, because according to the relational models theory, trust is critical in the communal sharing mode (keywords: Kinship, solidarity, in-group – basically asking: What do we have in common?), but it is thus only one part of describing a relationship. We argue that a multidimensional approach taken from the social sciences is promising, especially in the context of off-the-shelf conversational AI. Some other related work might be too narrow and not well applicable. For instance, empirical work on social presence. seems to lack depth as it does not allow for clear differentiation, e.g., how this social presence is characterized, and leaves open too much room for interpretation. Work in the human-machine interaction (HMI) field often has predefined roles, e.g., the robot as a friend or assistant, which does not apply well in the conversational AI context, where Alexa’s role is somewhat ambiguous. It could be an assistant-role as advertised, but might as well be perceived as a friend-role. In a nutshell, empirical, quantitative research focusing directly on the perceived relationship between humans and machines, from a multidimensional perspective, seems scarce (see. e.g., McLean & Osei-Frimpong, 2019). To fill this gap, we conducted this first study to better investigate human-AI relationships directly. We deemed this multidimensional approach very promising, especially as it has the potential to go beyond the traditionally dichotomous distinctions between emotional and rational dimensions of how humans perceive AI systems (see, for instance, Glikson & Woolley, 2020 for a detailed discussion).

Our factor analysis revealed that the human-AI relationship was perceived along three dimensions. The original two dimensions of communal sharing and equality matching merged into one factor, we deemed rather emotional. We called this third factor peer bonding. Thus, the following three dimensions remained:

• Authority ranking, meaning that there was a linear ordering between the human and the conversational AI perceived
• Market pricing, where humans’ perception is guided by ratio values , and
• Peer bonding, probably the most human-like dimension, where they treated the device as equivalent, yet may keep track of their actions so to speak.

Looking deeper into the human-AI relationship modes, the results showed, that users mostly viewed their relationship as a hierarchical owner-assistant type (authority ranking). However, similarily many characterized their relationship as non-hierarchical exchange (market pricing). Only very few saw their AI as a peer or friendlike relationship. In other words, peer bonding was least pronounced in our data. This outcome already is quite interesting as it is enriching the dichotomous (rational-emotional) views dominating the HMI field.

In order to see how the relational modes fit into the traditionally studied variables in the broader HMI context, we also measured various variables under the category of system perception and user characteristics. We conducted a partial correlational analysis and found that both market pricing and peer bonding showed interesting results in the bivariate and partial correlational analysis, rendering them more informative dimensions for future research. Surprisingly, the traditional assistant view offers no meaningful insights into system perception and user characteristics. Interestingly, peer bonding did not correlate with trust, whereas market pricing did, calling into question the emotional aspects of trust and supporting a rational account of trust in AI.

Of course, there are a number of limitations to this exploratory work. For instance, participants were mostly from the UK, hence we cannot make any assumptions across cultures. Furthermore, we did not measure how the relational models approach affects behavior. In other words, we cannot make any assumptions about the impact on user behavior. This is why we are now running the follow-up study, where we focus on a specific context, namely voice commerce. Voice commerce (or voice shopping) is when users make actual buying decisions via their digital assistant only. Thus, we want to explore whether the perceived human-AI relationships influence shopping behavior to make more concrete statements that may (or may not) be useful to investigate user behavior as well.

Ethical Considerations: Why could this matter?

During the course of our research, we always want to keep in mind and critically reflect on the bird’s eye view in terms of consequences of our findings and the subject matter in general. Why are we doing this? Who benefits from it? Who could be harmed? How are results understood by designers? How do we communicate our research in practice? One of the bigger issues is around anthropomorphism (by design and by the user) and the potential to manipulate by design or users being exploited. As a matter of fact, conversational AI are pieces of hard- and software, connected to the internet and devices of our homes, powered by AI, which enables them to process human natural language quite well, and reply in a similar manner. Yet we seem to be unable to draw a clear line in order to perceive them as what they are: Just a tool. Humans’ capacity to humanize non-human beings is a blessing, as it helps us make better sense of the world. On the other side, it is also a curse, as it can be exploited so easily.

We hope that our work will contribute to a better understanding of the users of AI systems in order to make better decisions for the development, commercialization, regulation, and/or use of AI. We would like to raise the question of who benefits from a better understanding of the user? In addition to other stakeholders, such as regulators, who benefit from a better user understanding in order to protect them, the critical role of developers shall be addressed here. Obviously, developers benefit from a better user understanding to develop better systems, better interfaces, better conversations, etc.. But what makes a ‘good’ system? The EU has some great ideas in different areas, but it seems research is still tapping in the dark how much anthroporphism by design in which context is good.

The line between marketing and manipulation is not so obvious and we need to evluate whether the proposed relationship perspective may have to come with necessary normative caveats, similar to other streams of research: For instance, anthropomorphizing design strategies, are criticized for manipulating the user, which can lead to the user overestimating the capabilities of a system (Sætra, 2021). This can have undesirable behavioral consequences, like oversharing personal data (Aroyo et al., 2021) or overrelying on an automated vehicle (Frison et al., 2019). Particularly interesting may be that some studies have shown that respondents have expressed feelings of sadness or distress with regard to conversational agents, for instance, when asked to abandon the relationship (Xie & Pentina, 2022; Loveys et al., 2021). It is highly questionable how to deal with presumably unintended consequences, such as, when this relational attachment turns into an addiction.

Li, the co-founder of Xiaoice, Microsoft’s conversational agent with over 600 million users worldwide, confirmed the negative consequences, like addiction and sadness, in an interview for Hongkong Free Press (2021). However, he still believes that the benefits of this technology outweigh the risks. So the dilemma remains, where do we draw the line? With regards to our research, until we understand the fundamental theory and consequences of relationships between humans and conversational AI better, we advise refraining from applying the relational models theory repurposed for this study for design and marketing strategies.

Conclusion: AI is a Tool, but it is not ‘just’ a Tool

Alexa, will you marry me? 6,000 times per day in India users propose to Alexa, Amazon’s digital assistant. Fortunately their love remains unreturned: We’re at pretty different places in our lives. Literally. I mean, you’re on Earth and I’m in the cloud. – is one of the replies to the millions of marriage proposals Alexa had received so far. Thankfully, responsible designers of commercially available conversational AI ensure, that users are always made aware, that these agents are computer systems with no feelings or thoughts. Nevertheless, in line with similar previous research, our study has shown that users still attach feelings, attribute agency and perceive a relationship with conversational AI, which shares similarities with human relationships. We believe that this topic will accompany us for a while and by far we cannot provide a comprehensive answer. But we hope to have shed a bit more light on a better understanding of how humans perceive their relationship with conversational AI and inspire future research to take this perspective into acount.

Acknowledgement

This is a brief summary of the first research collaboration between scip AG and the Social Processes Lab of the Leibniz Institut für Wissensmedien, which was presented at 5th AAAI/ACM conference on Artificial Intelligence Ethics, and Society (Oxford, August 1-3, 2022)

References

• Amazon. (2021). Customers in India say “I love you” to Alexa 19,000 times a day. Amazon. Retreived 12, 2021 https://www.aboutamazon.in/news/devices/customers-in-india-say-i-love-you-to-alexa-19-000-times-a-day
• Aroyo, A. M., Bruyne, J. de, Dheu, O., Fosch-Villaronga, E., Gudkov, A., Hoch, H., Tamò-Larrieux, A. (2021). Overtrusting robots: Setting a research agenda to mitigate overtrust in automation. Paladyn, Journal of Behavioral Robotics, 12(1), 423-436. https://doi.org/10.1515/pjbr-2021-0029
• Fiske, A. P. (1992). The four elementary forms of sociality: Framework for a unified theory of social relations. Psychological Review, 99(4), 689-723. https://doi.org/10.1037//0033-295×.99.4.689
• Frison, A.-K., Wintersberger, P., Riener, A., Schartmüller, C., Boyle, L. N., Miller, E., & Weigl, K. (2019). In UX We Trust. In S. Brewster, G. Fitzpatrick, A. Cox, & V. Kostakos (Eds.), Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-13). New York, NY, USA: ACM. https://doi.org/10.1145/3290605.3300374
• Glikson, E., & Woolley, A. W. (2020). Human Trust in Artificial Intelligence: Review of Empirical Research. Academy of Management Annals, 14(2), 627-660. https://doi.org/10.5465/annals.2018.0057
• Haslam, N., & Fiske, A. P. (1999). Relational models theory: A confirmatory factor analysis. Personal Relationships, 6(2), 241-250. https://doi.org/10.1111/j.1475-6811.1999.tb00190.
• Hong Kong Free Press (2021). How an ‘always there’ AI chatbot comforts China’s lonely millions. Hong Kong Free Press. Retrieved, 1, 2022 from https://hongkongfp.com/2021/08/29/always-there-the-ai-chatbot-comforting-chinas-lonely-millions/
• Loveys, K., Sagar, M., & Broadbent, E. (2020). The Effect of Multimodal Emotional Expression on Responses to a Digital Human during a Self-Disclosure Conversation: A Computational Analysis of User Language. Journal of Medical Systems, 44(9), 143. https://doi.org/10.1007/s10916-020-01624-4.
• McLean, G., & Osei-Frimpong, K. (2019). Hey Alexa … examine the variables influencing the use of artificial intelligent in-home voice assistants. Computers in Human Behavior, 99(4), 28–37. https://doi.org/10.1016/j.chb.2019.05.009
• Nass, C., & Moon, Y. (2000). Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues, 56(1), 81-103. https://doi.org/10.1111/0022-4537.00153
• Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. In C. Plaisant (Ed.), Conference companion on Human factors in computing systems – CHI ’94 (p. 204). New York, New York, USA: ACM Press. https://doi.org/10.1145/259963.260288
• Sætra, H. S. (2021). Social robot deception and the culture of trust. Paladyn, Journal of Behavioral Robotics, 12(1), 276-286. https://doi.org/10.1515/pjbr-2021-0021
• Smarthome Assistent. (2021). So erklären deutsche Nutzer ihre Zuneigung für Alexa – Alexa ich liebe dich. Smarthome Assistant. Retrieved, 1, 2022, from https://www.smarthomeassistent.de/so-erklaeren-deutsche-nutzer-ihre-zuneigung-fuer-alexa-alexa-ich-liebe-dich/
• The Guardian. (2020). ‘Alexa, I love you’: How lockdown made men lust after their Amazon Echo. The Guardian. Retrieved 12, 2021 from https://www.theguardian.com/technology/2020/jul/20/alexa-i-love-you-how-lockdown-made-men-lust-after-their-amazon-echo
• Xie, T., & Pentina, I. (2022). Attachment Theory as a Framework to Understand Relationships with Social Chatbots: A Case Study of Replika. In T. Bui (Ed.), Proceedings of the Annual Hawaii International Conference on System Sciences. Hawaii International Conference on System Sciences. https://doi.org/10.24251/HICSS.2022.258

About the Author

Marisa Tschopp has studied Organizational Psychology at the Ludwig-Maximilians-University in Munich. She is conducting research about Artificial Intelligence from a humanities perspective, focusing on psychological and ethical aspects. She held different talks like for TEDx and is also board member of the global Women in AI (WAI) initiative. (ORCID 0000-0001-5221-5327)

Links

• https://dl.icdst.org/pdfs/files4/19d87f5e161a4400ee99b6a2c708b0ed.pdf
• https://futurium.ec.europa.eu/en
• https://hongkongfp.com/2021/08/29/always-there-the-ai-chatbot-comforting-chinas-lonely-millions/
• https://journals.sagepub.com/doi/10.1518/hfes.46.1.50_30392
• https://psycnet.apa.org/doiLanding?doi=10.1037%2F0033-295×.99.4.689
• https://socialprocesseslab.iwm-tuebingen.de/
• https://www.aboutamazon.in/news/devices/customers-in-india-say-i-love-you-to-alexa-19-000-times-a-day
• https://www.aies-conference.com/2022/
• https://www.coli.uni-saarland.de/courses/agentinteraction/contents/papers/Nass00.pdf
• https://www.euronews.com/next/2021/08/26/meet-xiaoice-the-ai-chatbot-lover-dispelling-the-loneliness-of-china-s-city-dwellers
• https://www.frontiersin.org/articles/10.3389/frobt.2018.00114/full#:~:text=Social%20presence%20was%20first%20conceptualized,core%20components%20of%20social%20presence
• https://www.prolific.co/
• https://www.sciencedirect.com/topics/engineering/human-machine-interaction
• https://www.smarthomeassistent.de/so-erklaeren-deutsche-nutzer-ihre-zuneigung-fuer-alexa-alexa-ich-liebe-dich/
• https://www.theguardian.com/technology/2020/jul/20/alexa-i-love-you-how-lockdown-made-men-lust-after-their-amazon-echo