Exploring the Limits of AI Companions: Agency, Responsibility, and the Capacity for Care

Abstract

The rapid advancement of Large Language Models has led to the widespread adoption of AI companions in emotional support, raising fundamental questions about their agency, responsibility, and capacity for care. This research examines whether AI companions can provide good care and how to understand and allocate their agency and responsibility. Adopting an interdisciplinary perspective grounded in anthropology, philosophy, and sociology, this research primarily employs Actor-Network Theory (ANT), care ethics, and Foucault’s “technologies of the self” to bridge the gap between theoretical discourse and empirical studies in AI research. The study develops an integrated analytical framework through case analyses of mainstream AI companion platforms such as Replika and Character.ai, combined with a systematic examination of user experience reports and media coverage. The findings reveal AI agency stems not from autonomous consciousness but from its ability to “make a difference” within social networks. Due to the lack of moral understanding and sentience, AI cannot function as a genuine moral agent or bear direct responsibility, necessitating the establishment of distributed responsibility allocation mechanisms. Furthermore, AI exhibits fundamental limitations in providing care, with its “logic of choice”-driven design failing to achieve good care. This research proposes AI should be positioned as a supportive tool rather than an independent caregiver, responsibility allocation should adopt collective responsibility mechanisms similar to those used for global issues, and vigilance is needed regarding new power dynamics introduced by AI. These conclusions provide fresh perspectives for AI companion development and governance while contributing to the theoretical framework for human-AI coexistence.

Presenters

Meiting Wang
Student, Master in Research, University of Auckland, New Zealand

Details

Presentation Type

Paper Presentation in a Themed Session

Theme

2025 Special Focus—Learning from Artificial Intelligence: Pedagogical Futures and Transformative Possibilities

KEYWORDS

AI Companions, Agency, Responsibility, Ethics of Care, Human-AI Interaction