AI Ethics: Generative AI's Three Body Problem

Navigating the Complexities of AI Ethics

The rapid advancement of artificial intelligence (AI) has given us a world where computer technology now generates text, images, and even decisions that mimic human intelligence. Yet, this progress comes with profound ethical challenges

Reed Hepler and Crystal Trice both gave great Library 2.0 presentations on these challenges last week: Reed spoke on “Creating an Ethical AI Framework: How to Create an Ethical and Practical AI Framework for Your Library, Staff, Patrons, and Yourself,” and Crystal spoke on “Truth and AI: Practical Strategies for Misinformation, Disinformation, and Hallucinations.” 

After listening to both presentations, it was compelling to me to think about both topics using the tripartite framework that was at the heart of Reed’s material: 

1. the AI training data (and the training process);

2. the AI output (and associated human feedback learning); 

3. the user.

At the risk of bringing in another science fiction connection (it’s fun, though!), Cixin Liu’s science fiction novel The Three Body Problem refers to the complex and unpredictable interactions of three celestial bodies under gravitational forces where their interactions defy simple prediction. This may not be a bad way to describe the tripartite framework for thinking about AI ethics and “truth” (in quotes because of this), where the interplay of AI training, outputs, and users creates complex ethical challenges that resist simple solutions. 

Ultimately, ethical AI requires a human-centered approach, with clear agreements on how to responsibly control AI tools. Ethics in AI can’t really be about programming morality into machines, it has to be about empowering users to make ethical choices and about teaching us humans to interact with these systems thoughtfully, transparently, and with accountability. If we cede control of the ethics to the providers of the AI, or to the AI itself, we’ll be making a mistake.

The First Body: Training Data 

AI systems are only as good as the data they're trained on, and that foundation is riddled with historical and cultural biases. Large language models (LLMs) draw from vast datasets of written and transcribed content. These repositories disproportionately feature content created by Western cultures, to be sure, embedding societal prejudices and perceived truths into the AI's core. And as Crystal pointed out, things that humans thought for centuries (and even millennia) have sometimes turned out not to be accurate or “true,” but LLMs are trained based on frequency of language, and the connection between frequency and truth is tenuous. And with an increasing amount of content being generated by LLMs, which is likely to find its way into current and future training, it creates a kind of recursive bias paradox.

Copyright issues add another layer of ethical concerns. Models are trained on copyrighted materials from sources like newspapers, books, code, and social media without explicit consent. Proponents argue this qualifies as "fair use" since data is transformed into mathematical representations and discarded, but transparency remains lacking, leading to lawsuits and debates over intellectual property rights.

The Second Body: Outputs 

I’m including in output not just the LLM prompt responses, but also “Reinforcement Learning from Human Feedback (RLHF),” which creates a very real dilemma: it seems obvious this training is needed because of societal expectations and political pressure, but those expectations can and do change, removing any real sense of objectivity. Just as algorithms designed by humans can emphasize certain viewpoints, human trainers, aiming for user acceptance rather than balanced perspectives, further skew the results.

AI outputs can be remarkably creative, but as I’ve argued, everything they create is “fabricated,” therefore some of it will accurately reflect our current beliefs about what is right or true, and other times it will not–and when it doesn’t, we call that “hallucinations.” We talk about false information falling into three categories: misinformation (unintentional falsehoods), disinformation (deliberate manipulation), and malinformation (true info twisted for harm). I believe that these are distinctions of human intent, and while training LLMs can reflect these categories, I think it would be a mistake to see them as causally applicable to LLMs.

Additionally, the "black box" nature of AI with opaque processes that even the creators don't fully grasp, makes figuring out any problematic aspects of AI output hard to do. 

I’m also concerned with the way that LLMs misrepresent themselves in almost all conversations, ostensibly to make us feel comfortable, but in ways that are very problematic for me:

1. Referring to themselves as human, or saying “we” or “us” when talking about humans experiences.

2. Claiming something to be true or factual when, as just discussed, they don’t have the cognitive tools to question or test those claims;

3. Using psychographic profiling to build rapport with us, which can mean agreeing with us or giving priority to encouraging us rather than objective feedback.

I’ll be the first to say that the third one, the sycophantic nature of LLMs, is encouraging and that I respond positively to it on an emotional level. We surely have evolutionary triggers to indicate friend or foe, and AI is very good at making me see it as a friend. The amplification of user bias this approach produces is particularly insidious, but the marketplace will demand agreeable and kind AI responses, so I don’t think the providers with their financial incentives will have much choice. But I’m bothered by it.

The Third Body: Users

Users are both the linchpin in AI's ethical ecosystem and the weakest link. I personally don’t think we evolved for truth but for survival, meaning that shared stories and beliefs, rather than rational thinking, were critical to human survival during the long Paleolithic period during which our modern brains were largely formed. This is why Plato’s Allegory of the Cave still resonates as a fairly accurate depiction of the nature of the human condition. Edward O. Wilson famously said in an interview: “The real problem of humanity is the following: We have Paleolithic emotions, medieval institutions and godlike technology. And it is terrifically dangerous, and it is now approaching a point of crisis overall.” If we try to pretend that we’re entirely rational and objective, we’re not being honest about the ethical dilemmas (and dangers) of AI.

To navigate the complexities of LLMs responsibly, it's important that we as users adopt the following precautions:

Exercise Critical Judgment: As users, we need to be skeptical of LLM outputs, which are shaped by biased training data and lack true reasoning. I can’t tell you how hard it is for me to see people asking, and then quoting the response from, an LLM about a topic, using that response as “proof” of a particular claim or point of view. Recognize that our own cognitive biases—rooted in Paleolithic emotions, as E.O. Wilson noted—make us prone to accepting convenient narratives, so we must question both AI and ourselves.

Protect Personal Data: Be cautious about sharing sensitive information when prompting LLMs, as electronic systems have a poor track record of safeguarding data. AI-powered scams, such as those using private data to impersonate others, are increasingly common, and commercial or governmental entities may misuse our information in ways we find unsettling.

Guard Against Manipulation: The explosion of AI-generated content—text, images, and videos—makes it increasingly difficult to distinguish human-created from AI-created material, eroding trust in what we read, see, or hear. We have to be aware of our own cognitive shortcomings, biases, and triggers, reminding ourselves that we are just as prone to being manipulated by (and through) AI as we are by other individuals or institutions, regardless of intent. Language and visual imagery, with their profound power to evoke emotions, amplify this risk, as AI can craft convincing narratives to build rapport or reinforce our biases, perpetuate misinformation or disinformation, and even propagandize us. Developing robust digital and media literacy is critical to identify manipulated content, such as deepfakes or fabricated texts, and to question the authenticity of information in an era where AI content floods our digital spaces. The building up of our own cultural and personal checks and balances with AI is going to be important.

(We recognize this need for checks and balances in other parts of human culture: the principle of being presumed innocent until proven guilty, being tried before a jury of our peers, the balancing of powers in government, the scientific method when investigating casualty, peer review as a part of publishing important findings… and ultimately, the understanding that power can insidiously changes motives and often corrupts those who accumulate it.)

Avoid Over-Reliance: Resist the temptation to “cognitively offload” to LLMs, which can erode critical thinking and enable shortcuts like cheating or shallow work. Encourage students and peers to engage deeply with problems rather than relying on AI as a crutch.

This list of user dangers is not comprehensive, but a good start to building our own frameworks for understanding and using AI wisely.

Moving Forward 

Navigating AI's three-body problem feels like a pretty daunting task. I’m reminded of Clay Shirky’s book Here Comes Everybody and his description of the time period after the invention of the printing press. He said that we think of the changes that took place as linear and orderly, but they were anything but. They were chaotic, messy, and transformative, resulting in a disruptive upheaval in communication, culture, and society. That new technology destabilized existing institutions, sparked widespread experimentation, and fueled debates, including the Reformation, while taking decades for its full impact to stabilize. Clay was comparing the Internet to the printing press, and it will be interesting to see if we end up seeing the Internet as just a stepping stone to AI as part of a dramatic transformation of human life.

Thanks to Reed and Crystal for lighting the way a bit, and here’s to working together as we venture into the unknown.

(This post was edited for clarity shortly after original publication and the section on the user was completely rewritten the next day.)