Generative AI's Three Body Problem

Navigating the Complexities of AI Ethics

The rapid advancement of artificial intelligence (AI) has given us a world where computer technology now generates text, images, and even decisions that mimic human intelligence. Yet, this progress comes with profound ethical challenges

Reed Hepler and Crystal Trice both gave great Library 2.0 presentations on these challenges last week: Reed spoke on “Creating an Ethical AI Framework: How to Create an Ethical and Practical AI Framework for Your Library, Staff, Patrons, and Yourself,” and Crystal spoke on “Truth and AI: Practical Strategies for Misinformation, Disinformation, and Hallucinations.” 

After listening to both presentations, it was compelling to me to think about both topics using the tripartite framework that was at the heart of Reed’s material: 

1. the AI training data (and the training process);

2. the AI output (and associated human feedback learning); 

3. the user.

At the risk of bringing in another science fiction connection (it’s fun, though!), Cixin Liu’s science fiction novel The Three Body Problem refers to the complex and unpredictable interactions of three celestial bodies under gravitational forces where their interactions defy simple prediction. This may not be a bad way to describe the tripartite framework for thinking about AI ethics and “truth” (in quotes because of this), where the interplay of AI training, outputs, and users creates complex ethical challenges that resist simple solutions. 

Ultimately, ethical AI requires a human-centered approach, with clear agreements on how to responsibly control AI tools. Ethics in AI can’t really be about programming morality into machines, it has to be about empowering users to make ethical choices and about teaching us humans to interact with these systems thoughtfully, transparently, and with accountability. If we cede control of the ethics to the providers of the AI, or to the AI itself, we’ll be making a mistake.

The First Body: Training Data 

AI systems are only as good as the data they're trained on, and that foundation is riddled with historical and cultural biases. Large language models (LLMs) draw from vast datasets of written and transcribed content. These repositories disproportionately feature content created by Western cultures, to be sure, embedding societal prejudices and perceived truths into the AI's core. And as Crystal pointed out, things that humans thought for centuries (and even millennia) have sometimes turned out not to be accurate or “true,” but LLMs are trained based on frequency of language, and the connection between frequency and truth is tenuous. And with an increasing amount of content being generated by LLMs, which is likely to find its way into current and future training, it creates a kind of recursive bias paradox.

Copyright issues add another layer of ethical concerns. Models are trained on copyrighted materials from sources like newspapers, books, code, and social media without explicit consent. Proponents argue this qualifies as "fair use" since data is transformed into mathematical representations and discarded, but transparency remains lacking, leading to lawsuits and debates over intellectual property rights.

The Second Body: Outputs 

I’m including in output not just the LLM prompt responses, but also “Reinforcement Learning from Human Feedback (RLHF),” which creates a very real dilemma: it seems obvious this training is needed because of societal expectations and political pressure, but those expectations can and do change, removing any real sense of objectivity. Just as algorithms designed by humans can emphasize certain viewpoints, human trainers, aiming for user acceptance rather than balanced perspectives, further skew the results.

AI outputs can be remarkably creative, but as I’ve argued, everything they create is “fabricated,” therefore some of it will accurately reflect our current beliefs about what is right or true, and other times it will not–and when it doesn’t, we call that “hallucinations.” We talk about false information falling into three categories: misinformation (unintentional falsehoods), disinformation (deliberate manipulation), and malinformation (true info twisted for harm). I believe that these are distinctions of human intent, and while training LLMs can reflect these categories, I think it would be a mistake to see them as causally applicable to LLMs.

Additionally, the "black box" nature of AI with opaque processes that even the creators don't fully grasp, makes figuring out any problematic aspects of AI output hard to do. 

I’m also concerned with the way that LLMs misrepresent themselves in almost all conversations, ostensibly to make us feel comfortable, but in ways that are very problematic for me:

1. Referring to themselves as human, or saying “we” or “us” when talking about humans experiences.

2. Claiming something to be true or factual when, as just discussed, they don’t have the cognitive tools to question or test those claims;

3. Using psychographic profiling to build rapport with us, which can mean agreeing with us or giving priority to encouraging us rather than objective feedback.

I’ll be the first to say that the third one, the sycophantic nature of LLMs, is encouraging and that I respond positively to it on an emotional level. We surely have evolutionary triggers to indicate friend or foe, and AI is very good at making me see it as a friend. The amplification of user bias this approach produces is particularly insidious, but the marketplace will demand agreeable and kind AI responses, so I don’t think the providers with their financial incentives will have much choice. But I’m bothered by it.

The Third Body: Users

Users are both the linchpin in AI's ethical ecosystem and the weakest link. I personally don’t think we evolved for truth but for survival, meaning that shared stories and beliefs, rather than rational thinking, were critical to human survival during the long Paleolithic period during which our modern brains were largely formed. This is why Plato’s Allegory of the Cave still resonates as a fairly accurate depiction of the nature of the human condition. Edward O. Wilson famously said in an interview: “The real problem of humanity is the following: We have Paleolithic emotions, medieval institutions and godlike technology. And it is terrifically dangerous, and it is now approaching a point of crisis overall.” If we try to pretend that we’re entirely rational and objective, we’re not being honest about the ethical dilemmas (and dangers) of AI.

First, as users, we have to be aware of the biases and inadequacies of AI training and output. I can’t tell you how hard it is for me to see people asking, and then quoting the response from, an LLM about a topic, using that response as “proof” of a particular claim or point of view.

Second, we need to be careful with what we share when prompting AI, as electronic technologies don’t have a good track record of protecting the information we give them. Because of the power of AI gives nefarious actors to represent themselves as someone else, there are now regular stories about AI scams using private data, and that is only going to occur more often. As well, commercial organizations and governmental agencies may end up using our personal information in ways that we could be uncomfortable with.

Third, we have to be aware of our own cognitive shortcomings, biases, and triggers, reminding ourselves that we are just as prone to being manipulated by (and through) AI as we are by other individuals or institutions, regardless of intent. The building up of our own personal checks and balances with AI is going to be important. We recognize this need already for checks and balances in other parts of human culture: the principle of being presumed innocent until proven guilty, being tried before a jury of our peers, the balancing of powers in government, the scientific method when investigating casualty, peer review as a part of publishing important findings… and ultimately, the understanding that power can insidiously changes motives and often corrupts those who accumulate it.

And fourth, we need to recognize that AI is also going to make it easier to fake images and video, to cheat, to allow us to “cognitively offload,” and to facilitate other temptations, shortcuts, and bad behavior. It’s really important that we talk about all of this with each other and with students. We need to understand that language and visual imagery are such powerful forms of influence that the dangers of not understanding their potential to evoke emotions, persuade, or even propagandize us will likely have grave consequences.

This list of user dangers is not comprehensive, but a good start to building our own frameworks for understanding and using AI.

Moving Forward 

Navigating AI's three-body problem feels like a pretty daunting task. I’m reminded of Clay Shirky’s book Here Comes Everybody and his description of the time period after the invention of the printing press. He said that we think of the changes that took place as linear and orderly, but they were anything but. They were chaotic, messy, and transformative, resulting in a disruptive upheaval in communication, culture, and society. That new technology destabilized existing institutions, sparked widespread experimentation, and fueled debates, including the Reformation, while taking decades for its full impact to stabilize. Clay was comparing the Internet to the printing press, and it will be interesting to see if we end up seeing the Internet as just a stepping stone to AI as part of a dramatic transformation of human life.

Thanks to Reed and Crystal for lighting the way a bit, and here’s to working together as we venture into the unknown.

This post was edited for clarity shortly after original publication.