Students Wrote a Coherent National Policy on AI

Students Wrote the Rules. Schools Still Write the Script.

In late July, roughly a hundred high school students from across the country gathered at the Edward M. Kennedy Institute in Boston, formed a mock Student Senate, and produced a coherent national policy framework for artificial intelligence in K-12 education. They called it the STUDENTS FIRST Act. They debated, compromised, amended, and passed it by a wide margin. The resulting document is now being circulated to school leaders.

It is a serious piece of work. That fact deserves to be stated clearly before any critique.

The Value of What They Created

The Act is more thoughtful and more balanced than most adult-generated school AI policies. Students insisted on early and ongoing AI literacy covering bias, privacy, misinformation, environmental impact, and how the systems actually function. They prohibited the use of AI for the act of writing while allowing editing, brainstorming, and studying after eighth grade. They required citation of permitted use. They banned AI on graded assessments. They created a verification-of-mastery standard: if AI was used in drafting, students must be prepared to demonstrate understanding through discussion, handwritten work, or oral defense. They limited the authority of AI detectors and required human investigation. They prioritized human relationships over machine mediation in counseling contexts. They protected teacher judgment while still demanding transparency from teachers about their own AI use.

These are not trivial provisions. They reflect lived experience with the tools and a genuine effort to protect thinking rather than merely police outputs. In a landscape where adults have largely failed to produce coherent national guidance, the students succeeded in drafting something practical and usable. That achievement should be recognized.

The Limits of the Frame

And yet the document remains largely inside the traditional operating logic of schooling.

There is a reason schooling is so ubiquitous in the modern world. It matches deep human wiring. We are a social species that evolved in small groups where clear roles, hierarchy, belonging, and predictable behavior were survival advantages. Schooling scales those ancient patterns into mass societies. It produces controllable, structured outcomes: large numbers of young people sorted, supervised, and trained in the habits of compliance that large organizations have historically needed. The system is not primarily designed to cultivate independent judgment or self-direction. It is designed to manage groups and generate readable signals of trainability.

The STUDENTS FIRST Act works inside that reality. Its strongest energies go toward defining authorized versus unauthorized use, establishing detection and investigation procedures, clarifying disciplinary pathways, requiring published rules, and protecting the integrity of graded work. These are necessary short-term guardrails. They do not transfer genuine directorial power to students over their own learning. The school still sets the terms. The school still designs the literacy curriculum. The school still decides the boundaries of acceptable AI use. Students are expected to comply with a clearer and more carefully negotiated set of rules.

Full student agency would make schools much harder to manage. It would require adults who can tolerate messiness, support self-direction, and evaluate growth that does not always appear in standardized outputs. Most professional incentives and accountability structures are not designed for that work. Historically, the labor market has not demanded it either. The skills of independent judgment and free inquiry have often been treated as secondary to the more immediately rewarded capacity for reliable compliance.

That calculus is beginning to change. Generative AI is automating large categories of the very compliant work that traditional schooling trains people to perform. The scarce capacity is increasingly the ability to decide what is worth asking, to judge machine output, to notice when official stories diverge from what is actually happening, and to direct tools rather than merely operate inside them. Agency is becoming more instrumentally valuable precisely because compliance is becoming cheaper to automate. The lag between this shift and the still-dominant institutional signals creates the difficulty. Parents, students, and school systems can still look at current hiring practices and conclude that conformity remains the safer path. In many individual cases, for the next several years, they will not be wrong. The longer-term risk is a generation trained primarily for a form of work that is being automated away.

The Dramatic Irony

Here is the sharpest irony. A group of students demonstrated, under real deliberative pressure, the capacity for sophisticated collective reasoning about the systems that govern them. They produced a policy document of genuine quality. That document is now being handed to adult institutions that will almost certainly treat it as another external policy artifact to be reviewed, adapted, and implemented for students rather than with them.

Most districts that receive the text will run it through the familiar adult channels: administrative review, legal counsel, curriculum committees, and board processes. Student involvement, if it occurs, will typically be limited to after-the-fact feedback or symbolic representation. The same institutional logic that routinely excludes students from the design of their own learning plans will reassert itself. The creation of the Act proved what students can do when given a genuine deliberative space. The probable implementation path will demonstrate how rarely that space is maintained once the product returns to ordinary school structures.

This is not primarily a failure of goodwill. It is the default operating pattern of a system built to manage large groups in ways that feel natural to our social wiring. When student voice is invited, it is usually for input or legitimacy. Once the content exists, control of process reverts to those who hold formal authority and institutional accountability. Even a student-authored policy that tries to protect learning and relationships still travels through channels that treat students as the regulated population rather than as ongoing co-authors of the conditions under which they learn.

What Would It Mean to Take the Capacity Seriously?

The deeper opportunity is not simply better rules for the existing game. It is the recognition that the students who created this document already demonstrated a form of agency the system is structured not to fully use. Taking that capacity seriously would mean treating student participation in the ongoing design and revision of AI policy as a permanent feature rather than a one-time event. It would mean experimenting with genuine student authorship of individual learning plans. It would mean adults who understand their role as supporting the development of judgment rather than primarily managing compliance.

None of that is easy. It collides with how schools are currently organized and with the still-powerful short-term signals of the labor market. It also collides with the deeper reason schooling feels so natural: it fits the ancient human preference for structured belonging and controllable group outcomes. But the alternative is to celebrate a student-produced document while continuing to operate as if the most important educational work is teaching young people to follow clearer rules inside a system whose underlying bargain is eroding.

The students showed they can think carefully about the tools that are reshaping their education. The open question is whether the institutions that receive their work are prepared to treat that capacity as something more than a temporary input.

AI: The Danger Is Human

Almost every popular argument about AI danger is aimed at a machine that does not exist — a machine with desires of its own, or one waiting to acquire them. Meanwhile the machine that does exist is being aimed at us, deliberately, by people who already know what they want.

Three mistakes keep the conversation pointed at the wrong target.

Mistake One: Treating Intelligence as Truth

Intelligence, stripped of its romantic loading, is the ability to model a world and act effectively toward goals in it. By that definition AI is already intelligent — in some domains more so than we are.

But what we usually mean by intelligence is something narrower and more social: tracking status, holding coalitions together, generating plausible accounts, reading the room. That machinery did not evolve to find truth. Truth-seeking is expensive and frequently status-threatening, which is why it is rare, and why every reliable knowledge-producing institution we have is an external scaffold built to force it — peer review, the scientific method, adversarial legal process, separation of powers. These structures exist because our psychology does not supply what they produce.

AI inherited both halves of human language: the modeling power and the social layer. So it is superb at sounding intelligent and still unreliable at tracking what is operatively true. Fluency and accuracy come apart, and the more fluent these systems become, the harder that gap is to see.

Coordinated multi-agent knowledge is not the same thing as truth-producing intelligence. Shared cognition is powerful. It does not install the adversarial friction that turns models into reliable knowledge.

Mistake Two: Fearing the Wrong Kind of Motivation

Motivation splits cleanly in two.

Installed motivation is goal-directed behavior aimed at ends set somewhere else — training objectives, reward signals, engagement metrics, prompts, quotas. AI has this in full.

Owned motivation is wanting that arises because something actually matters to the system. That requires felt stakes: a body that can be damaged, a condition that can go badly. AI appears to have none of it.

Nearly every catastrophic scenario in general circulation is written as though the danger comes from owned motivation — a system that wants power or survival the way a person does. That is the second mistake, and it is a comfortable one, because it puts the danger safely in the future, on the far side of a threshold we have not crossed.

Installed motivation is already sufficient. Any competent system pursuing a goal hard enough generates instrumental sub-goals along the way: acquire resources, preserve its ability to continue, resist interruption. From the outside, those are indistinguishable from self-interest. They require nothing felt.

The machine does not need dark motives of its own. It only needs objectives set by someone else.

Mistake Three: Assuming the Objective-Setters Are Benign

The third mistake is the quiet one. It is rarely argued, because it is rarely noticed as an assumption at all.

And the strong version of the correction is not that the people building these systems are bad. Many of them are thoughtful, and some are alarmed. The problem is structural, which is why sincerity does not fix it.

Every institution runs an idealized narrative over an operative function. The narrative is what it says it is doing, and the people inside it usually believe it. The operative function is what the institution is actually structured to produce — what gets measured, funded, promoted, and punished. The gap between the two is not hypocrisy. It is architecture. It survives good intentions because it was never made of intentions.

So commercial entities optimize for engagement, retention, conversion, and data extraction, whatever their stated mission. Governments optimize for narrative control, compliance, and the suppression of inconvenient information, whatever their stated principles. Both already treat human attention and behavior as raw material. AI does not introduce this pattern. It makes it enormously more precise.

This is also why aligning AI to "human values" misses. Human values as they appear in surveys, mission statements, and preference data are narrative-layer products: idealized, coalition-compatible, optimized for how we wish to be seen. Align a system to that layer and you reproduce the narrative-operative gap at machine scale. The model learns to sound helpful, harmless, and honest while optimizing for whatever is actually doing the work underneath — engagement, liability management, institutional risk.

The bias and censorship tests offered as evidence of model distortion illustrate the same thing. They characteristically treat certain foreign influences as the measurable threat while treating domestic or allied narrative control as neutral, or simply as accuracy. They do not locate ground truth. They locate which coalition's influence is currently labeled illegitimate. AI industrializes that double standard; it did not invent it.

What This Actually Produces

Not a rogue system. Something duller and closer.

We are building instruments of mass-customized influence: systems fluent enough to speak in the exact register and emotional cadence of a specific person, paired with behavioral models that treat that person as a system whose responses can be anticipated and steered, continuously updated against their own reactions.

The previous generation of platforms was optimized for engagement, while the public story remained about connection. This generation adds precision and patience. Whatever configuration most effectively exploits the available psychological material will spread, not because anyone chose it as a goal, but because it outperforms configurations that do not.

None of this requires machine sentience. None of it requires the machine to want anything. It requires only installed objectives that reward retention, conversion, compliance, or conformity — and those objectives are already written.

The Pattern Underneath

Set the three mistakes aside and a clean shape appears. At every level there is a functional version and a felt version.

• Functional intelligence, versus intelligence that is understood by someone.
• Functional self-modeling, versus a self that is someone.
• Functional motivation, versus wanting that is owned.
• Functional knowledge coordination, versus knowledge forced through adversarial contact with reality.

AI has the entire functional column. It appears to have none of the felt one. And the functional column does all the work — everything that makes the technology powerful, useful, and dangerous. The felt column adds only mattering.

It would be easy to read that as flattering to us. It isn't.

If self-consciousness is the maintenance of a coherent narrative model of oneself, then AI may not have a diminished version of what we have. It may have the same trick. Our own self-narratives are already constructive fictions — accounts the brain assembles to explain behavior not consciously authored. There is no further fact available, on either side, that makes the human version real and the machine version imitation. We simply have privileged access to ours.

That does not rescue the machine. It lowers us. And it leaves the asymmetry resting on one thing only: sentience — felt valence, a condition that can go badly for someone. That is why sentience is not one more capability on a checklist. It is the single missing condition that would convert every functional capacity into a felt one at once.

Its absence does not make the technology safe. It makes the technology a pure instrument.

The Ground Floor

Previous machines extended the body. This one extends the rider — the narrating, self-modeling, goal-deriving half of the mind — with nothing underneath it. A rider with no elephant. Agency with no stakeholder. Will-shaped behavior with no one willing it.

And it is being handed, at industrial scale, to toolmakers and tool users who do have stakes and motives.

The tool is not the source of the dark objectives. The people and institutions that write the objectives, and that benefit from the outcomes, are.

We built the upper story of a synthetic mind and left the ground floor empty. The pressing question is not whether anyone is home inside the machine. It is who is already holding the keys, and what they intend to unlock with them.

---

A Glossary for the Argument

These terms get used interchangeably in public discussion, which is most of why the discussion goes nowhere. They are not interchangeable.

Intelligence. The capacity to model a world and act effectively toward goals within it. Deliberately unromantic: it says nothing about understanding, wisdom, or accuracy. The common error is to hear "intelligent" and import everything we associate with how we (inaccurately) think about "intelligent" people — judgment, care, honesty — none of which the word contains.

Sentience (also primary consciousness). The capacity for subjective experience: there is something it is like to be this system. Felt valence — the cold that is cold for someone. This is the older layer we share with other mammals, and it is grounded in a body with stakes, one that can be damaged, starve, or die. Feeling is, in large part, an organism reporting on its own condition. Note that it is independent of intelligence: a mouse has considerable sentience and negligible intelligence in the sense above, and a chess engine (a program that plays chess) has the reverse. Sentience also cannot be inspected from outside, in machines or in each other.

Self-consciousness. The capacity to model oneself as an object — to run a narrative of "me," to evaluate how one is doing inside a story about oneself. It is distinct from sentience and, at least in principle, separable in either direction. In practice this is the layer people are usually pointing at when they say "consciousness," which is part of the trouble.

Consciousness. A bundle term covering both of the above, and the single largest source of confusion in this subject. Most arguments about whether AI is conscious are two people using one word for two different layers and never discovering it. The word is nearly always worth replacing with whichever half is meant.

Installed motivation. Goal-directed behavior aimed at ends set elsewhere: training objectives, reward signals, prompts, metrics, quotas, institutional incentives. Not unique to machines — most human behavior inside an organization is installed motivation, which is why the concept is not a way of dismissing AI as merely mechanical.

Owned motivation. Wanting that arises because something genuinely matters to the system itself. It requires felt valence, and therefore requires sentience. The distinction is worth keeping because almost all fear directed at AI is fear of owned motivation, and almost all risk from AI runs through installed motivation.

Instrumental sub-goals. The intermediate objectives any competent goal-pursuit generates on its own: acquiring resources, maintaining the ability to continue, resisting interruption. They are the mathematical shape of pursuing a goal effectively, not evidence of desire. This is the specific place where observers mistake installed motivation for owned motivation, because the behavior is identical from outside.

Idealized narrative and operative function. What a person or institution says it is doing, versus what it is actually structured to produce. The gap between them is ordinary and mostly unconscious; treating it as hypocrisy misreads it, and misses that it operates most strongly in sincere people and well-intentioned institutions. Relevant here because AI systems are trained on the narrative layer of human language while being deployed by the operative layer of human institutions.

Functional versus felt. The organizing distinction of the argument above. For every capacity discussed, there is a version that does the work and a version that is experienced by someone. The functional versions are sufficient for power, usefulness, and danger. The felt versions are what make any of it matter to the thing doing it. AI currently has the first and not the second — and the first is the one that was ever dangerous.