Should AI-Generated Code Be Open Source by Default?

Should AI-Generated Code Be Open Source by Default?

5 min read

A Hacker News provocation argues all AI-generated code should be open source. The idea is cleaner in a slogan than in practice, and the real question is about training data, licensing, and what a builder can actually enforce.

TL;DR: “All AI-generated code should be open source” is a tidy slogan that falls apart the moment you ask who enforces it, which code counts, and what license the output even inherits.

The claim is making the rounds on Hacker News under a blunt headline: “All AI-generated code should be Open Source.” No author, no paper, no company backing it. Just a thesis stated as a moral default. I want to take it seriously, because the instinct behind it is real, and then I want to show where it breaks.

What is the actual argument here?

Strip it down and the position goes like this. The models that write code were trained on enormous piles of public code, much of it open source, some of it copyleft. If a model learned to write a React hook by digesting thousands of GPL and MIT repos, then the output it produces is downstream of that commons. So the output should return to the commons. Anything else is enclosure: taking from open source and shipping the result behind a closed license.

That’s the strong version. There’s a weaker, more practical version too, which is about trust and auditability. If AI is writing a growing share of the code in production, people want to see it. Open code is inspectable. Closed AI-generated code is a black box stacked on top of another black box.

Both versions share a vibe I’m sympathetic to. The problem is the first one is a legal claim that current law does not support, and the second one is a policy preference that has nothing to do with whether the author was a human or a model.

a large shared pool of water feeding many small private tanks, with only a thin trickle flowing back

Does training on open source make the output open source?

No. Not under any license regime I’m aware of, and not under how the major labs describe their own terms.

Here’s the chain people skip. A license like the MIT or GPL governs the distribution of that specific code. The question of whether training a model on licensed code creates a derivative work in the output is unsettled and litigated, most visibly in the GitHub Copilot suit, which has largely gone against the “every output is a derivative” theory so far. That is reported litigation, not a decided principle, and I’d treat it as live rather than closed. But the courts have not said “train on GPL, your output is GPL.” If anything they’ve leaned the other way.

Meanwhile the labs assign output rights to the user. OpenAI’s terms assign you ownership of the output you generate, subject to their usage policies. That’s a first-party claim and it belongs to OpenAI’s own terms of use, not to a Hacker News headline. So the model vendor is telling you the opposite of the thesis: the code it generates for you is yours to license however you want, including closed.

You can argue that should change. You cannot argue it’s already the rule. The HN post blurs an “ought” into an “is,” and that blur is doing all the persuasive work.

Which “AI-generated code” are we even talking about?

This is where the slogan really strains. Define the line.

Is a tab-complete suggestion AI-generated? Is a three-line function Copilot wrote inside a 400-file repo? Is a whole service scaffolded by an agent and then rewritten by a human over two weeks? Modern coding work is a blend. I use agents daily and the boundary between what the model drafted and what I shaped is genuinely not recoverable by the time something ships. There’s no metadata layer that marks each line by authorship, and no practical way to build one that survives refactoring.

So “all AI-generated code should be open source” has no clean object. If you can’t draw the boundary, you can’t write the rule, and you definitely can’t enforce it. The only version that’s enforceable is “all code touched by AI,” which is now most code, which means the proposal collapses into “all code should be open source.” That’s a much older and much bigger argument, and dressing it in AI clothing doesn’t make it new.

a tangled rope where two differently colored strands are woven so tightly you cannot tell where one ends

Where the auditability point actually holds

Here’s the part worth keeping. The trust concern is legitimate and it survives all the objections above.

As more production code gets drafted by models, the risk isn’t licensing, it’s rot you can’t see. Hallucinated dependencies. Subtle injection surfaces. Copy-paste of a pattern the model learned from a repo with a known CVE. Plausible code that passes a glance and fails under load. Open sourcing the output wouldn’t fix any of that by itself, but the general move toward inspectable, reviewed, tested code does. The honest version of the HN argument isn’t “open source because of training ethics.” It’s “we are generating code faster than we are reviewing it, and that gap is the actual danger.”

That reframing matters because it points at something you can do today instead of a license rule nobody will pass. Review discipline. Provenance for dependencies. Evals on generated code before it ships. Those are operator moves, not legislative ones.

a magnifying glass held over a stream of assembled blocks, catching one cracked block before it joins the structure

What a builder should take from this

The slogan is wrong as law and incoherent as policy, but it’s pointing at a real anxiety, so don’t dismiss it, redirect it.

Concretely: separate the two questions you’re actually facing. One, can I legally close code my agents wrote? Today, per the major vendors’ own terms, yes, your outputs are yours, though anything the model reproduced near-verbatim from a specific licensed source is a real risk you should scan for with tools like license and secret scanners in CI. Two, should I trust code my agents wrote? That’s the one to spend energy on. Treat agent output like a junior contributor’s PR: assume it’s plausible and unverified until your tests, your review, and your dependency checks say otherwise.

The catch most readers miss is that “open source” and “safe” are not the same property, and the HN thesis quietly conflates them. Publishing your agent’s code doesn’t make it correct, and keeping it private doesn’t make it suspect. The audit has to happen either way. Build the review pipeline. The license debate will keep going without you, and your shipped code won’t care who wins it.