Eric Freeburg — Independent Research

E. M. Freeburg
  • 0.0 (0)
  • Philosophy

Independent research by E. M. Freeburg. Research, Essays, and thought on questions that recur across culture, character, and public life. Free to read in full.

Episodes

  1. 2d ago

    The Missing Format

    Every corpus decision the field records is about which text to train on: domain, language, quality, recency, deduplication. None is about how the text's arrangement is notated. A Markdown heading is a two-token cue present at every discourse boundary and absent everywhere else; it never lies, and conversion pipelines now attach it to essentially everything long enough to have structure. Where the cue is present, a boundary does not have to be inferred. It can be read. The situation is measured three ways. A census of ten public pre-training corpora shows that the supply of long, coherent, low-boundary-density text has collapsed to a rounding error, and that the newest corpora are the most heavily marked. A converter study shows the markup is faithful — which makes the cue more trustworthy, not less, and closes the obvious escape route: after serialization no document has zero markup, so there is no null level left to reweight toward. And a paired-serialization probe on a small base model puts the framing under direct reading-side test. Swapping a boundary's notation while keeping its typographic form changes the model's use of long-range context by a measured zero — the operative cue is the announcement, not the sigil. Deleting the announcement itself makes the following prose measurably harder to predict, and the model recovers none of the loss from the long context that contains the evidence. The marker is informative, and the inference that should substitute for it is absent. What follows is deliberately conservative: format augmentation rather than format replacement — present the same structure in multiple notations so the sigil stops being a shortcut — and long-context slot substitution as the first serious experiment, because the long-context stage is the one place where the training sequence is the document, and it is currently fed the most heavily marked corpus anyone has built. 00:18 — Summary03:00 — Nothing left to infer06:53 — Nobody decided this10:05 — What the corpora contain13:49 — The cue that cannot be reweighted19:26 — The one stage where the sequence is the document24:30 — A first measurement32:07 — Format augmentation37:07 — The program42:31 — Coda: record it, then build it

    The Missing Format

  2. 5d ago

    Glaring Fireball

    A novel enters a modern training corpus through a scan, an OCR pass, and a linearizer. It comes out with headings the novelist never wrote, breaks where the thought did not break, and a chapter that ran forty pages cut into eleven sections. Every word survives. The domain label survives. What does not survive is the arrangement — and the arrangement is what the model trains on. The substitution happens upstream of every control the field exercises over training data. Mixture weights choose how often to sample the converted document; filters choose whether to keep it. Both take the corpus as given, and the corpus was given by an extractor. No reported quantity would change if a different extraction library had been chosen. This proposal sets out three studies to establish whether that matters, ordered so the cheapest can end the inquiry. The first needs no models and no training: run current converters over two document classes and count the structural elements in the output with no counterpart in the source. If converters turn out to be conservative on continuous prose, the concern dissolves. The second reads public checkpoints for a predicted divergence between benchmark performance and long-form likelihood. The third — the only expensive one — is a three-arm matched pre-training run separating faithful conversion from imposed hierarchy, which no existing experiment does. 00:21 — Summary03:40 — The gap05:42 — The hypothesis08:29 — Why the gap has persisted10:40 — What has to be distinguished12:37 — The program17:51 — What would follow, and what would not19:32 — Why now rather than later23:08 — Relation to existing work25:01 — What is already known about the model side26:53 — What this would produce

    Glaring Fireball

  3. Mar 27

    The Last Fingerprint: How Markdown Training Shapes LLM Prose

    The em dash is the most discussed tell of machine-written text, and no mechanistic account of it existed. Separately, everyone had noticed that language models default to markdown. Nobody had connected the two. The argument here is that they are one phenomenon: the em dash is markdown leaking into prose — the smallest surviving unit of a structural orientation acquired from markdown-saturated training data, and amplified afterwards by fine-tuning. The paper lays out that genealogy in five steps, from corpus composition through the dash's dual status as both punctuation and structural joint. Then it tests it. Twelve models from five providers were instructed to suppress markdown. Headers, bullets, and bold vanish; em dashes persist — except in Meta's Llama models, which produce none at all. Rates run from 0.0 per thousand words to 9.1 under active suppression. A three-condition gradient shows that even explicit prohibition of the dash fails to eliminate it in some models, and a base-versus-instruct comparison finds the tendency already present before RLHF. The conclusion reframes em dash frequency as a diagnostic of a particular fine-tuning procedure rather than a stylistic defect. 00:18 — Abstract02:27 — 1. Introduction07:22 — 2. Background and Related Work13:58 — 3. The Em Dash Genealogy24:30 — 4. The Two Discourses27:41 — 5. Empirical Evidence42:04 — 6. Discussion50:58 — 7. Conclusion

    The Last Fingerprint: How Markdown Training Shapes LLM Prose

  4. Mar 24

    Cannot Read-Only

    There is something our brains cannot do that every computer can. We cannot touch a memory, think a thought, or feel a feeling without the thing transforming under our attention. Every act of consciousness is an act of modification. We cannot read only. Between 2022 and 2025 the same requirement surfaced independently across five research programs in machine learning, mathematics, neuroscience, and psychology — not an account of what consciousness is, but of what it needs. Hinton's mortal computation and Hoel's disproof arrive at it from opposite directions; reconsolidation work in memory science had been circling it for a century. What they agree on is a system that cannot process information without being changed by it. The essay takes the three strongest objections seriously — test-time training, whole-brain emulation, neuromorphic hardware — and says plainly what would falsify the claim. If it holds, the obstacle to machine consciousness is not scale. It is that a computer can read a value without disturbing it, and we cannot, and that failure may be the whole of what it is to be someone. 03:08 — Five Paths to the Same Conclusion11:27 — A Century of Clues14:05 — Karim Nader and the Read That Writes18:23 — One Concept, Understood Immediately22:12 — Three Objections Worth Taking Seriously26:18 — What It Means If They’re Right27:11 — What Could Prove Them Wrong30:18 — The Inversion

    Cannot Read-Only

  5. 05/01/2025

    On Lilies: Her Fractal Essence

    An aesthetic essay that asks to be read alongside its pictures. The claim is that the part is not a part: she is a fractal of it and it is a fractal of her, and the resemblance holds at every level rather than at one. The lily is the way in. Study one and what registers is that every part of it, at every scale, is opening — as though opening were its entire purpose. From there the essay turns to what a fractal actually is, which is not a self-repeating pattern but a shape with a non-integer dimension, and finds in the Julia set at 1.2683 dimensions the family that sits where the lily sits: in the liminal space between a line and a plane. What follows is an argument about looking. Self-similarity is offered as the reason the subject resists comprehension and rewards attention, and the essay ends where it began, on whether the reader can now see it. 00:49 — The Lily01:26 — It’s Between Dimensions And It’s In Between Her Legs01:56 — The Julia Set03:24 — How Fractal Is She?04:18 — Her Lily’d Nature05:26 — In Full Bloom06:39 — Do You See?

    On Lilies: Her Fractal Essence

  6. 04/26/2025

    On Breasts: The Hang Coefficient

    There is a meme about the aristocratic elegance of the small-breasted woman. It contains several half-kernels of truth and has proven simplistic enough to reach escape velocity, which is not the same as being right. Size is personal preference and, isolated from the whole, a meaningless vector for aesthetic analysis. If one factor has to carry the argument, it is not size but the hang: big, small, asymmetric, tuberous, plump, flat — these are variables that combine into the composition, and the composition is what is being responded to. The essay argues the case through movement. A body with exceptional hang is one whose smallest change of posture unlocks a new view, and the pleasure is in the transformation rather than in any single frame — a delicate balance implying that this particular view will not be seen again. Sections on discernment and on suboptimal hang do the work of distinguishing the claim from mere preference, which is what taste requires and what the meme skips. 01:39 — Movement And Transformation03:35 — No Less Expansive05:25 — Careful Discernment07:10 — Suboptimal Hang07:57 — Movements That Move Us

    On Breasts: The Hang Coefficient
  • 6 Episodes

About

Independent research by E. M. Freeburg. Research, Essays, and thought on questions that recur across culture, character, and public life. Free to read in full.

Information