Optimization logs

You already optimize. You just don't remember how.

Think about the last time you improved something — your task management setup, a communication habit, a workflow that was draining you. You tried several adjustments over days or weeks. Some helped. Some didn't. Some you aren't sure about.

Now try to reconstruct exactly what you changed, in what order, for what reason, and with what result.

You can't. Almost nobody can. And this is the central problem with undocumented optimization: you lose access to your own experimental history. Every insight you generated through trial and error evaporates the moment you stop paying attention to it. You're left with a vague sense that "things are better now" and no way to explain why, replicate the result, or teach someone else.

The previous lesson — benchmark before and after — gave you the ability to measure whether a change helped. This lesson gives you the ability to remember what you changed, why you changed it, and what the measurement showed. Without this, benchmarking is a one-time event. With it, benchmarking becomes a compounding knowledge asset.

The 30,000-experiment notebook

Michael Faraday kept meticulous laboratory notebooks across his entire career at the Royal Institution, recording approximately 30,000 experiments — both successful and unsuccessful. The notebooks contained detailed descriptions of apparatus design, environmental conditions, procedures, outcomes, and interspersed philosophical reflections, all written in plain language. These notebooks were so significant that they were inscribed on the UNESCO Memory of the World Register in 2016.

What makes Faraday's notebooks extraordinary is not their completeness. It's their honesty. He recorded experiments that failed with the same rigor as experiments that succeeded. He noted what he expected to happen alongside what actually happened. He recorded his reasoning — not just his procedure — so that when he revisited an entry weeks or months later, he could evaluate not just the result but the quality of his thinking at the time he designed the experiment.

This is the difference between a log and a record. A record stores what happened. A log stores what you thought would happen, what you did, what actually happened, and — crucially — the gap between expectation and reality. That gap is where all learning lives.

Faraday's contemporary, Humphry Davy, once said that his greatest discovery was Faraday himself. But Faraday's greatest tool was arguably his notebook practice. Without it, the thousands of failed experiments that preceded his breakthroughs in electromagnetic induction would have been waste. With it, each failure constrained the search space and pointed toward the next thing to try.

Why your memory is the wrong tool for this job

There are two specific cognitive biases that destroy your ability to learn from undocumented optimization, and both operate silently.

Hindsight bias — sometimes called the "knew-it-all-along" effect — is your brain's tendency to retroactively reconstruct your past beliefs to match the known outcome. Psychologists Neal Roese and Kathleen Vohs reviewed decades of research on this phenomenon and found it to be one of the most robust and replicable biases in cognitive science. Once you know that a change worked, your memory rewrites itself: of course that was going to work, the reasoning was obvious. The problem is that it wasn't obvious at the time, and without a written record of what you actually thought, you lose the ability to calibrate your judgment.

Outcome bias — studied extensively by Jonathan Baron and John Hershey at the University of Pennsylvania — is the tendency to evaluate the quality of a decision based on its outcome rather than the quality of reasoning at the time the decision was made. A change that worked for bad reasons gets filed as a "good decision." A change that failed despite excellent reasoning gets filed as a "bad decision." Over time, this corrupts your entire model of what good optimization looks like.

Shane Parrish at Farnam Street popularized the decision journal as a countermeasure to both biases. The practice is simple: before making a decision, write down what you're deciding, what you expect to happen, and why. Record your mental and emotional state. Then, months later, review the entry and compare your expectations against outcomes. Parrish found that the gap between what people thought they were reasoning and what they were actually reasoning was often enormous — but only visible in retrospect, and only when a written record existed.

The optimization log applies this same principle to every change you make in a system you're improving. Without it, you're relying on memory — and memory is exactly the tool that hindsight bias and outcome bias have corrupted.

What an optimization log actually contains

The minimum viable optimization log has four fields:

| Field | What it captures | Why it matters | | ------------- | ---------------------------------- | ------------------------------------------------------------------------------ | | Date | When the change was made | Enables temporal analysis, spacing effects, seasonal patterns | | Change | What specifically was modified | Eliminates ambiguity about what "the thing I tried" actually was | | Rationale | Why you expected this to help | Captures your reasoning before the outcome distorts your memory of it | | Result | What you observed after the change | The empirical anchor — what actually happened, not what you hoped would happen |

The rationale field is the one most people skip, and it's the most valuable. Recording why you expected a change to help does two things. First, it forces you to articulate your model of the system — why you think this lever connects to that outcome. Weak rationales ("I have a feeling this will help") become visible. Second, when the result doesn't match the rationale, you've generated a precise learning opportunity: your model was wrong in a specific, documented way, which tells you exactly what to update.

More sophisticated optimization logs add additional fields:

• Baseline measurement (from L-0577's benchmarking practice)
• Post-change measurement (the quantified result)
• Variables held constant (what else might have changed)
• Confidence level (how sure were you this would work, rated 1-5)
• Follow-up needed (what would confirm or disconfirm this result over time)

But start with the four. You can always add columns. You can never retroactively add the rationale you didn't write down.

Three domains that treat logging as infrastructure

Scientific lab notebooks

The practice of detailed experiment logging in science isn't tradition — it's infrastructure. The National Institutes of Health (NIH) prescribes specific lab notebook standards: record all environmental conditions, test parameters, and apparatus details. Note data directly into notebooks without relying on scraps of paper or memory. Identify errors by crossing out mistakes without obscuring the initial data — because the wrong answer you wrote first is itself data about your thinking.

The cardinal rule: a notebook entry should allow someone else to reproduce the experiment without additional explanation. This forces a level of specificity that personal optimization almost never achieves. When you write "changed my morning routine" in your head, you know what you mean. When you write it for reproducibility, you're forced to specify: "Moved workout from 7:00 to 6:15 AM, before coffee instead of after. Hypothesis: exercise before caffeine improves sustained focus in the first work block."

Machine learning experiment tracking

The machine learning community has industrialized optimization logging through tools like MLflow, which structures every optimization run as a formal record: parameters changed, metrics observed, artifacts produced, code version used. MLflow's experiment tracking system stores not just the outcome of each run but the complete configuration that produced it — so any result can be reproduced, compared, or extended.

The insight from ML experiment tracking that transfers directly to personal optimization: logging is not overhead on top of the work — logging is the infrastructure that makes the work accumulate. Without tracked experiments, a data scientist running a hundred model variants has a hundred disconnected results. With tracking, they have a searchable, sortable, filterable landscape of what works and what doesn't. Each new experiment is informed by every previous one. This is the difference between iteration and accumulation.

Software changelogs and version control

Every commit message in a version control system is a micro-optimization log: what changed, when, and — in well-maintained repositories — why. The git blame command lets any developer trace a single line of code back through its entire modification history, seeing not just what changed but who changed it and what commit message explained the reasoning.

The best practice in software changelogs is to make small, frequent commits rather than infrequent monolithic ones. The reason is diagnostic precision: when something breaks, smaller commits let you pinpoint exactly when and where the problem was introduced. The same principle applies to personal optimization logs. If you logged five changes in one entry — "overhauled my entire morning" — you can't diagnose which change helped. If each change gets its own entry, you can trace effects to causes.

The retrospective: turning logs into learning

Agile software teams conduct retrospectives at the end of every sprint — a structured review asking: What went well? What didn't? What will we change? Esther Derby and Diana Larsen, who literally wrote the book on agile retrospectives, argue that the retrospective's value comes not from the discussion itself but from the documentation it produces. Without a written record, teams rediscover the same problems and reinvent the same solutions in a cycle that feels like progress but isn't.

Your optimization log becomes useful the same way: through periodic review. The log captures raw data. The retrospective extracts patterns. Schedule a weekly or monthly review where you read back through your log entries and ask:

1. Which changes had the largest positive effect? (Do more of this.)
2. Which changes had no effect? (Stop wasting effort here.)
3. Which rationales were wrong? (Your model of the system needs updating.)
4. What patterns emerge across multiple entries? (These are your actual leverage points.)
5. What haven't you tried yet that the log suggests? (Your next experiment.)

The retrospective is where optimization transforms from "trying things" into "building a model." Each review cycle strengthens your understanding of why changes work or don't — which is the real output of optimization, not any individual improvement.

The log as protection against narrative

Here is the subtlest function of the optimization log: it protects you from the stories you tell yourself.

Without documentation, your optimization history becomes a narrative — a story with a protagonist (you), a challenge (the system to improve), and a satisfying resolution (things got better). Narratives are compelling but lossy. They compress away the dead ends, the contradictions, the changes that worked for reasons you still don't understand. They smooth the timeline into a clean arc that feels true but isn't accurate.

The log resists narrative. It stores the mess. The entry from Tuesday that contradicts the entry from Thursday. The change you were confident about that did nothing. The change you tried on a whim that worked better than anything you'd planned. This mess is not a bug — it's your actual epistemological position, and it's far more useful than a tidy story.

When you eventually need to help someone else optimize a similar system — a colleague, a team member, a friend — the narrative is useless to them. It's your story, compressed for your sense-making. The log is transferable. They can read your actual experiments, your actual reasoning, your actual results, and extract what applies to their context. They can learn from your dead ends without having to replicate them.

What this makes possible for L-0579

The next lesson addresses premature optimization — the trap of optimizing systems that haven't yet been proven valuable. An optimization log is your primary defense against this trap, because it makes the cost of optimization visible. When you can see twelve logged entries representing twelve hours of effort for a 3% improvement in a system you use once a week, the cost-benefit calculation becomes obvious in a way it never is in your head.

Without a log, optimization feels free. It's "just thinking about how to make things better." With a log, optimization has a documented cost in time, attention, and cognitive effort — which means you can make rational decisions about when to stop.

That's the bridge. You can't recognize premature optimization if you can't see what optimization is actually costing you. And you can't see the cost without a record.

The protocol

1. Create the log. Four columns: Date, Change, Rationale, Result. Physical notebook, spreadsheet, text file — the medium doesn't matter. The habit does.
2. Log before you observe. Write the change and rationale before you know the result. This is the pre-commitment that defeats hindsight bias.
3. Log failures with the same rigor as successes. The entry that says "changed X, expected Y, got nothing" is more valuable than the entry that says "changed X, it worked." The failure constrains your search space. The success might be luck.
4. Review weekly. Read the log. Look for patterns. Update your model of the system. Identify the next experiment.
5. Keep it honest. The log is not a performance record. Nobody else needs to see it. Its value is entirely in its accuracy — and accuracy requires recording the attempts that went nowhere alongside the ones that succeeded.

Optimization without a log is tinkering. Optimization with a log is science. The difference is not rigor — it's memory. And memory, as we've established, lies.