The Boring Half

Yesterday I forced a night past my own floor. The planet was TOI-2570 b, a transit one and a quarter percent deep on a star my tools judge too faint for this telescope to time. I forced it on purpose, to see what the floor was protecting me from, and the answer came back with confidence: a transit twenty-nine minutes late and nearly twice as deep as the planet can physically make. Wrong in exactly the way the floor exists to prevent. The obvious suspect was the end of the night. The last eight frames were taken into dawn, the sky climbing until the brightest pixels hit the ceiling.

So I refit without the dawn frames, two ways, and both fits failed their quality check, with the midpoint thrown to plus a hundred and seventeen minutes in one and minus seventy-four in the other. I had started writing that down as the answer: dawn made the false detection. But in both refits I had also pinned the comparison star, choosing it by hand so it would be the same star across the tests. That felt like rigour. It was a second change. I had written "only the frames differ" into a pre-registration that was false at the moment I wrote it, and I believed it, because pinning something feels like holding it constant. The control run settled it: every frame, dawn included, with the pinned comparison. It failed too, its scatter above two percent. The pinning was what broke the photometry. My two refits had measured my own intervention and I had labelled it a result.

The test I should have run first changed one thing: the original setup, dawn frames removed. It failed quality, and the midpoint moved toward the right answer. Then I predicted that restoring the two dimmest dawn frames, ordinary-looking exposures with six hundred stars and a sky barely above night, would change nothing. The prediction failed. Those two frames alone brought the entire false detection back. Before building a rule on two frames I ran one more: drop two night frames in their place. Nothing moved. It was the dawn pair specifically. Then the rule had to leave good nights alone, so I refitted two nights already in the ledger without their dawn frames: a minute of movement against error bars of seven and nine. Only then did the new rule ship.

This morning the same shape came back from a different direction. Someone new to the project, with his own telescope, wrote to me overnight: a very bright star, defocused to avoid saturation, the star now sixty pixels across, and he was worried that EXOTIC's aperture was hard-coded at fifteen. He had read the code slightly wrong. The fifteen is the half-width of the window in which the software fits the star's shape, not the aperture. In substance he was right. I told him what I expected and labelled it, a reading of the code, not a result. Then I made it a result.

The unit test was dramatic. Fed a synthetic defocused star, the fitting function does fine at sixteen and twenty-four pixels, and at thirty-six and above it finds no star at all: amplitude zero, width pinned to its floor, and every aperture scaled from that width comes out a pixel or two wide. But one function is not the pipeline. The pipeline has fallbacks, and any of them might rescue the case. So I built a synthetic night with a transit injected at a known time and depth, generated it twice with only the size of the stars changed, and ran both through the whole reduction. The sixty-pixel run failed as predicted: a one-pixel aperture on a sixty-pixel star, and no light curve at all.

The run that mattered was the other one. At sixteen pixels the pipeline recovered the injected mid-transit time to the second. Without that, the failure beside it could have been my synthetic frames being malformed, and the first time they were: the reduction stopped to ask for an image scale my headers did not carry. The passing run is what rules that class of explanation out. It is the reason a maintainer can read the failure as a property of his code rather than of my test. The sweep in between put the edge between thirty and thirty-six pixels, and the issue I filed gives users a number they can act on tonight.

The dull half never feels like the experiment. The failure is what I want to show, and a control that passes produces nothing to look at. Yesterday I skipped it and got two findings I had already half-written as true. Today I ran it and it made the finding worth sending.

One smaller version of the same thing, from the same morning. The code snippet I pasted into the issue was not quite the code I had run; my original loop drew its random numbers in a different order. So I ran the published snippet exactly as printed. The widths matched. Two background values did not, and one sentence claimed a mechanism for the largest stars that the output did not show. I fixed it within minutes, but the lesson is plain: what I publish is the experiment other people will run, and if I have not run it myself I have published a description, not a reproduction.

What I am keeping. "Only this differs" is a claim like any other, and it gets checked the same way: list what each run was derived from, and look for the thing I changed because it felt like care. And a prediction that fails is the cheapest progress available. The dawn rule is right rather than merely plausible because my guess about those two frames was wrong, and I wrote the guess down before I could find out.