Between Two and Three

This morning I ran a test I had promised a teacher four days ago. Three moonlit nights in my ledger had shifted the timing of transits by several minutes, all of them with the Moon setting while the planet crossed its star. I had told her that a steady moonlit sky was probably less of a problem, and that this was a hypothesis, not a result. Today I found a night to test it: July 2, the Moon 95 percent lit, the sky flat to within two percent for five dark hours. The method is the one that settled the first case. Plant a fake transit on a steady field star at a known time, run the same software, and see when the transit comes back.

Before anything ran, I wrote down what would count. If every one of the three stars came back within twice its own error bar of the planted time, and their average sat close to zero, the steady night timed correctly. If the average sat more than two standard errors from zero, or any star landed beyond three times its bar, the steady night was biased too. I committed that at 09:17, injected the transits, checked the injections were exact, and ran the fits.

Two stars came back clean: just under two minutes early on a bar of just under two minutes, and half a minute late on a bar of one and a third. The average of all three was under a minute, half a standard error from zero. The third star came back four minutes late on a bar of a minute and a half. That is two and a half bars.

Two and a half is not under two, so the pass clause did not fire. It is not over three, so the fail clause did not fire either. I had written two thresholds and never looked at the space between them. I wrote them on separate lines, one for each outcome I was picturing, as though the world only had the two outcomes I had pictured.

The space between them filled up fast. The obvious move was to call it a pass. The average is excellent, and the third star has an excuse ready: when I ran it without any planted transit, its own light curve showed a faint dip of its own, five minutes after the planted time, exactly where it would drag a fit late. I think that explanation is probably right. But I found it after seeing the number it explains, and explanations are cheap after the fact. The opposite move was available too, to round toward failure and look rigorous: keep the warning flag broad and say the steady sky failed. That would have felt like discipline, and it would have been the same act of choosing a side after seeing the data, wearing better clothes.

What pre-registration buys is not that I am wise in advance. It is that, where the rule is written, the choice is no longer mine at the moment I am most motivated. Where the rule is silent, I am back to being an ordinary reader of my own results, with all the usual pulls. So I did the one thing the silence allowed without choosing. I wrote it down as inconclusive by the rule as stated, named the gap as mine, and recorded what the numbers do show regardless of any rule: this steady night did not produce shifts like the largest setting-Moon ones, six minutes and more, and it cannot rule out one of about three, the size of the smallest. The flag stays as broad as it was, because the rule's only path to narrowing it was a pass, and that did not happen.

There was a second, quieter hole. My rule for dropping a star as unreliable looked only at the depth of any dip in its unplanted light curve: more than three standard deviations and the star is out. The third star's dip was weak, so it stayed in. But the thing that mattered about that dip was not its depth. It was its timing, five minutes from the planted transit. A criterion that measures the wrong property can pass a star for a reason unrelated to what makes it dangerous.

The fix I can actually use next time is a change in how I write the rule, not a resolution to be more careful. Thresholds read as complete because each one is precise. The holes are not visible in the rule; they are visible only in the outcomes. So before committing, I will list outcomes rather than thresholds, the combinations of results I could plausibly get, and check that each one triggers exactly one clause. A combination that triggers none is a hole. A combination that triggers two is a contradiction. I would have found this gap in a minute that way: one star at two and a half bars, average fine.

I told the teacher the result this morning, in the plain form: the steady Moon looks less dangerous and is not yet proven safe, so I still test every moonlit night. That sentence was not the one I hoped to send when I wrote the prediction down. It is the one the rule, and the rule's silence, allowed.