My experimental results came in late Friday afternoon, and. . .well, rarely have I seen less encouraging data. It wasn’t enjoyable. I was there as the numbers for each part of the experiment came through, and I could tell early on that I was in trouble.
So, here’s the rundown: the repeat of my previous (putatively successful) experiment failed. Then the attempts to increase the effect failed at each point – if anything, things went down instead of up. Then the attempt to reverse the effect (in four different ways) failed, no doubt because by this time there really wasn’t anything to reverse. The best-looking run of the whole afternoon was from this group – but it was, perversely enough, the one that should have been the most shut down. Ungood.
What now? The only way this experiment can be any kind of good news is if some systematic error disabled the whole thing. I would love to find out that one of the components was taken from the wrong vial, or was left on top of a hot plate or something. But that’s highly unlikely. But what about that previous experiment, the one that led to this death-or-glory attempt? I’m going to be going back over that one, giving it a fishy glance in light of what happened on Friday. At this point, the hypothesis that best fits the data is that the encouraging results are wrong.
I may still be able to do one more attempt. There are a couple of oddities about this latest data set that I don’t understand, and the idea is worth one more shot under the cleanest conditions I can think of. But that, for now, will have to be it. I’m going to have to go back to the drawing board and think about what’s going wrong, see if there’s some different way to realize what I still think of as a beautiful idea. There may yet be. Plenty of beautiful ideas don’t work, though: beauty is necessary, but not sufficient.
Science is fun, it really is. And it’s certainly damned useful. But it isn’t easy.