I came across a pretty cool bug that threw me for a loop. In the spirit of Car Talk, I’m turning this into a puzzler.
We have developed some cool performance enhancements for Grace (a runtime system that enables safe multithreaded C/C++ programming). To make it easier to selectively turn them on and off, I decided to reorganize some of the code. Not a big deal – essentially splitting a few classes into some component classes and moving code around. Pretty routine stuff.
And then something odd happened.
After testing the code (debug build), and verifying that it worked fine, I re-ran the code in an optimized build, just to make sure that I hadn’t inadvertently introduced any performance regressions.
Instant segfault. For every benchmark.
In sum – crashes when optimized:
- Everything works fine in debug builds.
- Complete disaster in optimized builds.
Enter your guesses below.
Emery Blogger 
Most likely explanation: you had a pointer to some data, either static or global, that was accidentally writing beyond that data into something unimportant. By reorganizing the code, the position of that data occupied a different position in the program text such that writing past the end of it blew away some vital code or data structure. The debug version program is not affected because it had extra padding that the optimized version got rid of. What kind of extra padding? Maybe some extra information you or the compiler put in to aid debugging, or maybe the optimizer cleverly did something like replacing ‘printf’ with ‘puts’ and elided the format string which would have provided the padding.
Less likely: the optimizer, faced with refactored code, makes different inlining decisions exposing a bug in your code. For example, the unoptimized version allocates variables on the stack even though they are always kept in registers. The optimized version optimizes out the allocation, removing padding that would have otherwise prevented an out-of-bounds access to something in the stack frame e.g. the return address, saved frame pointer, or some other important pointer.
Even less likely: gremlins.
Less likely than gremlins: compiler bug.
In reverse order:
* Correct, it was not a compiler bug.
* No gremlins either. It was before midnight :).
* Optimizer: That’s almost exactly it. In fact, stack allocation versus register allocation was the culprit, but not because of out-of-bound accesses or anything like that. It’s actually something the compiler really should have warned me about, but didn’t.
* I am surprised you didn’t mention side-effecting asserts (my first guess — discarded when -DNDEBUG -O0 still worked fine).