I have spent the better part of the last three weekends trying to isolate the bug that was making the program hang for long periods at a time.
The symptom was simple: the program would stop and display the last timer/settings on the LCD. It would occur between 3 and 7 hours of operation. The start/stop/silence buttons were inoperable while the program was hanging.
My initial thoughts were that the program had completely stopped and was hanging somewhere. Trying different techniques and waiting 3-7 hours for a tell was utterly annoying.
An long story cut short: I was using the LCD_RW pin in an effort to reduce the time taken by screen updates (instead of connecting it to GND). The result was a program that appeared non-responsive and intermittent as hell to debug.
I tied LCD_RW to GND and removed about 30+ debugging lines of code and the program has been stable for two complete 7 hour intervals.