Buttons that are permanently selected, such as popup toggle buttons and the active layer in the layers bar, display a gray background. Just normally clicking on a button, though, it displays the bg image fine...

This clashes somewhat with my sense of aesthetics... It's not a real problem, just an annoyance...

That depends on your Windows theme. Just change your Windows theme or adjust it slightly and you don't have to see any gray. If you notice in these screenshots, the Popup Toggle buttons are closer to a white background.