Marbles - You are almost always dead on
However, I decided to try the ground lift, because I treid other sources and still had buzz. So before you start to rip apart you PC, please get one of these...
and don't connect the little metal tab. See if the noise goes away. That's what happened to me.
What's really strange is that I hooked up a portable, battery powered CD player and still got buzz. Much less, but it was still there. As soon as I lifted the ground, it was dead silent.
So I have a wiring issue. You might too.