I had almost the exact same problem as the OP. I have a 2005 Legacy GT, and about 9 months ago I started seeing my battery go dead after a few days of not driving the car. I replaced the battery, which was a few years old, but then the new optima battery started to have the same behavior.
Around the same time, my factory CD changer got jammed and I just figured I'd live with it until I got around to fixing it.
This week I got tired of the battery draining problem and found this thread - thank you all for the great suggestions!
Checking all of the fuses showed no major culprit - current drain with the car off stayed about .7 amp. I pulled the stereo out, and like magic it dropped to .02 amp (20 mA). Apparently, the stereo is not on one of the fuse circuits that I could see.
With the stereo/CD changer apart, I could see that a homemade CD with a sticker label had lifted up and gotten all snagged in the mechanism. I was able to slide the CDs back into their center position near the spindle. Then, I plugged the white harness connector back into the car, and held the eject button until "Eject All" appeared on the display. As it shuffled the CDs out, I helped it along by pulling them gently as they popped out.
After I put it all back together (minus the nemesis sticker CD), the CD changer worked again and current draw with the car off was back where it should be.