So I am having this exact same problem. A/C button light turns on, a relay clicks under the dash, and the compressor doesn't engage. Turn on the defrost and the compressor engages and it pumps cold air. But nothing any other time.
BTW, this is on a 2007 Impreza 2.5i, just like the OP.