1. Yes, if you plan to replace the ENTIRE harness, it MIGHT work.
2. You likely will have wiring, sensor and all kinds of other headaches. Have you been doing swaps for years and can do it in your sleep?Although everything should bolt together, the wiring is way different, both ON THE ENGINE, ON THE TRANSMISSION and between the ECU, TCU (trans computer), and DASH harness.
3. Likely won't pass emissions testing.
4. Likely will not run correctly.
5. It's just a bad idea.
6. Main issue is you're 'crossing phases'. '96 is a Phase 1 Subaru electronics; '04 is Phase 2.