I would also say buy it, at least it will be in the community.
I've owned my turbo wagon since 2003 and it has been one of the most reliable cars I've owned. The bodies and interiors are all 99.99% the same as non-turbos, the differences start to show up when you get to the engine (and the trans, but its not really that different from any other 3AT).
Obviously any part of the intake and exhaust system will unique to the EA81T. The exhaust can be fabricated by anyone, but the intake hose from the filter housing to the turbo is NLA and they are getting old and brittle, so check that hose. The cylinder heads are different as well, since they have castings for the fuel injectors, and they will all crack between the valves eventually (This is not the death sentence it is in an EA82T, my engine has cracks on at least two cylinders and doesn't use coolant or overheat. The blocks can interchange with the later EA81s, but it must have a threaded boss on the case for the knock sensor. The fuel system is different, from the tank to the cylinder heads, as is the ignition system and emissions control parts.
Most of these parts can be replicated, repaired or recreated using ingenuity. The parts that are really going to hang you out to dry are the ignition control module, ECU, cylinder heads and a few other electrical engine control related bits. These parts do not exist (give or take) in the aftermarket and will stop you dead if they fail.
As for the manual transmission conversion, don't let GD scare you, it's actually a really simple conversion. I didn't hammer my trans tunnel, but I did grind down an unused boss on the transmission case that was tapping the trans tunnel. New transmission mounts help keep the trans from flopping around too, but both EA-series chassis have garbage transmission mounts. The conversion would allow an opportunity to improve that. Most of the conversion is just unbolting the AT parts and replacing them with the appropriately modified MT part. The wiring is simple as pie and you can keep cruise control if you do it right, again, really simple. THis could be done in a long weekend if you had all the parts on hand and no bolts are rusted/broken during R&R. I only drove the car for about 40 miles with the AT (no 3rd gear when I bought it) before I swapped mine. I went with the EA82 transmission, flywheel and clutch, but that made the swap a little more complicated. I needed sleeve-shims for the flywheel bolts and customized shifters and transmission crossmember to adapt the later trans. I wouldn't keep the AT in the long run. Drive it until the 3AT dies, then swap it, these cars deserve a manual.
As far as the turbo setup being completely useless, I'm going to call bullshit. Stock vs. stock, the EA81T makes 13% more hp and 36% more torque than a big-valve USDM EA81, and that torque peaks at a lower rpm as well. The area under the curve on the EA81T puts the n/a version to shame. All it takes is one WOT pull through third gear (with a manual trans) and you'll know who's king of the EA81s.
Tuning can be done to the engine because it's such an old design, it's fairly simple to trick the system into making more power. True, there are no software upgrades available outside of stand-alone engine management, but you can turn up the wick safely, just make sure you've got enough fuel to avoid a lean condition and that your cooling system, temperature gauge and cooling fans are all working. Turbocharged engines love exhaust work and swapping the mechanical fan for an electric one will free a few more ponies without getting into the range of making "unreliable power." I've done just exhaust work on my car so far, but have had plans for more power for years. Then I got a WRX and left the wagon alone, but I may dip back into hopping up the EA81T it soon.
Let us know if you have any other questions.