I have to agree with Rich (MPREZYA) on this one. If you've already got the 4EAT, you'd be money and heartache saved to beef it up, and perhaps install a manual shift valve body instead of going to a manual transmission.
There are a number of vendors who offer beefed up 4EAT internals. Rancho Transmission in Temecula, CA and Level 10, somewhere in Pennsylvania (I believe) for sure offer beefed up 4EAT's.
Trust me, I'm strongly torn between converting my 5 speed 98RS to a 4EAT or going with dog gears. This coming Wednesday will move me one big step closer to having to make a decision as the built shortblock for my forthcoming turbo system is getting put in the car.
If Subaru offered a 5EAT, it'd be no contest, I'd go with a manual shift slushbox and be done with the worries.