This "the only time a swap makes sense" bs is just that... bull****. Who cares? It'll never make financial sense but modding a car doesn't make financial sense either. In fact, owning a car doesn't make financial sense, its a depreciating asset the moment you own it. You will never get out of a car what you put into it.
I did my swap because I wanted to. I wanted to learn about the car and learn some more mechanical skills. Two years previous, I had trouble figuring out how to put a CBE on my car.
Now, I've got a fairly unique motor (V7 EJ207) that is stronger than any Subaru motor ever released for USDM consumption and I can enjoy the fact that, while there are many cars that are faster or worth more, this car is more mine than most.
While there are times that its a complete pain in the ass and it might have been simpler to just sell my 02 RS and buy an STI, I can still go into the garage, look at my car, and it makes me smile.