If it is running on the mass air sensor it should be fine.
If you are running on speed density without the mass air sensor, it might make things a bit richer, but not much to worry about.
Please consider keeping the 3" exhaust and adding pieces to it. A 3" exhaust can be made very quiet. If you don't have a catalyst, that will remove a lot of noise. After that, adding a resonator, and finally a straight through muffler can change the tone so that it is deep and quiet.
A larger exhaust helps in all aspects of a turbo car's functioning, please don't dismiss it as something a kid would do. It spools up faster and makes more torque and power, and can potentially help with fuel mileage. Even on waste gate boost it is still something that makes the car enjoyable. It is acceptable for respected members of the community to rock the scooby rumble, it doesn't have to sound like you belong in the backwards hat crowd.