This is the response I got from Subaru.
I'm disappointed but not really surprised. .
Thank you for contacting Subaru of America about the hands free system in your 2011 Impreza Outback Sport.
On page 5-48 of your owner's manual we acknowledge that your voice may be difficult to be heard when driving on a rough road, at a high speed or with a window open. Some tips that may help the sound quality would be to adjust the ventilator so the air does not blow toward the microphone and also make sure the cell phone is not too close to the microphone.
Again, we do appreciate that you took the time to bring this concern to our attention. Your comments will be noted in our Customer/Dealer Services database. We use such information to continually track and review the performance of our products, always with the intentions of improving upon them. We will also forward your comments to our Product Planning Department for their review and future consideration. In the past, input from our owners has been helpful in initiating changes to our newer models.
Customer Dealer Services
Subaru of America, Inc.
I wouldn't mind paying $50 (USD) or so for a fix to the problem.