The only problems I've had are getting tickets when I was clearly speeding well over the limit (in my WRX).
Otherwise, (in the WRX) I get waves from 50% of Subarus with hood scoops (nobody seems to notice me in the BRZ however).
Nobody tries to race me.
But I usually avoid eye contact with anyone in a different brand car that looks like an enthusiast (i.e. lowered, stickered, tinted, loud exhaust, HID's, booming subwoofers).
My WRX isn't super flashy though, so maybe if I tried harder in that area I would get more attention, haha.