Thanks for the reply!
I do not have a processor. I am using the crappy stock HU. I was thinking it was the HU but wondered why the HP/LP filter on my amp wasnt fixing/helping.
Also I was doing the "push in on the sides" test to see if the sub was blow. And Im pretty sure it is. Some sides were good (no noise) and others had a slight rubbing noise so that is probably another issue.