I've dealt with this issue for many years in my 02 WRX. I used to run the lightweight clutch and flywheel setups and it made the decel noise even worse. However even with OEM parts, the sound was reduced but still noticeable.
Recently had my center differential and transfer case unit replaced and the noise is completely GONE. Running OEM clutch and flywheel.