Gas is cheaper in NJ, insurance may be more but since it's highly variable depending on many factors you need to get an individual quote from a broker in the area where you plan to move. You'll get a ton of responses of "I pay this much" but that number may not apply to you.
Inspection is similar as it's an OBDII scan + visual inspection in both states, so the check engine light must be functional (on with key and off with engine start), no stored codes and readiness monitors set. However the systems are somewhat different, in NYS it's an annual inspection at a private shop/dealer, in NJ it's every two years at either a state facility or private station.
How thorough the visual inspection is depends on the particular station and inspector.
You can check the NY and NJ DMV websites for more info on inspections.