Everything I've read says the bigger the cable, the better. If 4-gauge works, well, that's good, but I myself would go for overkill. All the guys in the Mustang world that do the trunk-mounted battery use 0/1-gauge welding cables - apparently the welding cable is more flexible than "normal" 0/1-gauge cable, and it's readily available at your local welding supply store.
Here's some advice on proper grounding I've had saved for a while. This is also from the Mustang world, but obviously the principles apply regardless of what type of vehicle it is.
"If you purchase one of the typical relocation kits out of the popular parts catalogues, what you receive is a length of 4 or 2 gauge cable, (in my case a couple of years ago, I received 4-gauge cable) and a marine battery box with a hold-down kit.
The instructions basically have you put the battery in the trunk and ground it back there to the frame or chassis.
This doesn't work for a variety of reasons. First, 4-gauge cable is simply too small to handle the resistance necessary for the power to go all the way from the trunk to the starter/solenoid. It is also too small for proper grounding.
Now, I am no expert in DC power. In fact, it is quite confusing to me. But my father is a Master Electrician so I called on his help. He explained to me that in a DC system, the grounding is actually the most important link in the power chain. Therefore, it is the grounding system that has to be the most "powerful" for lack of a better term, and have the most integrity. He drew up the plans and gave me this:
MINIMUM 2-gauge, preferably 1/0 gauge cable, unbroken, running from the factory ground location (on the timing chain cover) all the way to the negative post on the battery. Another length of the same gauge cable going from the same factory grounding location to the inner fender well or frame in the front (mine is attached to the front K-member, if I recall correctly), a ground strap from the back of the block to the firewall (this is already there from the factory but it is always a good idea to check it's condition and replace it with a heavier one if it doesn't look in perfect condition. Crap Boyz has these straps right on the shelf in different gauges). Then run a cable of the same gauge from the nagative post on the battery to a nearby ground on the chassis in the trunk. Also very important is that the auxiliary ground cable (I think it is white or black from the factory) in the engine bay needs to be grounded up there as well. With mine, I put it together with the additional strap that went from the timing chain to a nearby frame/chassis location. Then, run a minimum 2, preferably 1/0 gauge cable from the positive of the battery up to the starter/solenoid. Both the positive and negative cables can be the same gauge, or the negative can be larger, but the positive should not be larger than the negative.
This grounding system cures any problems with charging the battery. It also makes a solid power/ground connection necessary for the electronics to work properly.
According to Pro-Flow Technologies (the company that makes the Pro-M MAF meters) many people relocate their batteries and have insufficient ground in general, or do not properly ground the auxiliary wire and have all kinds of driveability problems that they can't figure out. Or they splice cables together and so on which makes for noise in the lines and screws up the electronic readings."
That may be overkill, but you'll never have any problems, regardless of what you throw at your electrical system. The guy who wrote the above passage had a BIG stereo system in the car and never had a single problem.
'97 Legacy 2.5GT sedan