I kinda figured that the power loss when stepping to 31's on steelies would be enough to notice...just didn't know how bad.
Real bad. Unless you live in a flat area - stick with 29" tires. Your gas mileage will go to crap, acceleration will be non-existent, going uphill with camping gear will put you in 3rd or 4th gear on the highway, way added stress to the driveline, breaking distance will increase by about 50%, and your low crawl ratio won't be very low anymore.
29" tires on stock wheels is cheap ($550 ish). The money you save will pay for lockers ($500) and spacer lift ($300) - which will do you more good than the 31" tires. I'd also consider the OME lift ($750) if you can afford it (if you can afford 31" tires, you can afford the OME lift). I think the tracker is superior to the samurai in every way, but it's Achilies heel is gearing options.
If you want a serous wheeling monster & want to be able to drive on the street, get a Kicker 3 (search) - but that's expensive and a whole new ball of wax.
Are there any decent Aluminum Wheels out there that you guys are running? Seems like most have steelies...
The #1 failure on trails is tires (popping a bead, poking a hole in the tire, or bending a wheel on a rock / tree). If you bend a steel wheel on the edge, you can beat it back with a hammer / rock. With aluminum wheels - you're stuck.