Rebuilt the front brakes and did some other stuff up front the other weekend.
After putting the tires on I noticed there's a little bit of play. Watching from the front it doesn't seem to be the ball joint. The wheel bearing hub inside the knuckle seems to have the movement.

Seemed like I could get a quarter inch or so of up and down movement at the tire and visibly see the CV-shaft move up and down as I did so.

Didn't seem "loose", just that it had this movement. Do I have a set of unit bearing hubs to replace or am I chasing shadows?

I honestly can't remember how much or if this play existed in my previous Trackers/Grand Vitara/XL7.

I had that.  It will cause your calipers to bind a little and ended up killing my inside brake pads waaaay prematurely in addition to some pulling.
 
I believe you can replace the bearings in the hub if you have a press, but I just replaced the entire hub.  The lockring tool is a pricey little bastard, btw.

http://www.costplustools.com/Kent-Moore-J-37763-WHEEL-BEARING-NUT-WRENCH_p_9505.html

I didn't have pulling and my pads were pretty even.
One slide pin on each side was firmly rusted into the caliper bracket though. That and slight warping of the rotors are why I did new pads, rotors, and as I was going, new caliper brackets.

Anybody else have any thoughts on this?

AJM, pull a front wheel, the hub and check the actual torque on the hub retaining nuts.  159# Lb Ft is the torque standard on them.  If they have been tightened with punches or anything other than the spanner socket on a torque wrench there is no way they can be torqued properly and with bearing wear movement will become more pronounced.