Are you using EZCam bolts for alignment?
If so, my passenger side goes out after tough terrain as well. I think I need to replace them as I have had them on and off about 100 times doing mods and fits. May have stripped the ecentric a little or something. It definately rolls around after hard wheeling though.