It's a combination of weight distribution and having your drive motors at the front. I can't remember the exact explanation behind it but basically robots with front mounted drive wheels are much harder to control. Most robots have their wheels either centered or rear of centre for better control, and most of them won't go in a straight line in reverse, they'll pull to one side like yours does in forward.