The RC arena floor is pretty damn flat, so much so that ground-scraping wedges or flipper blades very rarely get stuck on the joins. 2mm-3mm should be fine (particularly if your robot is 4WD, but 2WD will still cope comfortably).

The blue wheels are pretty tough too. They will wear down over time (and by how much depends on how much pushing your robot does and how often it runs) but you could be looking at as much as a couple of years use, if not more, before they get so worn down that your ground clearance becomes an issue.