Does the referee take the balls off the game field after the robot has knocked them off the bases?

No, the balls – like all game field objects – remain on the game field until the end of the round. They may not be touched by hand. If the robot moves them later, that’s okay.