Originally referred to the governor on old steam engines that had two ball that rotated across one another that would control speed depending on how fast they spun. Balls out meant going full speed or power.
This refers to the governor on a steam engine. Two heavy balls are attached to the engine so that as engine speed increases, the centrifigal force of the flywheel causes the balls to rise. As the balls top out, they govern (limit) the engine, thereby controlling maximum engine speed. "Balls out," then, refers to running the engine at maximum speed.
The engineer was running the train at a "balls out" pace while trying to make up time.
Old timey terminology: on steam engines, they had steel balls in there that resricted steam flow. When the conductor was ready to pick up speed, he'd yell "Balls out!", signaling the help to do just that, thereby letting them haul some ass.