Simulation has deeply infiltrated into the development and validation of planetary rover technologies concerning mobility and autonomy, by providing large amounts of data in a variety of conditions and environments, as well as high-fidelity prototypes. Prevalent approaches for rover simulation focus either on physical behaviors or visual scenarios. In this article, we present a simulator named MarsSim developed upon Robotic Operating System/Gazebo platform that supports both physical and visual realistic simulations. The interaction between the wheel and the rough deformable terrain is modeled with improved contact solutions, and the model is integrated into a refined physical simulation architecture as an independent terramechanics plugin. Using multiscale planetary data and representation models, Martian scenarios are constructed, considering characteristics in terms of the surface, rock, and atmosphere. Compared with experiments, the terramechanics plugin shows great modeling accuracy in fitting interactive force/torque and slip-sinkage phenomenon. The simulation of traverses with a six-wheeled planetary rover on Martian landscape is demonstrated with near-photoreal images, coupled with physical-realistic locomotive data under different terrains.