Virtual reality (VR) has been used in many medical training systems for surgical procedures. However, the current systems are limited due to inadequate interactions, restricted possibilities of patient data visualization, and collaboration. We propose a collaborative VR system for laparoscopic liver surgical planning and simulation. Medical image data is used for model visualization and manipulation. Additionally, laparoscopic surgical joysticks are used to provide an opportunity for a camera assistant to cooperate with an experienced surgeon in VR. Continuous clinical feedback led us to optimize the visualization, synchronization, and interactions of the system. Laparoscopic surgeons were positive about the systems' usefulness, usability, and system performance. Additionally, limitations and potential for further development are discussed.