Aiming at the high requirements of marine diesel engine safety operation, based on depth analysis of the requirement of vibration monitoring and fault diagnosis for marine diesel engine, a design scheme of vibration monitoring and fault diagnosis system for marine diesel engine is proposed from three aspects, including system architecture, hardware design and software design. According to the design scheme, a principle prototype is developed and verified through experiments. The experimental result shows that fault diagnosis and early warning can be realized by the principle prototype and disassembling the diesel engine is not required. The principle prototype is simply operation and highly reliability, which means the design scheme of vibration monitoring and fault diagnosis system is feasibility.