In order to optimize the orthogonal design of MIMO radar waveform, an improved algorithm based on niche genetic algorithm is proposed. The algorithm is able to eliminate individuals with similar structures in the population or data, maintain a certain distance between individuals, and effectively hold the diversity of them It thus eliminates the phenomenon of premature convergence, strengthens the global search ability of the algorithm, and improves its convergence speed. It is shown that the algorithm has both the advantages of overall search ability of genetic algorithm and the ability of the niche genetic algorithm to discover multiple optimal solutions. Simulation was given to show that the algorithm was cable to reduce the side-lobe of the auto-correlation function of the orthogonal polyphase code and the peak value of the cross-correlation function of the code with good performance of autocorrelation and cross-correlation obtained