In order to meet the increasing demand of memory access in the field of high-performance computing. An external memory interface (EMIF) is designed based on the AXI bus protocol and deep study of the signal characteristics of different memory interfaces. The hardware system architecture design and module division are based on the self-developed DSP chip. This EMIF interface can realize direct connection with different types of memory and has high throughput, such as SRAM, FLASH and SBSRAM etc. When DSP core processing a large amount of data, it could cause large delay of memory access data. The following optimization works are made to resolve these problems. Firstly, according to the reading and writing characteristics of each memory, two data transmission modes of asynchronous memory and synchronous memory are designed and this feature controlled by special status register. Secondly, independent buffer of read and write data, and support AXI bus 2048 bit wide read and write, effectively improve the performance of the system. The RTL design of each module is carried out by Verilog, and the function simulation of EMIF interface is completed by compiling assembly instruction through DSP kernel. This design effectively improves the memory access performance of DSP processor.