With the development of intelligent phones and speech recognition technology, there is a great demand for generating meeting minutes automatically. In this paper, we design and implement Yanji, an automated system for generating meeting minutes based on speech and speaker recognition. The Yanji system realizes the following functions: recording the audio of the conference, uploading audios to IBM cloud in real time, transcribing audios to texts and identifying various speakers with IBM Speech to Text API, and finally generating complete meeting minutes. Yanji greatly reduces the recording storage space and the labor cost of listening and writing, and improves the meeting efficiency.