In the article, we describe comparison of different architectures for machine translation of the Slovak language using the Fairseq toolkit. The article describes current trends and the most used architectures (e.g. convolutional neural network, Long short-term memory, recurrent neural network and Transformer) in connection with machine translation. We also focus on the issue of parallel corpus and its use in connection with machine translation. We have created a short overview of the issue, we described our experimental results, and suggested possible solutions to problems in machine translation field for Slovak language.