Natural language processing or computational linguistics is a subfield of Artificial intelligence that is growing very fast. Unfortunately, there has not been much work done in terms of the Bengali language in this sector. There have been works done on spell checking, Text to Speech (TTS), and Optical Character Recognition (OCR) for Bengali. Since there has not been any significant work on Bengali grammar checking, we have built a system for checking the grammar of Bengali text. To check the grammar of Bengali sentences, we have written a few grammar rules considering the fundamental dependencies of different parts of speech on each other. We ran our grammar program on the sentences split into words and tagged using pre-trained parts of speech (POS) tagger. This paper describes the grammatical rule checking and POS tagging to identify grammatical errors and pointing out misplaced words in Bengali sentences.