Intrusion detection systems (IDS) based on machine learning (ML) can be used to detect anomalies in data traffic. Common challenges for IDSs are low detection rates, high false alarm rates, and the need to process large amount of data. In order to overcome these challenges various types of supervised, semi-supervised and unsupervised ML methods are being widely researched. However, the need for high-performance processing capabilities in order to perform the calculations restricts the use in industrial control systems (ICS) with usually small embedded processing elements. In this paper, a comparison of key ML methods is carried out that focuses on data traffic and processing requirements for the application to ICS. Furthermore, a variant of growing hierarchical self-organizing maps (GHSOM) is described that proposes a new method to detect anomalies based on the GHSOM behaviour. The influence of the GHSOM parameters on the detection ratio of different kinds of attacks on a production line for the proposed method is examined. To detect attacks on the ICS, growth criteria of the GHSOM have been applied as alarm generating conditions. The comparison of the performance of GHSOM, Feedforward Artificial Neural Network (ANN), Support Vector Machine (SVM) and Local Outlier Factor (LOF) took place based on receiver operating characteristics of the classification between normal and anomalous data captured from a real industrial line testbed. The proposed method has shown comparable classification results with one-class SVM and LOF as novelty detection method in the studied case scenario.