Background Chemical hazard communication (e.g., label) is intended to alert users of the potential hazards of chemicals. Apart from the fact that hazard information needs to be understood, it is also important that it is recalled. Recall of hazard communication is critical when the written form of the information is not available at the time it is required. Methods A cross-sectional study investigating associations between the recall of chemical safety information on labels and safety data sheets among 402 participants including 315 workers (industry, transport and agriculture sectors) and 87 consumers in two provinces of South Africa was conducted. The recall of label information by participants was tested using two modules from a Hazard Communication Comprehensibility Testing (CT) Tool developed by the Centre for Environmental and Occupational Health Research at the University of Cape Town for the UN Institute for Training and Research (UNITAR). Results Respondents were predominantly male (67.7%), the median age was 37 years (IQR: 30–46 years) and less than half of the participants completed high school (47.5%). The skull and crossbones symbol was the label element most recalled, both freely without prompting (79.6%) and recognition after prompting (94.8%). Whereas, the first aid and treatment measures were the least frequently freely recalled (least frequent item 6.0%; most frequent item 29.9%). Multivariate analysis identified the following positive associations with the recall of all the label elements listing the strongest association: call appropriate services and industrial vs. consumer sector (OR = 2.4; 95% CI: 1.2; 4.6); call appropriate services and transport vs. consumer sector (OR = 4.4; 95% CI: 1.2; 16.0); flammable symbol and male vs. female gender (OR = 2.3; 95% CI: 1.0; 5.3); flammable symbol and home language English vs. African languages (OR = 6.6; 95% CI: 2.1; 21.2); any hazard statement and home language Afrikaans vs. African languages (OR = 14.0; 95% CI: 3.6; 54.2), any first aid statement and further education vs. none (OR = 3.3; 95% CI: 1.3; 8.0), correct chemical name and industry blue collar workers vs. non-industry blue collar workers (OR = 2.6; 95% CI: 1.1; 6.1), correct chemical name and non-industry white collar occupations vs. non-industry blue collar workers (OR = 2.7; 95% CI: 1.0; 7.1). Conclusion The study found a number of potential positive associations that influence recall of label elements of which some (e.g., sector, gender, occupation) suggest further research. Relevant policies in South Africa should ensure that the safety information on chemical labels is clearly visible to read and understandable which aids recall and the reduction in harmful chemical exposures.