• The JH-HLM has excellent reliability when used by physical therapists in the ICU. • The JH-HLM has excellent reliability across several types of ICUs. • Trained physical therapists can use the JH-HLM to reliably measure observed patient mobility in the ICU. The Johns Hopkins Highest Level of Mobility (JH-HLM) scale is used to document the observed mobility of hospitalized patients, including those patients in the intensive care unit (ICU) setting. To evaluate the inter-rater reliability of the JH-HLM, completed by physical therapists, across medical, surgical, and neurological adult ICUs at a single large academic hospital. The JH-HLM is an ordinal scale for documenting a patient's highest observed level of activity, ranging from lying in bed (score = 1) to ambulating > 250 feet (score = 8). Eighty-one rehabilitation sessions were conducted by eight physical therapists, with 1 of 2 reference physical therapist rater simultaneously observing the session and independently scoring the JH-HLM. The intraclass correlation coefficient was used to determine the inter-rater reliability. A total of 77 (95%) of 81 assessments had perfect agreement. The overall intraclass correlation coefficient for inter-rater reliability was 0.98 (95% confidence interval: 0.96, 0.99), with similar scores in the medical, surgical, and neurological ICUs. A Bland–Altman plot revealed a mean difference in JH-HLM scoring of 0 (limits of agreement: −0.54 to 0.61). The JH-HLM has excellent inter-rater reliability as part of routine physical therapy practice, across different types of adult ICUs. [ABSTRACT FROM AUTHOR]