Abstract Background The clinical evaluation of the upper limb of severely impaired stroke patient is challenging. Sensor-based assessments may allow for an objective evaluation of this patient population. This study investigated the validity of a device-assisted approach in comparison to the clinical outcome that it is supposed to reflect. Methods In nineteen severely impaired chronic stroke patients, we applied a gravity-compensating, multi-joint arm exoskeleton (Armeo Spring) and compared this sensor-based assessment with the clinical outcome measure Upper Extremity Fugl-Meyer Assessment (UE-FMA) scale. Specifically, we assessed separately and subsequently the range of motion in joint space for four single joints (i.e., wrist, elbow and shoulder flexion/extension (FE), and shoulder internal/external rotation (IER)), and the closing and opening of the hand with a pressure sensor placed in the handle. Results Within the kinematic parameters, a strong correlation was observed between wrist and elbow FE (r > 0.7, p