Vital sign monitoring is essential in evaluating a person's health status and disease progress. However, most commonly used contact-based techniques may have drawbacks, such as loss of contact and skin irritations that often turn them inappropriate for long term remote continuous monitoring. Contactless approaches on the other hand offer great ease of use, are non-disturbing, and may be more suitable for everyday usage and long-term monitoring. By exploiting advances on machine learning, they offer comparable accuracy to the well-established contact-based sensors. In this work we use video as a modality to develop a contactless method to assess respiratory rate in a real-life clinical environment with non-optimal illumination, where the patient may be occluded by a blanket or may be lying in a non-optimal position. The method leverages Eulerian magnification with a 3D CNN to directly predict respiration rate. This approach eliminates the need for ROI detection and further post-processing. Moreover, since it is a motion-based method it can be applied in settings with low ambient illumination. In our case it was tested during clinical sleep studies in the course of the night. Taking these benefits into account, the resulting average MAE of 2.29 breaths/min is very well placed among and even outperforming some of the state-of-the-art methods.