Background: The objective of this work is to evaluate the performance of a modified quantitative Barrack Cement grading in primary THA. Previous studies demonstrated both poor intraobserver and interobserver reliability which may be due to the qualitative nature of the scale. Methods: Interobserver reliability of the Barrack Cement Grading System in its original format and then in combination with a quantitative measurement of implant/cement lengths was evaluated on 50 immediate post-operative radiographs of primary cemented arthroplasties. Intraobserver reliability was also assessed on a sub-sample of radiographs. Three evaluators with different skill levels and specialty participated: an arthroplasty surgeon, an orthopaedic resident and a radiologist. Reliability was measured using a weighted kappa coefficient for paired comparisons among the evaluators. Results: Interobserver reliability was poor (κ < 0.10) for all pairings of the Barrack System. The modified quantitative system achieved slight (κ < 0.20) to poor reliability. Intraobserver reliability was dependent on the skill and specialty of the evaluator with maximal values achieved for the experienced arthroplasty surgeon using the modified quantitative system (κ = 0.62). Conclusion: Use of the modified scale may improve the reliability of ratings when used by individual experienced arthroplasty surgeons. [ABSTRACT FROM AUTHOR]