The accuracy assessment of remote-sensing derived built-up land data represents a specific case of binary map comparison, where class imbalance varies considerably across rural-urban trajectories. Thus, local accuracy characterization of such datasets requires specific strategies that are robust to low sample sizes and different levels of class imbalance. Herein, we examine the suitability of commonly used spatial agreement measures for their localized accuracy characterization of built-up land layers across the rural-urban continuum, using the Global Human Settlement Layer and a reference database of built-up land derived from cadastral and building footprint data.