Identification of causal drivers behind expression is essential for understanding gene function. Here authors develop a method for the large-scale inference of gene–gene interactions in observational population genomics data and characterize a network of trans-effects for 6600 genes.