A remarkable consequence of the Hohenberg-Kohn theorem of density functional theory is the existence of an injective map between the electronic density and any observable of the many electron problem in an external potential. In this work, we study the problem of predicting a particular observable, the band gap of semiconductors and band insulators, from the knowledge of the local electronic density. Using state-of-the-art machine learning techniques, we predict the experimental band gaps from computationally inexpensive density functional theory calculations. We propose a modified Behler-Parrinello (BP) architecture that greatly improves the model capacity while maintaining the symmetry properties of the BP architecture. Using this scheme, we obtain band gaps at a level of accuracy comparable to those obtained with state of the art and computationally intensive hybrid functionals, thus significantly reducing the computational cost of the task.