Simple Summary: Reliable preoperative differentiation of pediatric brain tumors can be challenging. While deep learning models have made significant progress in radiology, their use in pediatric populations is limited, typically through limited data availability. In this proof-of-concept study, we investigated the potential of a deep learning classifier trained on a multicenter data set of 195 children to learn to differentiate between pilocytic astrocytoma and medulloblastoma, the two most common infratentorial pediatric brain tumors, which in general present with overlapping imaging features. Our model is validated against the assessment of five independent readers of varying expertise. The final models performed strongly (AUC 0.986) on the unseen test set, correctly predicting the tumor diagnosis in 62 of 64 patients (97%). Compared to human readers, the classifier performed significantly better than relatively inexperienced readers and was on par with pediatric neuroradiologists with specific expertise in pediatric neuro-oncology. Our work highlights the potential of deep learning even in this challenging population and warrants future studies, including different tumor types and diverse acquisition protocols. Medulloblastoma and pilocytic astrocytoma are the two most common pediatric brain tumors with overlapping imaging features. In this proof-of-concept study, we investigated using a deep learning classifier trained on a multicenter data set to differentiate these tumor types. We developed a patch-based 3D-DenseNet classifier, utilizing automated tumor segmentation. Given the heterogeneity of imaging data (and available sequences), we used all individually available preoperative imaging sequences to make the model robust to varying input. We compared the classifier to diagnostic assessments by five readers with varying experience in pediatric brain tumors. Overall, we included 195 preoperative MRIs from children with medulloblastoma (n = 69) or pilocytic astrocytoma (n = 126) across six university hospitals. In the 64-patient test set, the DenseNet classifier achieved a high AUC of 0.986, correctly predicting 62/64 (97%) diagnoses. It misclassified one case of each tumor type. Human reader accuracy ranged from 100% (expert neuroradiologist) to 80% (resident). The classifier performed significantly better than relatively inexperienced readers (p < 0.05) and was on par with pediatric neuro-oncology experts. Our proof-of-concept study demonstrates a deep learning model based on automated tumor segmentation that can reliably preoperatively differentiate between medulloblastoma and pilocytic astrocytoma, even in heterogeneous data. [ABSTRACT FROM AUTHOR]