Session-based recommendation (SBR) aims to predict which item will be clicked next for the current session. Many studies have shown the advantages of utilizing graph neural networks (GNN) to learning complicated item relations in session-based recommendations. Most existing methods only utilize the item IDs to model the sequential characteristics for the current session while ignoring the inherent features of items, such as category. In some sessions, all items belong to the same category while in other sessions each item belongs to a different category. It is nontrivial to capture the inter-category and intra-category item relations effectively under this circumstance. In light of this insight, we propose Category-Aware Graph Neural Network (CAGNN) for session-based recommendation. We first construct two graphs to learn inter-category and intra-category item relations over all sessions. Then we fuse the item representations in the same category and get category representations and apply graph attention network to it to learn both item and category relations. Finally, we generate representations for each session and predict the next click. Extensive experiments on two real-world datasets show that our CAGNN model is better than most state-of-the-art baselines.