In this paper, we investigate a learning-based caching policy in serverless edge computing (SEC), especially where critical packages for container are cached and shared. To this end, we formulate a cooperative multi-agent decision problem and solve it using deep multi-agent reinforcement learning (DMARL) approach where homogenous agents share a common reward, cache hit rate and QoS violations. The simulation results indicate that the DMARL approach performs better than baselines in the QoS violation rate.