As an excellent clean medium for hydrogen storage and fuel cell applications, the photolysis of ammonia via localized surface plasmon could be invoked as a promising route towards significantly reducing the temperature for conventional thermolysis. Here, we explore the underlying microscopic mechanism of ultrafast carrier dynamics in plasmon-mediated NH3 photodecomposition at the single-molecular level using real-time time-dependent density functional theory. The NH3 molecule adsorbed on the tip of archetypal magic metal clusters represented by tetrahedral Ag20 and icosahedral Ag147, splits within a hundred femtoseconds upon laser pulse illumination. We found that the splitting of the first N-H bond is dominated by the intramolecular charge transfer driven by localized surface plasmon. Surprisingly, the phase of laser pulse could modulate the dynamics of charge transfer and thus affect the plasmon-induced bond breaking. These findings offer a new avenue for NH3 decomposition and provide in-depth insights in designing highly efficient plasmon-mediated photocatalysts.