Based on the problems of sparse environmental rewards and slow convergence speed encountered in the application of deep reinforcement learning algorithms to robotic arm control tasks, this paper proposes an algorithm to improve the sampling strategy and optimize the reward function. First introduce the basic principles of deep reinforcement learning and the key technology of deep deterministic strategy gradient algorithm. Then design a priority-based experience playback pool and sampling strategy, and add a reward function for the kinematic error of the robotic arm. Finally, the robot arm grasping task is used as the simulation experiment scene to obtain the comparison of the algorithm effects. Experiments show that the improved algorithm can speed up the convergence, which proves the feasibility and effectiveness of the method.