The three-layer cellular network integrating mobile cloud computing (MCC) and mobile edge computing (MEC) provides an effective example for emerging services with intensive computing and low latency requirements. Computing intensive and delay sensitive tasks can be processed by local, edge nodes or cloud services. In the paper, we consider a three-tier collaborative computing network composed of mobile devices, edge nodes and cloud servers. In this network, we jointly optimize the offloading decision, the computation, communication and caching (3C) resource to minimize task delay. However, the formulated problem is a large-scale mixed integer nonlinear optimization problem with the increasing number of base stations and devices, which is difficult to solve. To deal with the problem, we propose a parallel processing scheme based on alternating direction method of multipliers (ADMM) and relaxation method. The proposed scheme first reconstructs the original problem through relaxation and auxiliary variables, then decomposes the large-scale problem into several sub problems and improves the task processing efficiency through parallel processing. Simulation results show that the proposed scheme can achieve near optimal performance under low complexity, and can reduce the task delay about 26% compared with other schemes.