In congested urban traffic scenarios, accurately counting vehicles with video camera is a critical task with profound implications for traffic management and urban planning. However, this endeavor is fraught with challenges, including occlusions, undetected vehicles, and other environmental factors and data acquisition noises. To address these issues, we propose a pioneering multi-directional Multi-Line Aggregated Tracking (MLAT) approach. This approach strategically deploys multiple virtual lines across roadways, ensuring that most vehicles intersect at least one line at the specified direction. In this study, custom trained deep learning based robust vehicle detection, classification and tracking model is deployed to count vehicles across a series of virtual lines. Each detected vehicle is tracked with a unique identifier and classified while count data from these lines is aggregated to yield a comprehensive vehicle count on that specific direction. Empirical validation using real-world traffic video data underscores the superiority of our approach in terms of accuracy and robustness when compared to conventional single line counting and multi-line highest counting methods. The experiment shows that MLAT mitigates error rate by 66.5% in comparison to multi-line highest count method and 78.9% than single line count method over the sample time frame. Our approach offers significant advantages over traditional single line crossing based counting methods, excelling in mitigating occlusions, reducing the likelihood of missed vehicle counts, providing a more accurate tally of traffic flow, and accommodating multi-way street scenarios. By surmounting the limitations of existing techniques, our approach presents a valuable tool for traffic management and urban planning, enabling informed decision-making and resource allocation in congested urban traffic environments.