This paper investigates the finite-time and fixed-time bipartite consensus tracking (Fin- and Fix-TBCT) problems for second-order multi-agent systems (MASs) under a signed directed communication network, in which both cooperative and competition exist. To achieve bipartite consensus tracking (BCT) within finite time and fixed time, two novel distributed control protocols utilizing integral sliding-mode control concept are presented and discussed, respectively. By virtue of Lyapunov stability and homogeneity with dilation, several sufficient conditions for achieving Fin- and Fix-TBCT for second-order MASs are obtained. Eventually, numerical simulation results are provided to verify the validity of the obtained theoretical results.