Long-term urban crowd flow prediction involving the evolution trends of crowd flow is of great importance of traffic management, public safety and urban planning. However, learning long-term crowd flow is very challenging due to the latent effect of varied urban Point-of-Interests distribution, which is quite different from the short-term crowd flow mainly influenced by readily available external factors like weather, date, etc. The key issue for us is how to learn the interaction between POI distribution and human mobility in a dynamic way. To address this problem, we propose a POI-flow interaction based spatial-temporal framework (PFIST) for long-term crowd flow prediction. First, we model the long-term evolution representations of crowd flow and POI distribution. Then we study the dynamic interaction between POI transition patterns and crowd flow variation on different POI periods and categories. Afterwards, we decompose the flow sequence into long-term trend and daily variation parts and apply the normalized POI-flow interaction attention to the long-term trend parts. Finally, we model the spatial and multi-scale temporal dependencies to predict long-term crowd flow. Extensive experiments on Beijing map query track dataset and NYC taxi dataset demonstrate the superiority of PFIST.