ATF4 is a major effector of the Integrated Stress Response (ISR), a homeostatic mechanism coupling cell growth and survival to bioenergetic demands. Although the pro-tumorigenic role of the ISR in a tumor cell-intrinsic manner has been established, its role in cell-extrinsic processes remains unexplored. Using novel conditional knockout ATF4 mouse models, we show that global, or fibroblast (FB)-specific loss of host ATF4 results in abnormal tumor vascularization and a pronounced tumor growth delay in syngeneic melanoma and pancreatic tumor models, a phenotype which is largely reversed by co-injection of ATF4wt/wt FBs. Single-cell tumor transcriptomics uncovered a reduction of markers associated with FB activation in a cluster of perivascular cancer-associated fibroblasts (CAF) in ATF4Δ/Δ mice. ATF4Δ/Δ FBs displayed significant defects in collagen biosynthesis and deposition and reduced ability to support angiogenesis in vitro. Mechanistically, ATF4 directly regulates the expression of the Col1a1 gene as well as the biosynthesis of glycine and proline, the major amino acids comprising collagen fibers. Analysis of human tumor samples revealed a strong correlation between ATF4 and collagen levels and between an ATF4 FB signature and expression of collagen genes. Our findings uncover a novel role of stromal ATF4 in shaping CAF functionality, a key driver of disease progression and therapy resistance.