The manipulation of exciton-polaritons and their condensates is of great interest due to their applications in polariton simulators and high-speed, all-optical logic devices. Until now, methods of trapping and manipulating such condensates are not dynamically reconfigurable or result in an undesirable reduction in the exciton-photon coupling strength. Here, we present a new strategy for the ultrafast control of polariton resonances via transient modification of an optical cavity mode. We have constructed multilayer organic semiconductor microcavities that contain two absorbers: one strongly- and one weakly-coupled to the cavity photon mode. By selectively exciting the weakly-coupled absorber with ultrashort laser pulses, we modulate the cavity refractive index and generate fully-reversible blueshifts of the lower polariton branch by up to 8 meV in sub-ps timescales with no corresponding reduction in the exciton-photon coupling strength. Our work demonstrates the ability to manipulate polariton energy landscapes over ultrafast timescales with important applications in emerging computing technologies.