While general relativity has passed all current observational tests, the strong-field regime is still much less constrained, leaving room for deviations that can become important in the vicinity of compact objects. In the present talk, I will discuss how black holes and neutron stars can be used to test gravity beyond general relativity, with particular emphasis on gravitational-wave observations. I will focus on several examples where modified gravity can lead to qualitatively new strong-field phenomena and potentially distinct observational signatures. Special attention will be paid to the dynamics of compact objects during stellar collapse and binary mergers.