We study hypersurfaces of constant mean curvature immersed into warped product spaces of the form $\mathbb{R}\times_\varrho\mathbb{P}^n$, where $\mathbb{P}^n$ is a complete Riemannian manifold. In particular, our study includes that of constant mean curvature hypersurfaces in product ambient spaces, which have recently been extensively studied. It also includes constant mean curvature hypersurfaces in the so-called pseudo-hyperbolic spaces. If the hypersurface is compact, we show that the immersion must be a leaf of the trivial totally umbilical foliation $t\in\mathbb{R}\mapsto\{t\}\times\mathbb{P}^n$, generalizing previous results by Montiel. We also extend a result of Guan and Spruck from hyperbolic ambient space to the general situation of warped products. This extension allows us to give a slightly more general version of a result by Montiel and to derive height estimates for compact constant mean curvature hypersurfaces with boundary in a leaf.