2.1 Congruence arithmetic quotients

The general setup we work in is as follows. Let G be a semisimple algebraic group over ${\mathbb Q}$ , K the maximal compact subgroup of $G({\mathbb R})$ , and $X=G({\mathbb R})/K$ the symmetric space. For a congruence subgroup $\Gamma \subset G({\mathbb Q})$ , the quotient

$$ \begin{align*} M_{\Gamma} = \Gamma \backslash X \end{align*} $$

is noncompact when G is ${\mathbb Q}$ -isotropic. The following three cases will be the main ones of interest to us:

1. (i) $G({\mathbb R})^{nc}=SO(1,d)$ for $d \geq 2$ , so that X is a real hyperbolic d-space and $M_{\Gamma }$ is a congruence hyperbolic manifold;

2. (ii) $G({\mathbb R})^{nc}=SU(1,n)$ for $n \geq 2$ , so that X is the complex unit n-ball and $M_{\Gamma }$ is a congruence ball quotient (or congruence complex hyperbolic manifold);

3. (iii) $G({\mathbb R})^{nc}= SO(2,n)$ for $n\geq 3$ , so that

   $$ \begin{align*} X=SO(2,n)/S(O(2) \times O(n))= SO_0(2,n)/SO(2) \times SO(n) \end{align*} $$
   and $M_{\Gamma }$ is a Hermitian locally symmetric space which we will refer to, by an abuse of terminology, as an orthogonal Shimura variety.

In cases (ii) and (iii) the symmetric space X has an Hermitian structure, so that $M_{\Gamma }$ is a smooth complex manifold if $\Gamma $ is small enough. We will also be interested in the general case when X has an Hermitian structure; by an abuse of terminology, we will then refer to $M_{\Gamma }$ as a Shimura variety.

Let $G({\mathbb R})^c$ be the compact real form of $G({\mathbb R})$ . The compact symmetric space dual to X is

$$ \begin{align*} X^c=G({\mathbb R})^c/K. \end{align*} $$

In our three cases, $X^c$ is (i) the d-sphere, (ii) the complex projective n-space $\mathbb {P}^n$ , and (iii) a quadric in $\mathbb {P}^{n+1}$ . In all three cases there is a natural embedding $X \subset X^c$ and the action of $G({\mathbb R})$ extends to the closure of X in $X^c$ : In cases (ii) and (iii) it is the familiar $G({\mathbb R})$ -equivariant Borel embedding of X in the flag variety, and in case (i) it is clear, for example, from the upper half-space model of hyperbolic space.

2.2 Minimal compactification

In all three of our cases, there is a canonical open immersion

$$ \begin{align*} j:M_{\Gamma} \hookrightarrow M^*_{\Gamma} \end{align*} $$

into a compact space, which we will call the minimal compactification. For cases (i) and (ii) it is the obvious cusp compactification and also coincides with the reductive Borel–Serre compactification. For cases (ii) and (iii) and more generally, for any arithmetic quotient of an Hermitian symmetric domain, it is the Satake–Baily–Borel compactification of $M_{\Gamma }$ as a projective variety, and we will describe it in some more detail in this generality.

The closure of X in $X^c$ decomposes as a disjoint union of boundary components, which are (by definition) the maximal connected complex submanifolds of the closure. The stabilizer of a proper (i.e., $\neq X$ ) boundary component is a product of maximal parabolic subgroups of the simple factors of $G({\mathbb R})$ , and the boundary component is called rational if the stabilizer is defined over ${\mathbb Q}$ , in which case it is a maximal ${\mathbb Q}$ -parabolic of G. As a topological space, $M^*_{\Gamma } = \Gamma \backslash X^*$ , where

(2.1) $$ \begin{align} X^* = {\bigsqcup}_{F\textrm{rational}} F \quad \subset X^c \end{align} $$

is the union of all rational boundary components of X, equipped with the Satake topology. The action of $G({\mathbb Q})$ on X extends to a continuous action on $X^*$ ; the stabilizer of a rational boundary component F is a maximal ${\mathbb Q}$ -parabolic subgroup (in which case F is proper – i.e., $F \subset X^*-X$ ) or G itself (the case $F=X$ ). The Baily–Borel theory [Reference Baily and Borel5] puts an analytic structure on $M^*_{\Gamma }$ inducing the given holomorphic structure on each stratum, and this structure is unique. Moreover, $M^*_{\Gamma }$ has a unique structure of projective algebraic variety compatible with this analytic structure, and this gives a canonical quasi-projective structure on $M_{\Gamma }$ . The decomposition (2.1) induces an algebraic stratification of $M^*_{\Gamma }$ .

Example 2.3. If $G({\mathbb R})^{nc}$ is isogenous to $SO(2,n)$ , the boundary components have complex dimension $1$ (i.e., they are upper half-planes) or $0$ (points). The natural filtration of $X^*$ induces a filtration by Zariski open subsets

$$ \begin{align*} M_{\Gamma} \subset M_{\Gamma}^1 \subset M^*_{\Gamma}, \end{align*} $$

with $Z^1_{\Gamma }=M^1_{\Gamma }-M_{\Gamma }$ a disjoint union of curves and $Z^0_{\Gamma }=M^*_{\Gamma }-M^1_{\Gamma }$ a finite set of cusps. (Either or both of $Z^0_{\Gamma }$ and $Z^1_{\Gamma }$ may be empty.)

The local geometry of the stratification of $M^*_{\Gamma }$ is closely tied to the structure of parabolic subgroups, as we now review (see, e.g., [Reference Ash, Mumford, Rapoport and Tai3, Sections III.4.1–III.4.2], [Reference Baily and Borel5, Section 3], [Reference Looijenga and Rapoport31, Section 6.1], or [Reference Goresky and Pardon25, Sections 7.1–7.3]). We will assume that the adjoint group $G^{ad}$ is ${\mathbb Q}$ -simple. Let P be a maximal rational parabolic subgroup. The unipotent radical W is an extension $1 \to U \to W \to V \to 1$ , where U is the center of W and V is abelian. For the Lie algebras ${\mathfrak w}=\textrm{Lie} W({\mathbb R}), \mathfrak {u}=\textrm{Lie} U({\mathbb R}), {\mathfrak v}=\textrm{Lie} V({\mathbb R})$ , we have an extension $0 \to \mathfrak {u}\to {\mathfrak w}\to {\mathfrak v} \to 0$ . The action of A on $\mathfrak {u}=\textrm{Lie} U({\mathbb R})$ is by the square of the positive (with respect to P) generator $\chi $ of $X^*(A)$ , and the action on $\mathfrak {v}=\textrm{Lie} V({\mathbb R})$ is by $\chi $ (if $\mathfrak {v}\neq 0$ ). The Levi quotient $M=P/W$ has a decomposition $M=M_{\ell } M_h A$ , where $A \cong \mathbb {G}_m$ is the maximal ${\mathbb Q}$ -split central torus in M, $M_{\ell }$ and $M_h$ commute, (any lift of) $M_h$ centralizes U, $M_h$ contains no nontrivial connected ${\mathbb Q}$ -anisotropic subgroup, and $M_h({\mathbb R})$ gives an Hermitian symmetric space, which is the rational boundary component corresponding to P. (The relation with the ‘five-factor decomposition’ of [Reference Ash, Mumford, Rapoport and Tai3, §4.1] is the following: If P is the stabilizer of F and $P= G_h(F)G_{\ell }(F)M(F)V(F)U(F)$ as in [Reference Ash, Mumford, Rapoport and Tai3, §4.1], then $W=W(F), U=U(F), V\cong V(F), M_{\ell } \cong G_{\ell }(F)M(F)$ , and $M_hA\cong G_h(F)$ .) Note that if G is simply connected, the same is true of the derived group of the Levi, so $M^{der}= M_{\ell }^{der} \times M_h$ , and hence $M_h$ is also simply connected.

2.3 Direct limits

Let G be a semisimple ${\mathbb Q}$ -algebraic group, $X = G({\mathbb R})/K$ the symmetric space, and $M_{\Gamma } = \Gamma \backslash X$ for congruence $\Gamma $ . For $\Gamma ' \subset \Gamma $ the covering map $M_{\Gamma '} \to M_{\Gamma }$ gives pullback maps in cohomology and compactly supported cohomology, so taking colimits over all congruence subgroups, we define

(2.2) $$ \begin{align} \begin{aligned} \textrm{H}^i(\mathscr{M}_G) &:= \textrm{colim}_{\Gamma} \textrm{H}^i(M_{\Gamma}), \\ \textrm{H}^i_c(\mathscr{M}_G) &:= \textrm{colim}_{\Gamma} \textrm{H}^i_c(M_{\Gamma}), \\ \textrm{H}^i_{!}(\mathscr{M}_G) &:= \textrm{colim}_{\Gamma} \textrm{H}^i_{!}(M_{\Gamma}), \end{aligned} \end{align} $$

where, as usual, $\textrm{H}^i_{!}(M_{\Gamma }) = \textrm{im}\left (\textrm{H}^i_c(M_{\Gamma }) \to \textrm{H}^i(M_{\Gamma })\right )$ is the interior cohomology. All these are smooth $G({\mathbb Q})$ -modules, in the sense that the stabilizer of a vector is a congruence subgroup. The action of $g\in G({\mathbb Q})$ on $\textrm{H}^i(M_{\Gamma })\subset \textrm{H}^i(\mathscr {M}_G)$ is given by the pullback $\textrm{H}^i(M_{\Gamma }) \to \textrm{H}^i\left (M_{g\Gamma g^{-1}}\right )$ by the isomorphism $M_{g\Gamma g^{-1}} \to M_{\Gamma }$ induced by left translation by $g^{-1}$ on the universal cover X. The transition maps in the colimits are injective, and $\textrm{H}^*(M_{\Gamma })$ can be recovered as the $\Gamma $ -invariants in $\textrm{H}^*(\mathscr {M}_G)$ . The same remarks apply to $\textrm{H}^*_c(\mathscr {M}_G)$ and $\textrm{H}^*_{!}(\mathscr {M}_G)$ .

When the symmetric space X is Hermitian or $G({\mathbb R})^{nc}$ is isogenous to $SO(1,d)$ , we also have the minimal compactification $M^*_{\Gamma }$ as in §2.2, and we can define

(2.3) $$ \begin{align} \textrm{H}^i(\mathscr{M}^*_G) := \textrm{colim}_{\Gamma} \textrm{H}^i\left(M_{\Gamma}^*\right). \end{align} $$

This is a smooth $G({\mathbb Q})$ -module, and in the Hermitian case it carries a mixed (ind-)Hodge structure. In particular, it has a weight filtration with weights $\leq i$ in degree i, and the graded pieces are

$$ \begin{align*} \textrm{Gr}^W_j\textrm{H}^i\left(\mathscr{M}^*_G\right) = \textrm{colim}_{\Gamma} \textrm{Gr}^W_j\textrm{H}^i\left(M_{\Gamma}^*\right) \end{align*} $$

by strictness of the weight filtration.

The inductive setup requires the use of nontrivial coefficients (at the boundary) to treat the case of trivial coefficients. A finite-dimensional algebraic representation E of $G({\mathbb C})$ gives a local system on $M_{\Gamma }$ which, for simplicity, we continue to denote E, and we can consider $\textrm{H}^*(M_{\Gamma },E)$ , the colimit

$$ \begin{align*} \textrm{H}^*(\mathscr{M}_G,E)=\textrm{colim}_{\Gamma} \textrm{H}^*(M_{\Gamma},E), \end{align*} $$

and similarly $\textrm{H}^*_c(\mathscr {M}_G,E)$ and $\textrm{H}^*_!(\mathscr {M}_G,E)$ . For minimal compactifications $j_{\Gamma }:M_{\Gamma } \hookrightarrow M^*_{\Gamma }$ , we take the sheaf $\textrm{H}^0(j_{\Gamma *}E)$ – this is the ordinary, underived push-forward – and define

$$ \begin{align*} \textrm{H}^*\left(\mathscr{M}^*_G, E\right) := \textrm{colim}_{\Gamma}\textrm{H}^*\left(M^*_{\Gamma}, \textrm{H}^0(j_{\Gamma*}E)\right). \end{align*} $$

2.4 Restriction maps

Now suppose that $H \subset G$ is an injective homomorphism of semisimple ${\mathbb Q}$ -groups. Choosing (as we may) a maximal compact K in $G({\mathbb R})$ such that $K_H=K\cap G({\mathbb R})$ is maximal compact, and letting $\Gamma _H = \Gamma \cap H({\mathbb Q})$ and $M_{H,\Gamma _H} = \Gamma _H \backslash H({\mathbb R})/K_H$ , we get a map $\iota :M_{H,\Gamma _H} \longrightarrow M_{\Gamma }$ which is well known to be proper. Thus there are induced pullback maps $\textrm{H}^i(M_{\Gamma }) \to \textrm{H}^i\left (M_{H,\Gamma _H}\right )$ and $\textrm{H}^i_c(M_{\Gamma }) \to \textrm{H}^i_c\left (M_{H,\Gamma _H}\right )$ (the latter because $M_{H, \Gamma _H} \to M_{\Gamma }$ is proper). These are compatible under the natural maps $\textrm{H}^*_c(\cdot ) \to \textrm{H}^*(\cdot )$ , forgetting supports, and hence induce $\textrm{H}^i_{!}(M_{\Gamma }) \to \textrm{H}^i_{!}\left (M_{H,\Gamma _H}\right )$ . In the limit over $\Gamma $ we have $H({\mathbb Q})$ -equivariant maps

$$ \begin{align*} \iota^*:\textrm{H}^i_{(c)}(\mathscr{M}_G) \to \textrm{H}^i_{(c)}(\mathscr{M}_H) \end{align*} $$

in cohomology and compactly supported cohomology. There are induced homomorphisms of smooth $G({\mathbb Q})$ -modules

$$ \begin{align*} \textrm{Res}: \textrm{H}^i_{(c)}(\mathscr{M}_G) \longrightarrow I_H^G\,\textrm{H}^i_{(c)}(\mathscr{M}_H), \end{align*} $$

where $I_H^G$ is an induction functor such that for a smooth $H({\mathbb Q})$ -module U, $I_H^GU$ consists of functions $f:G({\mathbb Q}) \to U$ such that $f(gh)= h^{-1}\cdot f(g)$ and f is left-invariant by a congruence subgroup of $G({\mathbb Q})$ , and the action of $g \in G({\mathbb Q})$ is by $(g\cdot f)(x) =f\left (g^{-1}x\right )$ . Then $I_H^G$ is exact, takes smooth modules to smooth modules, and is right adjoint to restriction (these facts are completely elementary, see [Reference Nair35, Section 3.1]). Explicitly, $\textrm{Res}$ is given by

$$ \begin{align*} \textrm{Res}(\alpha)(g) = \iota^*\left(g^{-1}\cdot \alpha\right) = \iota^*((g\cdot )^*\alpha). \end{align*} $$

Note that $\textrm{Res}$ restricts to a map $\textrm{Res}:\textrm{H}^i_!(\mathscr {M}_G) \to I_H^G\,\textrm{H}^i_!(\mathscr {M}_H)$ on interior cohomology.

When both H and G give Hermitian symmetric spaces and K is chosen (as it may be) so that the map $H({\mathbb R})/K_H \to G({\mathbb R})/K$ is holomorphic, the map $M_{H,\Gamma _H} \to M_{\Gamma }$ extends to a morphism $M_{H,\Gamma _H}^* \to M_{\Gamma }^*$ of varieties of minimal compactifications. This well-known general fact (compare [Reference Satake44] or [Reference Harris27, Section 3.3]) is easily seen in our primary cases of interest using the description of $M^*_{\Gamma }$ given in §2.2. Pullback induces an $H({\mathbb Q})$ -equivariant map $\textrm{H}^i\left (\mathscr {M}_G^*\right ) \to \textrm{H}^i\left (\mathscr {M}^*_H\right )$ , which gives a homomorphism of mixed Hodge structures

$$ \begin{align*} \textrm{Res}: \textrm{H}^i\left(\mathscr{M}^*_G\right) \longrightarrow I_H^G\textrm{H}^i\left(\mathscr{M}_H^*\right) \end{align*} $$

by adjunction. There is a similar mapping in the real hyperbolic cases where $H({\mathbb R})^{nc} \subset G({\mathbb R})^{nc}$ is $SO(1,c) \subset SO(1,d)$ (up to isogeny) and in the ‘mixed’ case $SO(1,n) \subset SU(1,n)$ , coming from the obvious extension of $M_{H,\Gamma _H} \to M_{\Gamma }$ to minimal compactifications.

Now assume that $H\subset G$ is such that the restriction of finite-dimensional representations from G to H is multiplicity-free. The situations we will treat are well known to be of this type, by classical branching laws (e.g. [Reference Goodman and Wallach22, Section 8.1.1]). Choose Borel subgroups $B_H \subset B$ and maximal tori $T_H \subset B_H$ and $T\subset B$ of $H({\mathbb C})$ and $G({\mathbb C})$ , and for E with B-highest weight $\lambda \in X^*(T)$ let $E_H$ be the unique summand of $E\rvert _{H({\mathbb C})}$ with highest weight $\lambda \rvert _{T_H}$ . The composition $\textrm{H}^*(\mathscr {M}_G,E) \to \textrm{H}^*(\mathscr {M}_H, E\rvert _H) \to \textrm{H}^*(\mathscr {M}_H, E_H)$ is $H({\mathbb Q})$ -invariant and induces a restriction map

$$ \begin{align*} \textrm{Res}:\textrm{H}^*(\mathscr{M}_G, E) \longrightarrow \textrm{H}^*(\mathscr{M}_H,E_H) \end{align*} $$

with coefficients. There are similar maps for $\textrm{H}^*_c(\mathscr {M}_G,E)$ , $\textrm{H}^*_{!}(\mathscr {M}_G,E)$ , and $\textrm{H}^*\left (\mathscr {M}^*_G,E\right )$ .

The use of $\textrm{Res}$ for several different maps should cause no confusion, as we will always specify the domain when discussing injectivity results. We will also frequently write $\textrm{H}^i(\mathscr {M}), \textrm{H}^i(\mathscr {M}^*)$ , and so on, for $\textrm{H}^i(\mathscr {M}_G), \textrm{H}^i\left (\mathscr {M}^*_G\right )$ , and so on – dropping the subscript G when it is clear from context.

2.5 Higher direct images in the minimal compactification

We will use a well-known description of the restriction of $j_*{\mathbb Q}_{M_{\Gamma }}$ to a stratum of $M^*_{\Gamma }$ in the case of Shimura varieties. (Here by $j_*$ we mean the push-forward on the level of derived categories – that is, $Rj_*$ in old-fashioned notation.)

To fix notation, let $i_S:S \hookrightarrow M^*_{\Gamma }$ be a stratum of the minimal compactification. Choose a rational boundary component $F \twoheadrightarrow S$ and let $P=MW$ be the stabilizer of F and $M=M_hM_{\ell } A$ as in §2.2. For the congruence subgroup $\Gamma $ , let $\Gamma _W = \Gamma \cap W({\mathbb Q})$ , $\Gamma _P = \Gamma \cap P({\mathbb Q})$ , $\Gamma _M =\Gamma _P/\Gamma _W$ , $\Gamma _{M_{\ell }} = \Gamma _M \cap M_{\ell }({\mathbb Q})$ , and $\Gamma _{M_h} = \Gamma _M/\Gamma _{M_{\ell }}$ . These are all neat arithmetic subgroups when $\Gamma $ is neat.

The description (2.5) of cohomology sheaves can be found, for example, in [Reference Looijenga and Rapoport31, Proposition 5.6] or [Reference Goresky, Harder and MacPherson23, Corollary 22.8]. The weight filtration on $\textrm{H}^k\left (i_S^*j_*{\mathbb Q}_{M_{\Gamma }}\right )_s$ comes from the theory of mixed Hodge modules, and the assertion in (2) is due to Looijenga and Rapoport [Reference Looijenga and Rapoport31, Proposition 5.6]. (The analogue in the l-adic setting is in [Reference Pink39].) The existence of the decomposition (2.4) in the derived category can in fact be deduced from this, but instead one can use [Reference Burgos and Wildeshaus19, Theorem 2.9], which proves the direct sum decomposition (2.4) in the derived category of mixed Hodge modules and the identification of the graded in the context of Shimura varieties. The isomorphisms (2.4) and (2.5) are equivariant for the actions of $(M_{\ell } A W)({\mathbb Q})$ on both sides (which factor through $(M_{\ell } A)({\mathbb Q})$ ).

3.1 Some cohomological facts

We will use some facts about the cohomology of (possibly) singular varieties, summarized in Proposition 3.3 and Lemma 3.5.

Recall that by [Reference Deligne20] the rational cohomology $\textrm{H}^*(X)=\textrm{H}^*(X,{\mathbb Q})$ and homology $\textrm{H}_*(X)=\textrm{H}_*(X,{\mathbb Q})$ of a complex algebraic variety X carry rational mixed Hodge structures; in particular, they have weight filtrations. The theory of mixed Hodge modules ([Reference Saito42], especially §4) gives a relative version of mixed Hodge structures and allows for sheaf-theoretic arguments, mirroring the situation in l-adic cohomology over finite fields [Reference Beilinson, Bernstein and Deligne6]. Let X be an irreducible complex variety of dimension d. Let ${\mathbb Q}_X^H$ be the canonical lift of ${\mathbb Q}_X$ to an object in the derived category of mixed Hodge modules on X – that is, ${\mathbb Q}_X^H=a_X^*{\mathbb Q}^H$ , where $a_X:X \to Spec({\mathbb C})$ and ${\mathbb Q}^H$ is the trivial Hodge structure. The rational cohomology, compactly supported cohomology, homology, and Borel–Moore homology groups of X acquire mixed Hodge structures via

(3.1) $$ \begin{align} \textrm{H}^i(X)&={\mathbb H}^i\left(X,{\mathbb Q}_X^H\right), & \textrm{H}^i_c(X) & = {\mathbb H}^i_c\left(X,{\mathbb Q}_X^H\right), \end{align} $$

(3.2) $$ \begin{align} \textrm{H}_i(X) &= {\mathbb H}^{-i}_c\left(X,\mathbb{D}{\mathbb Q}_X^H\right), & \textrm{H}_i^{BM}(X) & = {\mathbb H}^{-i} \left(X,\mathbb{D}{\mathbb Q}_X^H\right), \end{align} $$

where $\mathbb {D}$ is the Verdier duality functor, normalized so that $\mathbb {D}{\mathbb Q}_X^H = {\mathbb Q}^H_X[2d](d)$ if X is smooth. The weights are determined by the fact that ${\mathbb Q}_X^H$ has weights $\leq 0$ , so, for example, $\textrm{H}^i_c(X)$ has weights $\leq i$ and $\textrm{H}_i^{BM}(X)$ has weights $\geq -i$ . When X is proper, which is the main case of interest to us, we have $\textrm{H}^i(X)=\textrm{H}^i_c(X)$ and $\textrm{H}_i(X) = \textrm{H}_i^{BM}(X)$ , so that $\textrm{H}^i(X)$ has weights $\leq i$ and $\textrm{H}_i(X)$ has weights $\geq -i$ .

We will also use the intersection complex $IC_X^H = \left (j_{!*}{\mathbb Q}_U^H[d]\right )[-d]$ , where $j:U \hookrightarrow X$ is the inclusion of an open dense smooth subset; this lifts the topological intersection complex $IC_X=\left (j_{!*}{\mathbb Q}_U[d]\right )[-d]$ of X, and it is pure of weight $0$ . (Our notation is slightly different from [Reference Saito42], where $IC_X\left ({\mathbb Q}^H\right )$ is used for $j_{!*}{\mathbb Q}_U[d]$ .) Replacing ${\mathbb Q}_X^H$ by $IC_X^H$ in equation (3.2) defines rational mixed Hodge structures

$$ \begin{align*} \textrm{IH}^i(X),\quad \textrm{IH}^i_c(X),\quad \textrm{IH}_i(X),\quad \textrm{IH}_i^{BM}(X) \end{align*} $$

on the intersection cohomology, intersection cohomology with compact support, intersection homology, and Borel–Moore intersection homology, respectively. When X is proper, these are all pure, $\textrm{IH}^*(X)=\textrm{IH}^*_c(X)$ and $\textrm{IH}_*(X)=\textrm{IH}_*^{BM}(X)$ , and the isomorphism $\mathbb {D}IC_X^H = IC^H_X[2d](d)$ extending $\mathbb {D}{\mathbb Q}_U^H = {\mathbb Q}^H_U[2d](d)$ on any smooth open subset $U\subset X$ induces duality isomorphisms $\textrm{IH}^i(X) \cong \textrm{IH}_{2d-i}(X)(-d)$ for all i.

Proposition 3.3. If $f:X \to Y$ is a morphism of varieties, there are maps

(3.3) $$ \begin{align} & f^{*} : {\textrm{Gr}}^{W}_{i} {\textrm{H}}^{i}(Y) \to \textrm{Gr}^{W}_{i} {\textrm{H}}^{i}(X), \notag \\ & f_{*} : W_{-i}{\textrm{H}}_{i}(X) \to W_{-i} {\textrm{H}}_{i}(Y) \end{align} $$

for each i satisfying

(3.4) $$ \begin{align} f_*(f^*(\alpha)\cap \beta) = \alpha \cap f_*(\beta) \qquad \textrm{for }\alpha \in \textrm{Gr}^W_i\textrm{H}^i(Y),\,\beta \in W_{-j}\textrm{H}_j(X). \end{align} $$

If X is an irreducible proper variety of dimension d, then the following are true:

1. (1) $\textrm{H}^i(X)$ has weights $\leq i$ , $\textrm{H}_i(X)$ has weights $\geq -i$ , and the extreme weights are given by

   (3.5) $$ \begin{align} \begin{aligned} \textrm{Gr}^W_i \textrm{H}^i(X) &= \textrm{im}\left(\textrm{H}^i(X) \to \textrm{IH}^i(X)\right), \\ W_{-j}\textrm{H}_j(X) &= \textrm{im}\left(\textrm{IH}_j(X) \to \textrm{H}_j(X)\right) \end{aligned} \end{align} $$
   for all $i,j$ .

2. (2) If $[X] \in \textrm{H}_{2d}(X)(-d)$ is the fundamental class of X, then

   (3.6) $$ \begin{align} \cap [X]: \textrm{Gr}^W_i \textrm{H}^i(X) \longrightarrow W_{-(2d-i)}\textrm{H}_{2d-i}(X)(-d) \end{align} $$
   is an isomorphism for all i.

3. (3) If $i:Z \hookrightarrow X$ is an irreducible closed subvariety of codimension c, then the cycle class

   $$ \begin{align*} \xi_{X,Z}:= (\cap [X])^{-1}(i_*[Z]) \in \textrm{Gr}^W_{2c}\textrm{H}^{2c}(X)(c) \end{align*} $$
   has the property that if $\alpha \in \textrm{H}^{2\dim Z}(X)$ with $i^*(\alpha ) = \xi _{Z,pt}$ , then $\alpha \cdot \xi _{X,Z} = \xi _{X,pt}$ .

Proof. The statements about $f^*$ and $f_*$ are simply the functoriality of the weight filtration and the fact that when homology is considered as a module over the cohomology ring using the cap product, push-forward in homology is a module over pullback in cohomology. Statements (1) and (2) are contained in [Reference Saito42, §4.5], but for the reader’s convenience we outline the arguments.

For the standard cohomology functor $\textrm{H}^0$ on mixed Hodge modules (which corresponds to the perverse cohomology functor ${}^p\textrm{H}^0$ on sheaves), we have dual natural isomorphisms

(3.7) $$ \begin{align} \begin{aligned} \textrm{Gr}^W_d \textrm{H}^d\left({\mathbb Q}_X^H\right) &= IC_X^H[d], \\ W_{-d} \textrm{H}^{-d}\left(\mathbb{D}{\mathbb Q}_X^H\right) &= IC_X^H[d](d) \end{aligned} \end{align} $$

(see [Reference Saito42, §4.5] for details). If X is proper, then these dual statements and the hypercohomology spectral sequence imply that

(3.8) $$ \begin{align} \begin{aligned} \textrm{Gr}^W_i \textrm{H}^i(X) &= \textrm{im}\left(\textrm{H}^i(X) \to \textrm{IH}^i(X)\right), \\ W_{-j}\textrm{H}_j(X) &= \textrm{im}\left(\textrm{IH}_j(X) \to \textrm{H}_j(X)\right), \end{aligned} \end{align} $$

as claimed in (1).

An irreducible variety X has a fundamental class in Borel-Moore homology,

$$ \begin{align*} [X] \in \textrm{H}_{2d}^{BM}(X)(-d) = {\mathbb H}^0\left(X, \left(\mathbb{D}{\mathbb Q}_X^H\right)[-2d](-d)\right) = \textrm{Hom}\left({\mathbb Q}_X^H, \left(\mathbb{D}{\mathbb Q}_X^H\right)[-2d](-d)\right), \notag \end{align*} $$

giving the fundamental class homomorphism ${\mathbb Q}_X^H \to \left (\mathbb {D}{\mathbb Q}^H_X\right )[-2d](-d)$ , which is an isomorphism if X is smooth. By the identities (3.7) and standard facts about the t-structure and weights, it factors as

(3.9) $$ \begin{align} {\mathbb Q}_X^H \longrightarrow IC_X^H \longrightarrow \left(\mathbb{D}IC_X^H\right)[-2d](-d) \longrightarrow \left(\mathbb{D}{\mathbb Q}_X^H\right)[-2d](-d), \end{align} $$

where the first arrow is the unique extension of the identity morphism ${\mathbb Q}_U \to {\mathbb Q}_U$ on U and the third arrow is its dual (up to a twist). The second is the Verdier duality isomorphism extending ${\mathbb Q}_U^H = \left (\mathbb {D}{\mathbb Q}_U^H\right )[-2d](-d)$ , and induces Poincaré duality isomorphisms

$$ \begin{align*} \textrm{IH}^i_c(X)={\mathbb H}^i_c\left(X, IC_X^H\right) \cong {\mathbb H}^{i-2d}_c\left(X, \mathbb{D}IC_X^H\right)(-d)=\textrm{IH}_{2d-i}(X)(-d) = \textrm{IH}^{2d-i}(X)^*(-d) \end{align*} $$

and

$$ \begin{align*} \textrm{IH}^i(X) = {\mathbb H}^i\left(X,IC_X^H\right)\cong {\mathbb H}^{i-2d}\left(X, \mathbb{D}IC_X^H\right)(-d)=\textrm{IH}_{2d-i}^{BM}(X)(-d), \end{align*} $$

and hence a nondegenerate pairing $\textrm{IH}^i(X) \times \textrm{IH}^{2d-i}_c(X)\to {\mathbb Q}(-d)$ . The fundamental class homomorphism induces the identity isomorphism $\textrm{Gr}^W_d \textrm{H}^d\left ({\mathbb Q}_X^H\right ) = W_{-d} \textrm{H}^{-d}\left (\mathbb {D}{\mathbb Q}_X^H)(-d\right )$ from equation (3.7) (as indeed it must, since it is the unique extension of ${\mathbb Q}^H_U[d]=\mathbb {D}{\mathbb Q}_U^H[d](d)$ on any smooth open $U \subset X$ ).

Now assume that X is proper. Then the duality of the first and third arrows in expression (3.9) implies that the cap product with the fundamental class $[X]$ induces an isomorphism

(3.10) $$ \begin{align} \cap [X]: \textrm{Gr}^W_i \textrm{H}^i(X) \overset{\simeq}\longrightarrow W_{2d-i}\textrm{H}_{2d-i}(X)(-d). \end{align} $$

This proves (2). For (3), note that if $i^*(\alpha ) = \xi _{Z,pt}$ , then $[Z]\cap i^*(\alpha )=1$ , so that

(3.11) $$ \begin{align} \begin{aligned} 1 &= i_*(i^*(\alpha) \cap [Z])\\ &= \alpha \cap i_*[Z]\\ &= [X] \cap \left(\alpha\cdot (\cap [X])^{-1}(i_*[Z])\right)\\ &= [X]\cap \alpha \cdot \xi_{X,Z}, \end{aligned} \end{align} $$

and thus $\alpha \cdot \xi _{X,Z} = (\cap [X])^{-1}(1)=\xi _{X,pt}$ .

Remark 3.4. The assertion in (2) is related to some facts in classical mixed Hodge theory which we will also use. If X is irreducible and proper and $Y \to X$ is a resolution of singularities, then

$$ \begin{align*} \textrm{Gr}^W_i\textrm{H}^i(X) = \textrm{im}\left(\textrm{H}^i(X) \to \textrm{H}^i(Y)\right) \end{align*} $$

for all i by [Reference Deligne20, Prop. 8.2.5]. This is equivalent to equation (3.5), because the pullback factors as $\textrm{H}^i(X) \to \textrm{IH}^i(X) \hookrightarrow \textrm{H}^i(Y)$ for any inclusion $\textrm{IH}^*(X) \subset \textrm{H}^*(Y)$ coming from the decomposition theorem [Reference Beilinson, Bernstein and Deligne6, Reference Saito42]. Since one also has

$$ \begin{align*} \textrm{Gr}^W_i\textrm{H}^i_c(U) = \textrm{im}\left(\textrm{H}^i_c(U) \to \textrm{IH}^i(X)\right) = \textrm{im}\left(\textrm{H}^i_c(U) \to \textrm{H}^i(Y)\right), \end{align*} $$

where U is smooth dense open over which $Y \to X$ is an isomorphism, one sees that $\textrm{Gr}^W_i\textrm{H}^i_c(U) \subset \textrm{Gr}^W_i\textrm{H}^i(X)$ for all i.

The following purity lemma will be used later in §7:

3.2 Invariants and the compact dual

We will assume from now on that G is semisimple and $X=G({\mathbb R})/K$ is an Hermitian symmetric domain. In addition, we assume in this subsection that G is simply connected. We return to the use of cohomology with complex coefficients and ignore Tate twists.

We will consider the $G({\mathbb Q})$ -module $\textrm{IH}^i(\mathscr {M}^*) := \textrm{colim}_{\Gamma } \textrm{IH}^i\left (M^*_{\Gamma }\right )$ , which is smooth and admissible. Note that equation (3.5) gives an inclusion

$$ \begin{align*} \textrm{Gr}^W_*\textrm{H}^*(\mathscr{M}^*):= {\bigoplus}_i \textrm{Gr}^W_i\textrm{H}^i(\mathscr{M}^*) \subset \textrm{IH}^*(\mathscr{M}^*) \end{align*} $$

of $G({\mathbb Q})$ -modules.

Proposition 3.6. The $G({\mathbb Q})$ -modules $\textrm{Gr}^W_*\textrm{H}^*(\mathscr {M}^*)$ and $\textrm{IH}^*(\mathscr {M}^*)$ are semisimple, and the summand of invariants is given by

$$ \begin{align*} \textrm{Gr}^W_*\textrm{H}^*(\mathscr{M}^*)^{G({\mathbb Q})}= \textrm{IH}^*(\mathscr{M}^*)^{G({\mathbb Q})} = \textrm{H}^*(X^c). \end{align*} $$

The embedding of $\textrm{H}^*(X^c)$ in $\textrm{Gr}^W_* \textrm{H}^*(\mathscr {M}^*)$ is functorial: If $H\subset G$ gives a complex subdomain $X_H \subset X$ , then the obvious diagram coming from $X^c_H\subset X^c$ and $M^*_{H,\Gamma _H} \to M^*_{\Gamma }$ commutes.

Proof. There is a natural isomorphism

(3.12) $$ \begin{align} \textrm{IH}^*(\mathscr{M}^*) = \textrm{H}^*\left({\mathfrak g},K,L^2_{dis}\left(G({\mathbb Q})\backslash G({\mathbb A})\right)\right) \end{align} $$

thanks to [Reference Looijenga30, Reference Saper and Stern43] and [Reference Borel and Casselman16]. Here $L^2_{dis}$ is the $L^2$ discrete spectrum; see [Reference Nair35, Proposition 3.8] for a detailed discussion of equation (3.12). This proves the semisimplicity statements. The inclusion of the constants in $L^2$ functions induces an embedding of $\textrm{H}^*(X^c)= \textrm{H}^*({\mathfrak g},K,{\mathbb C})$ in $\textrm{IH}^*(\mathscr {M}^*)$ . It follows from equation (3.12) using strong approximation and the density of $G({\mathbb Q})$ in $G({\mathbb R})$ (weak approximation) – see, for example, the proof [Reference Nair35, Proposition 3.8], which works verbatim here – that these are all the invariants.

To show that the invariants are actually in $\textrm{Gr}^W_*\textrm{H}^*(\mathscr {M}^*)$ , we will use Chern classes of automorphic vector bundles [Reference Goresky and Pardon25, Reference Mumford32]. A finite-dimensional representation V of K gives a homogenous bundle ${\mathscr V}^c$ on $X^c=G({\mathbb R})^c/K$ . Restricting by the Borel embedding $X \subset X^c$ (see §2.1) and dividing by $\Gamma $ gives a bundle ${\mathscr V}_{\Gamma }$ on $M_{\Gamma } = \Gamma \backslash X$ for any $\Gamma $ . The bundle ${\mathscr V}_{\Gamma }$ does not, in general, extend to a vector bundle on $M^*_{\Gamma }$ (although see Example 3.7 for an important exception), but Goresky and Pardon [Reference Goresky and Pardon25] defined classes $c^*_k({\mathscr V}_{\Gamma }) \in \textrm{H}^{2k}\left (M^*_{\Gamma }\right )$ which behave like the Chern classes of a putative extension $\mathscr {V}^*_{\Gamma }$ to $M^*_{\Gamma }$ . The main property is that for $\pi :M_{\Gamma }^{\Sigma } \to M^*_{\Gamma }$ a smooth toroidal desingularization [Reference Ash, Mumford, Rapoport and Tai3], the pullback $\pi ^*\left (c_k^*({\mathscr V}_{\Gamma })\right )=c_k\left ({\mathscr V}_{\Gamma }^{\Sigma }\right )$ is the Chern class of Mumford’s canonical extension ${\mathscr V}_{\Gamma }^{\Sigma }$ [Reference Harris27, Reference Mumford32]. It is a well-known consequence of Mumford’s generalization of Hirzebruch proportionality that the classes $c_k\left ({\mathscr V}_{\Gamma }^{\Sigma }\right )$ generate a copy of $\textrm{H}^*(X^c)$ in $\textrm{H}^*\left (M_{\Gamma }^{\Sigma }\right )$ . More precisely, there is an injective homomorphism $\theta :\textrm{H}^*(X^c) \to \textrm{H}^*\left (M_{\Gamma }^{\Sigma }\right )$ such that $\theta (c_k({\mathscr V}^c)) = (-1)^kc_k\left ({\mathscr V}_{\Gamma }^{\Sigma }\right )$ for all $k, {\mathscr V}$ (see Lemma C.1 for a proof, following [Reference Nair34, Lemma 3.7.2]). They are contained in $\textrm{Gr}^W_*\textrm{H}^*\left (M^*_{\Gamma }\right ) = \textrm{im}\left ( \textrm{H}^*\left (M^*_{\Gamma }\right ) \to \textrm{H}^*\left (M_{\Gamma }^{\Sigma }\right )\right )$ , since $\pi ^*\left (c^*_k({\mathscr V}_{\Gamma })\right )=c_k\left ({\mathscr V}_{\Gamma }^{\Sigma }\right )$ by [Reference Goresky and Pardon25]. Moreover, the compatibility of the construction for different $\Sigma $ (see, e.g., [Reference Harris27, Section 4.3.1]) shows that we have a well-defined embedding $\theta :\textrm{H}^*(X^c) \to \textrm{Gr}^W_*\textrm{H}^*\left (M^*_{\Gamma }\right )$ .

It remains to show that the classes are $G({\mathbb Q})$ -invariant and the embedding is functorial. The direct limit $\textrm{colim}_{\Sigma ,\Gamma }\textrm{H}^*\left (M_{\Gamma }^{\Sigma }\right )$ over all pairs $(\Sigma , \Gamma )$ where $\Sigma $ is admissible for $\Gamma $ is a $G({\mathbb Q})$ -module, and contains $\textrm{Gr}^W_*\textrm{H}^*(\mathscr {M}^*)$ as a $G({\mathbb Q})$ -submodule. Standard properties of the canonical extensions listed in [Reference Harris27, Section 4.3] show that the Chern classes are $G({\mathbb Q})$ -invariants in $\textrm{colim}_{\Sigma ,\Gamma }\textrm{H}^*\left (M_{\Gamma }^{\Sigma }\right )$ and hence in $\textrm{Gr}^W_*\textrm{H}^*(\mathscr {M}^*)$ . Finally, functoriality follows from [Reference Harris27, Section 4.3.4] and the definition of the map $\theta $ given in Lemma C.1.

Example 3.7. The representation of K on the top exterior power of ${\mathfrak p}$ , where ${\mathfrak g}={\mathfrak k}+{\mathfrak p}$ is the Cartan decomposition given by K, gives a special automorphic bundle called the Baily–Borel bundle. This extends as a line bundle $\mathscr {L}^{bb}$ over $M^*_{\Gamma }$ , and some power of $\mathscr {L}^{bb}$ is the $\mathscr {O}(1)$ in the Baily–Borel projective embedding (see [Reference Mumford32, Prop. 3.4(b)]). So $\mathscr {L}^{bb}$ is ample and the Chern class $c_1^{bb}:= c_1^*(\mathscr {L})= c_1\left (\mathscr {L}^{bb}\right )$ fixes a generator

(3.13) $$ \begin{align} \left(c_1^{bb}\right)^n \in \textrm{Gr}^W_{2n}\textrm{H}^{2n}(\mathscr{M}^*) = \textrm{IH}^{2n}(\mathscr{M}^*) \end{align} $$

in top degree.

Example 3.9. If $G({\mathbb R})^{nc}=Spin(2,n)$ , the compact symmetric space dual to X is

$$ \begin{align*} X^c = Spin(2+n)/Spin(2) \times_{\{\pm I\}} Spin(n) = SO(2+n)/SO(2){\times} SO(n), \end{align*} $$

which is a quadric in $\mathbb {P}^{n+1}$ . The complex cohomology ring of quadrics is well known. Let $E_2$ and $E_n$ be the vector bundles on $X^c$ corresponding to the natural representations of $SO(2) \times SO(n)$ of dimension $2$ and n, respectively. When n is odd, the complex cohomology is generated by the Euler class (or first Chern class) $c_1=c_1(E_2) \in \textrm{H}^2(X^c)$ – that is, $\textrm{H}^*(X^c) = {\mathbb C}[c_1]/\left (c_1^{n+1}\right )$ . When $n=2d$ is even, the complex cohomology of $X^c$ is generated as a ring by $c_1$ and the Euler class $c_d=c_d(E_n)\in \textrm{H}^{n=2d}(X^c)$ , with the relations $c_1^{n+1}=0, c_d^2=(-1)^dc_1^{2d}$ , and $c_1c_d=0$ .

For later use, we remark that if $n=2d\geq 4$ and $X_a^c \subset X^c$ is the inclusion of quadrics coming from $SO(2,a) \subset SO(2,n)$ for $a<n$ , then $c_d\rvert _{X_a^c}=0$ . Indeed, $0=(c_1c_d)\rvert _{X_{n-1}^c} = c_1\rvert _{X_{n-1}^c} \cdot c_d\rvert _{X_{n-1}^c}= c_1 \cdot \left (c_d\rvert _{X_{n-1}^c}\right )$ . Since $c_1\cdot $ is injective on $\textrm{H}^*\left (X^c_{n-1}\right )={\mathbb C}[c_1]/\left (c_1^n\right )$ in degrees $<2(n-1)$ , we must have $c_d\rvert _{X_{n-1}^c}=0$ if $d\geq 2$ .

3.3 Cycle classes and an injectivity criterion

We will prove a criterion for the nonvanishing of $\textrm{Res}$ between Shimura varieties and apply it to prove Theorem 3.1 and Corollary 3.2. Since we are only interested in cohomology with complex coefficients, we will ignore Tate twists henceforth and write $\textrm{H}^*(X)$ for $\textrm{H}^*(X,{\mathbb C})$ .

Suppose now that $\iota :M^*_{H,\Gamma _H} \to M^*_{\Gamma }$ is the extension to minimal compactifications of a morphism of Shimura varieties (compare §2.4), and let $n=\dim M_{\Gamma }, m=\dim M_{H,\Gamma _H}$ . Let

$$ \begin{align*} \xi_{\Gamma} := \left(\cap \left[M^*_{\Gamma}\right]\right)^{-1}\left(\iota_*\left[M^*_{H,\Gamma_H}\right]\right) \in \textrm{Gr}^W_{2(n-m)} \textrm{H}^{2(n-m)}\left(M^*_{\Gamma}\right) \end{align*} $$

be the cycle class defined earlier in Proposition 3.3, ignoring Tate twists and simplifying the notation $\Big ($ in the notation of that proposition, this would be $\xi _{M^*_{\Gamma },M^*_{H,\Gamma _H}}\Big )$ . It is easily checked that if $\Gamma ' \subset \Gamma $ is normal, then $\xi _{\Gamma } = \lvert \Gamma /\Gamma '\rvert ^{-1} \sum _{\gamma \in \Gamma /\Gamma '} \gamma \cdot \xi _{\Gamma '}$ , where $\xi _{\Gamma '}= \left (\cap \left [M^*_{\Gamma }\right ]\right )^{-1}\left (\iota ^{\prime }_*\left (\left [M^*_{H,\Gamma ^{\prime }_H}\right ]\right )\right )$ for $\iota ':M^*_{H,\Gamma ^{\prime }_H} \to M^*_{\Gamma '}$ at level $\Gamma '$ .

We will also consider the closed immersion $\iota ^c:X^c_H \to X^c$ , which gives the cycle class

$$ \begin{align*} \xi_{X^c_H} :=(\cap [X^c])^{-1}\left(\iota^c_*\left[X^c_H\right]\right), \end{align*} $$

which is nonzero since $X^c_H$ is a subvariety of the algebraic variety $X^c$ .

Proof. The $G({\mathbb Q})$ -submodule $V \subset \textrm{Gr}^W_{2(n-m)}\textrm{H}^{2(n-m)}(\mathscr {M}^*) \subset \textrm{IH}^{2(n-m)}(\mathscr {M}^*)$ generated by $\xi _{\Gamma }$ admits a decomposition $V=V^0\oplus V^1$ , where $V^1$ has no invariants or coinvariants and $V^0$ is contained in the summand of invariants $\textrm{H}^{2(n-m)}(X^c)\subset \textrm{Gr}^W_{2m}\textrm{H}^{2m}\left (M^*_{\Gamma }\right )$ . (This is because of semisimplicity in Proposition 3.6.) Write $\xi _{\Gamma } = \xi _{\Gamma }^0 + \xi _{\Gamma }^1$ , with $\xi _{\Gamma }^i\in V^i$ . Since $V^1$ has no coinvariants, $\xi _{\Gamma }^0\cdot \alpha = \xi _{\Gamma } \cdot \alpha $ for any $\alpha \in \textrm{H}^{2m}(X^c)$ , so that

(3.14) $$ \begin{align} \begin{aligned} \left[M^*_{\Gamma}\right] \cap \left(\xi_{\Gamma}^0 \cdot \alpha\right) &=\left[M^*_{\Gamma}\right] \cap (\xi_{\Gamma} \cdot \alpha)\\ &= \left[M^*_{\Gamma}\right] \cap \xi_{\Gamma} \cap \alpha \\ &= \iota_*\left[M^*_{H,\Gamma_H}\right]\cap \alpha \\ &= \iota_*\left(\left[M_{H,\Gamma_H}^*\right] \cap \iota^*(\alpha)\right). \end{aligned} \end{align} $$

On the other hand,

(3.15) $$ \begin{align} \begin{aligned} [X^c] \cap \left(\xi_{X^c_H} \cdot \alpha\right) &= [X^c] \cap \xi_{X^c_H} \cap \alpha\\ &=\iota^c_*\left[X^c_H\right] \cap \alpha\\ &= \iota^c_*\left(\left[X^c_H\right] \cap \iota^{c*}(\alpha)\right). \end{aligned} \end{align} $$

Now the pullbacks $\iota ^{c*}$ and $\iota ^*$ are compatible, while under the isomorphism $\textrm{H}^{2n}(X^c) \cong \textrm{Gr}^W_{2n}\textrm{H}^{2n}\left (M^*_{\Gamma }\right )$ the two nonzero linear forms $\left [M^*_{\Gamma }\right ]\cap :\textrm{Gr}^W_{2n}\textrm{H}^{2n}\left (M^*_{\Gamma }\right ) \to \textrm{H}_0\left (M^*_{\Gamma }\right )={\mathbb C}$ and $[X^c]\cap : \textrm{H}^{2n}(X^c)\to \textrm{H}_0(X^c)={\mathbb C}$ are necessarily proportional; and the same holds for $M^*_{H,\Gamma _H}$ and $X^c_H$ . It follows from equations (3.14) and (3.15) that for $\alpha \in \textrm{H}^{2m}(X^c)$ , we have

$$ \begin{align*} [X^c] \cap \left(\xi_{\Gamma}^0 \cdot \alpha\right) \sim [X^c] \cap \left(\xi_{X^c_H} \cdot \alpha\right), \end{align*} $$

where $\sim $ means up to a fixed nonzero constant independent of $\alpha $ . Thus $\xi _{\Gamma }^0\cdot \alpha \sim \xi _{X^c_H} \cdot \alpha $ for any $\alpha $ , and so $\xi _{\Gamma }^0\sim \xi _{X^c_H}$ by Poincaré duality for $X^c$ .

Proof. Suppose that $\alpha \in \textrm{Gr}^W_i\textrm{H}^i\left (M^*_{\Gamma }\right )\subset \textrm{Gr}^W_i\textrm{H}^i(\mathscr {M}^*)$ is such that $\textrm{Res}(\alpha )=0$ . Set $g \in G({\mathbb Q})$ and choose $\Gamma '$ normal in $\Gamma $ with $\Gamma ' \subset \Gamma \cap g^{-1}\Gamma g$ . Let $\gamma _1, \dotsc , \gamma _r$ be the representatives for cosets of $\Gamma '$ in $\Gamma $ , let $p:M^*_{\Gamma '} \to M^*_{\Gamma }$ , and let $\iota ':M^*_{H,\Gamma ^{\prime }_H} \to M^*_{\Gamma '}$ be the natural map at level $\Gamma '$ . Then we have

$$ \begin{align*} p^{-1}\left(\iota\left(M^*_{H,\Gamma_H}\right)\right) = \bigcup_i \gamma_i \cdot\iota'\left(M^*_{H, \Gamma^{\prime}_H}\right). \end{align*} $$

If $\textrm{Res}(\alpha )=0$ for $\alpha \in \textrm{Gr}^W_i\textrm{H}^i\left (M^*_{\Gamma }\right )$ , then $\left (g\gamma _i^{-1}\right )^*\alpha = \left (\gamma _i^{-1}\right )^*g^*\alpha $ restricts to zero on $\iota '\left (M^*_{H,\Gamma ^{\prime }_H}\right )$ for each i – that is, $\iota ^{\prime *}\left (\left (\gamma _i^{-1}\right )^*g^*(\alpha )\right ) =0$ for each i. Using equation (3.4), we have

(3.16) $$ \begin{align} \begin{aligned} 0 &= \iota^{\prime}_*\left(\iota^{\prime *}\left(\left(\gamma_i^{-1}\right)^*g^*\alpha\right) \cap \left[M^*_{H,\Gamma_H'}\right]\right)\\ &=\left(\gamma_i^{-1}\right)^*g^*\alpha \cap \iota^{\prime}_*\left(\left[M^*_{H,\Gamma_H'}\right]\right)\\ &=\left[M^*_{\Gamma'}\right] \cap \left(\left(\gamma_i^{-1}\right)^*g^*\alpha \cdot \xi_{\Gamma'}\right)\\ &=\left[M^*_{\Gamma'}\right] \cap \left(g^*\alpha \cdot \gamma_i^*\xi_{\Gamma'}\right). \end{aligned} \end{align} $$

By Proposition 3.3(2), we have that $g^*\alpha \cdot \gamma _i^*\xi _{\Gamma '}=0$ . Summing over $\Gamma /\Gamma '$ gives

$$ \begin{align*} 0 = g^*\alpha\cdot \xi_{\Gamma} = \alpha \cdot \left(g^{-1}\right)^*\xi_{\Gamma}. \end{align*} $$

Since this holds for all $g \in G({\mathbb Q})$ , Proposition 3.10 implies that $\alpha \cdot \xi _{X^c_H}=0$ .

Proof of Corollary 3.2. The complex hyperbolic case follows easily from the observation that

$$ \begin{align*} \textrm{H}^k_{!}(M_{\Gamma}) = \textrm{Gr}^W_k\textrm{H}^k_c(M_{\Gamma}) \qquad \textrm{for }k\leq n \end{align*} $$

(see the proof of [Reference Nair36, Proposition 1.6]) and the fact that $\textrm{Gr}^W_k\textrm{H}^k_c(M_{\Gamma }) \subset \textrm{Gr}^W_k\textrm{H}^k\left (M_{\Gamma }^*\right )$ for all k (Remark 3.4).

Now consider the orthogonal case. First note that

(3.17) $$ \begin{align} \textrm{H}^i_{!}(M_{\Gamma}) = \textrm{im}\left(\textrm{H}^i_c(M_{\Gamma}) \to \textrm{IH}^i\left(M^*_{\Gamma}\right)\right) = \textrm{Gr}^W_i \textrm{H}^i_c(M_{\Gamma}) \qquad \textrm{for } i \leq n-1. \end{align} $$

The first equality holds because $\textrm{IH}^i\left (M^*_{\Gamma }\right ) \to \textrm{H}^i(M_{\Gamma })$ is injective for $i \leq n-1$ (and an isomorphism for $i\leq n-2$ ), because the boundary has dimension $1$ , and the second holds because $\textrm{Gr}^W_i \textrm{H}^i_c(M_{\Gamma }) \hookrightarrow \textrm{IH}^i\left (M^*_{\Gamma }\right )$ for all i (see Remark 3.4). On the other hand,

$$ \begin{align*} \textrm{Gr}^W_i \textrm{H}^i_c(M_{\Gamma}) \subset \textrm{Gr}^W_i\textrm{H}^i\left(M^*_{\Gamma}\right) \end{align*} $$

for all i (by Remark 3.4 again). Thus $\textrm{H}^i_{!}(\mathscr {M}) \subset \textrm{Gr}^W_i\textrm{H}^i(\mathscr {M}^*)$ for $i\leq n-1$ , and similarly for $\mathscr {M}_H$ . So the corollary follows from Theorem 3.1 in degrees $\leq m-1$ in the case $rk_{\mathbb Q}(G)=2$ . If $rk_{\mathbb Q}(H)=1$ , then this can be improved slightly, because the singularities of $M^*_{H,\Gamma _H}$ are isolated and so $\textrm{H}^i_{!}(\mathscr {M}) \subset \textrm{Gr}^W_i\textrm{H}^i(\mathscr {M}^*)$ holds for $i=m$ also.□

The following example shows that these bounds can sometimes be improved on:

Example 3.15. Consider the case of $SO(2,2) \subset SO(2,n)$ for $n\geq 3$ , and assume $rk_{\mathbb Q}(H)=2$ , so that $H=SL(2) \times SL(2)$ . Theorem 3.1 gives the injectivity of $\textrm{Gr}^W_2\textrm{H}^2(\mathscr {M}^*) \to I_H^G\textrm{Gr}^W_2\textrm{H}^2\left (\mathscr {M}^*_H\right )$ . Since $\textrm{H}^2_!(M_{\Gamma }) = \textrm{IH}^2\left (M^*_{\Gamma }\right )= \textrm{Gr}^W_2\textrm{H}^2_c(M_{\Gamma })$ , the map $\textrm{Gr}^W_2\textrm{H}^2\left (M^*_{\Gamma }\right ) \to \textrm{Gr}^W_2\left (M^*_{H,\Gamma _H}\right )$ factors as

$$ \begin{align*} \textrm{Gr}^W_2\textrm{H}^2_c(M_{\Gamma}) \to \textrm{Gr}^W_2\textrm{H}^2_c\left(M_{H,\Gamma_H}\right) \to \textrm{Gr}^W_2\textrm{H}^2\left(M^*_{H,\Gamma_H}\right). \end{align*} $$

Now $M_{H,\Gamma _H}=X_1 \times X_2$ is a product of two modular curves, so $\textrm{Gr}^W_2\textrm{H}^2_c(X_1\times X_2) = \textrm{Gr}^W_1\textrm{H}^1_c(X_1)\otimes \textrm{Gr}^W_1\textrm{H}^1_c(X_2)$ injects into $\textrm{Gr}^W_2\textrm{H}^2\left (X_1^*\times X_2^*\right )$ , and so the second map is injective. It follows that $\textrm{Res}$ is always injective on $\textrm{H}^2_!(\mathscr {M})$ , improving Corollary 3.2 slightly.

4.1 Kostant’s theorem

We recall results of [Reference Kostant29]. Fix a complex semisimple Lie group G, a maximal torus $T\subset G$ , and a Borel subgroup $B\supset T$ , and let $\Phi =\Phi (T,G)$ be the root system, $\Phi ^+$ the positive roots determined by B, $\Phi ^-=-\Phi ^+$ the negative roots, $\rho = \frac {1}{2}\sum _{\alpha \in \Phi ^+} \alpha $ the half-sum of positive roots, and $W=W(T,G)$ the Weyl group of T in G. Let P be a standard parabolic subgroup of G, N its unipotent radical, and ${\mathfrak n}$ its Lie algebra. The Weyl group of the Levi $L=P/N$ is a subgroup $W_L \subset W$ , and we let $W^P$ be the set of minimal length coset representatives of $W_L\backslash W$ . For each $w \in W^P$ , the associated set of positive roots

$$ \begin{align*} \Phi(w) = \left\{\alpha \in \Phi(T,G): \alpha>0, w^{-1}\alpha <0\right\} = \Phi^+\cap w\Phi^-, \end{align*} $$

which has cardinality $\ell (w)$ . For a dominant $\lambda \in X^*(T)$ , the weights $w(\lambda +\rho )-\rho $ for $w \in W^P$ are dominant for L and distinct. The mapping $w \mapsto \Phi (w)$ sets up a bijection between $W^P$ and the subsets S of $\Phi ({\mathfrak n})= \{\alpha \in \Phi (T,G): {\mathfrak g}^{\alpha } \subset {\mathfrak n}\}$ for which both S and $\Phi ^+{-}S$ are closed under $\dot {+}$ [Reference Kostant29, 5.10] (recall that $\alpha \dot +\beta $ is $\alpha +\beta $ when this is a root and empty otherwise).

Let $E_{\lambda }$ be the irreducible finite-dimensional G-representation with highest weight $\lambda \in X^*(T)$ with respect to B. The Lie-algebra cohomology $\textrm{H}^*({\mathfrak n},E_{\lambda })$ is the cohomology of $\wedge ^*{\mathfrak n}^*\otimes E_{\lambda }$ with the Lie-algebra differential. The natural P-module structure on $\wedge ^*{\mathfrak n}^*\otimes E_{\lambda }$ descends to an $L=P/N$ -module structure in cohomology. For an L-dominant weight $\mu \in X^*(T)$ , let $E^L_{\mu }$ denote the irreducible finite-dimensional algebraic representation of L with highest weight $\mu $ . Then by [Reference Kostant29, Theorem 5.14] there is a multiplicity-free decomposition of L-modules

(4.1) $$ \begin{align} \textrm{H}^k({\mathfrak n},E_{\lambda}) = \bigoplus_{w \in W^{P}, \ell(w)=k} E^L_{w(\lambda+\rho){-}\rho}. \end{align} $$

Kostant also identified a highest weight vector in each summand. Let ${\mathfrak n}^{-}$ be the nilradical of the Lie algebra of the parabolic subgroup opposite to P. The Killing form gives isomorphisms ${\mathfrak n}^{-} \cong {\mathfrak n}^*$ and $\wedge ^i{\mathfrak n}^{-} \cong \wedge ^i{\mathfrak n}^*$ . Choose a nonzero vector $e_{\alpha }$ in the root space ${\mathfrak g}^{\alpha }$ for each $\alpha \in \Phi (T,G_{\mathbb C})$ , and for $w\in W^P$ define $e_w := \bigwedge _{\alpha \in \Phi (w)} e_{-\alpha } \in \wedge ^{\ell (w)} {\mathfrak n}^{-}$ . Let $v_{w\lambda }\in E_{\lambda }$ be a weight vector for the extremal weight $w\lambda $ . Then under the identification of ${\mathfrak n}^*$ with ${\mathfrak n}^{-}$ , the element

(4.2) $$ \begin{align} e_w \otimes v_{w\lambda} \in \wedge^{\ell(w)} {\mathfrak n}^{-} \otimes E_{\lambda} \end{align} $$

is closed in $\wedge ^*{\mathfrak n}^*\otimes E_{\lambda }$ and its cohomology class is a highest weight vector for the summand $E^L_{w(\lambda +\rho )-\rho }$ in equation (4.1) [Reference Kostant29, Theorem 5.14]; it is a harmonic representative for a natural Laplacian. A lowest weight vector is given by $\wedge _{\alpha \in \Phi (w)}e_{-w_0^L(\alpha )}\otimes v_{w_0^Lw\lambda }$ , where $w_0^L$ is the longest element of $W_L \subset W$ [Reference Kostant29, Remark 8.2]. In fact, taking the sum of the L-submodules of $\wedge ^*{\mathfrak n}^{-}\otimes E_{\lambda }$ generated by the $e_w\otimes v_{w\lambda }$ as w runs over $W^P$ gives (using the identification $\wedge ^*{\mathfrak n}^*\cong \wedge ^*{\mathfrak n}^{-}$ ) a canonical L-equivariant inclusion $\textrm{H}^*({\mathfrak n},E_{\lambda }) \subset \wedge ^*{\mathfrak n}^*\otimes E_{\lambda }$ inducing the identity in cohomology and compatible with products (see [Reference Kostant29, Theorem 5.7]).

4.2 Restriction maps in ${\mathfrak n}$ -cohomology

Now consider the situation where $\iota :H \to G$ is a homomorphism of real semisimple groups with finite kernel. Then for a parabolic P of G with Levi $L=P/N$ , we have the parabolic $P_H= \iota ^{-1}(P)$ of H with unipotent radical $N_H=\iota ^{-1}(N)$ and Levi $L_H=P_H/N_H$ . Let ${\mathfrak n}=\textrm{Lie} N({\mathbb R})$ and ${\mathfrak n}_H=\textrm{Lie} N_H({\mathbb R})$ . For a finite-dimensional $G({\mathbb C})$ -representation E and $E_H$ a summand of $E\rvert _{H({\mathbb C})}$ , the restriction map

$$ \begin{align*} \textrm{H}^*({\mathfrak n}_{\mathbb C}, E) \longrightarrow \textrm{H}^*\left({\mathfrak n}_{H,{\mathbb C}},E\right) \longrightarrow \textrm{H}^*\left({\mathfrak n}_{H,{\mathbb C}}, E_H\right) \end{align*} $$

is $L_H({\mathbb C})$ -equivariant. Consider the map

$$ \begin{align*} Res_{\mathfrak n}:\textrm{H}^*({\mathfrak n}_{\mathbb C}, E) \longrightarrow \prod_{m\in L({\mathbb C})} \textrm{H}^*\left({\mathfrak n}_{H,{\mathbb C}}, E_H\right), \end{align*} $$

with coordinate indexed by $m \in L({\mathbb C})$ given by precomposing the previous map with the adjoint action of m. Note that the kernel of $\textrm{Res}_{\mathfrak n}$ is an $L({\mathbb C})$ -module; in particular, for each irreducible summand $E^L_{w(\lambda +\rho )-\rho }$ , we have that $\textrm{Res}_{\mathfrak n}$ is injective on $E^L_{w(\lambda +\rho )-\rho } \Longleftrightarrow \textrm{Res}_{\mathfrak n}$ is nonzero on $E^L_{w(\lambda +\rho )-\rho } \ \Longleftrightarrow \ Res_{\mathfrak n}(e_w\otimes v_{w\lambda })\neq 0$ .

Let us assume that the restriction of finite-dimensional irreducible representations by $H({\mathbb C}) \subset G({\mathbb C})$ is multiplicity-free. Choose maximal tori $T_H \subset T$ and Borel subgroups $B_H \subset B$ (of $H{\mathbb C}) \subset G({\mathbb C})$ ), and for an irreducible representation E with highest weight $\lambda \in X^*(T)$ , let $E_H$ be the summand of $E\rvert _{H({\mathbb C})}$ with highest weight $\lambda \rvert _{T_H} \in X^*(T_H)$ . The following propositions are proved by explicit elementary calculations with roots and weights using Kostant’s theorem and take up the rest of this section. In each case, the restriction from $G({\mathbb C})$ to $H({\mathbb C})$ is multiplicity-free by classical results ([Reference Goodman and Wallach22, Section 8.1.1]).

To treat $SO(1,n)\subset SU(1,n)$ , we will need the following:

The bigrading in Proposition 4.4 refers to the Hodge structure coming from the identification of $\textrm{H}^i({\mathfrak w},{\mathbb C})$ with the link cohomology $\textrm{H}^i\left (i_{\{x\}}^*j_*{\mathbb C}\right )$ , where $j:M_{\Gamma } \hookrightarrow M^*_{\Gamma }$ and $i_{\{x\}}:\{x\}\hookrightarrow M^*_{\Gamma }$ is the inclusion of the cusp corresponding to P (see Proposition 2.7 or [Reference Nair36, Lemma 1.2].) It can also seen from the decomposition (4.1) [Reference Nair36, Remark 1.11].

The rest of this section will be taken up with the proofs of these propositions.

4.3 Proof of Proposition 4.3

To make computations we will fix some notation for roots. We may assume ${\mathfrak g}=\mathfrak {so}(2,n) = \mathfrak {so}(J)$ , where

$$ \begin{align*} J= \begin{pmatrix} &&&&1 \\ &&&1\\ && I_{n-2} &&\\ & 1\\ 1 \end{pmatrix}. \end{align*} $$

Fix a Cartan subalgebra ${\mathfrak s}$ of $\mathfrak {so}(n-2)_{\mathbb C}$ and let ${\mathfrak t} \subset {\mathfrak g}_{\mathbb C}$ be the Cartan subalgebra defined by

$$ \begin{align*} {\mathfrak t} := \{diag(a,d,C,-d,-a): a,d \in {\mathbb C}, C \in {\mathfrak s}\}. \end{align*} $$

Then ${\mathfrak t}$ is defined and maximally split over ${\mathbb R}$ , and the subspace ${\mathfrak a}_{\mathbb C} \subset \mathfrak {t}$ given by $C=0$ is the complexification of the Lie algebra ${\mathfrak a} \subset {\mathfrak g}$ of a maximal ${\mathbb R}$ -split Cartan in ${\mathfrak g}$ . Let $\alpha _1, \alpha _2 \in \mathfrak {t}^*$ be defined by

(4.3) $$ \begin{align} \begin{aligned} \alpha_1(diag(a,d,C,-d,-a))=a, \\ \alpha_2(diag(a,d,C,-d,-a))=d. \end{aligned} \end{align} $$

The relative roots are $\Phi ({\mathfrak a}_{\mathbb C},{\mathfrak g}_{\mathbb C})=\{\pm \alpha _1, \pm \alpha _2, \pm (\alpha _1-\alpha _2), \pm (\alpha _1+\alpha _2)\}$ .

Now choose for ${\mathfrak s}$ the Cartan subalgebra of block-diagonal matrices

(4.4) $$ \begin{align} {\mathfrak s} = \left\{diag\left(\left(\begin{smallmatrix} 0 & b_1 \\ -b_1 & 0\end{smallmatrix}\right), \dotsc, \left(\begin{smallmatrix} 0 & b_k \\ -b_k & 0\end{smallmatrix}\right)\right): b_1, \dotsc, b_k \in {\mathbb C}\right\} \end{align} $$

when $n-2=2k$ is even and

(4.5) $$ \begin{align} {\mathfrak s} = \left\{diag\left(\left(\begin{smallmatrix} 0 & b_1 \\ -b_1 & 0\end{smallmatrix}\right), \dotsc, \left(\begin{smallmatrix} 0 & b_k \\ -b_k & 0\end{smallmatrix}\right),0\right): b_1, \dotsc, b_k \in {\mathbb C}\right\} \end{align} $$

when $n-2=2k+1$ is odd. Let $\eta _1, \dotsc , \eta _k \in {\mathfrak s}^*$ be defined by

(4.6) $$ \begin{align} \eta_i\left(diag\left(\left(\begin{smallmatrix} 0 & b_1 \\ -b_1 & 0\end{smallmatrix}\right), \dotsc, \left(\begin{smallmatrix} 0 & b_k \\ -b_k & 0\end{smallmatrix}\right)\right), (0)\right)= \sqrt{-1}b_i, \end{align} $$

where $(0)$ means the entry is omitted when $n-2$ is even and the entry is zero when $n-2$ is odd. Fix the positive system $\Phi ^+({\mathfrak s},\mathfrak {so}(n-2)_{\mathbb C})$ with simple roots ${\{\eta _i-\eta _{i+1}:1\leq i< k\}}\sqcup \{\eta _k\}$ for $n-2=2k+1$ odd and $\{\eta _i-\eta _{i+1}:1\leq i< k\}\sqcup \{\eta _k+\eta _{k-1}\}$ for $n-2=2k$ even, and take the positive system in $\Phi ({\mathfrak t},{\mathfrak g}_{\mathbb C})$ containing it and compatible with $\Phi ^+({\mathfrak a}_{\mathbb C}, {\mathfrak g}_{\mathbb C})= \{\alpha _1,\alpha _2,\alpha _1-\alpha _2,\alpha _1+\alpha _2\}$ . An explicit computation shows that the positive roots are

$$ \begin{align*} \Phi^+({\mathfrak t},{\mathfrak g}_{\mathbb C}) = \Phi^+(\mathfrak{s},\mathfrak{so}(n-2)_{\mathbb C}) \sqcup \{\alpha_1\pm \eta_i, \alpha_2\pm \eta_i: 1\leq i \leq k\} \sqcup \{\alpha_1-\alpha_2, \alpha_1+\alpha_2\} \end{align*} $$

for $n-2=2k$ even and

$$ \begin{align*} \Phi^+({\mathfrak t},{\mathfrak g}_{\mathbb C}) = \Phi^+(\mathfrak{s},\mathfrak{so}(n-2)_{\mathbb C}) \sqcup \{\alpha_1\pm \eta_1, \alpha_2\pm \eta_i: 1 \leq i \leq k\} \sqcup \{\alpha_1,\alpha_2,\alpha_1+\alpha_2,\alpha_1-\alpha_2\} \end{align*} $$

for $n-2=2k+1$ odd.

The parabolic $P=MW$ in the proposition is the stabilizer of an isotropic plane, which may be assumed to be the obvious plane ${\mathbb R} e_1 + {\mathbb R} e_2$ in ${\mathbb R}^{n+2}$ . We will need the set $\Phi ({\mathfrak w}) = \left \{\alpha \in \Phi ({\mathfrak t},{\mathfrak g}_{\mathbb C}): {\mathfrak g}_{\mathbb C}^{\alpha } \subset {\mathfrak w}_{\mathbb C}\right \}$ . Using the foregoing description and explicit matrix descriptions, we have

$$ \begin{align*} \Phi({\mathfrak w}) = \{ \alpha_1\pm \eta_i\}_{1 \leq i \leq k} \sqcup \{\alpha_2 \pm \eta_i\}_{1 \leq i \leq k}\sqcup \{\alpha_1+\alpha_2\} \end{align*} $$

if $n-2=2k$ is even and

$$ \begin{align*} \Phi({\mathfrak w}) = \{\alpha_1\pm \eta_i\}_{1\leq i \leq k}\sqcup \{\alpha_2 \pm \eta_i\}_{1 \leq i \leq k} \sqcup \{\alpha_1,\alpha_2,\alpha_1+\alpha_2\} \end{align*} $$

if $n-2=2k+1$ is odd.

Now let us prove the proposition. We may assume that ${\mathfrak h} \subset {\mathfrak g}$ is given by the subspace ${\mathbb R}^{m+2} = \left ({\mathbb R} e_{m+1} + \dotsb + {\mathbb R} e_n \right )^{\perp } \subset {\mathbb R}^{n+2}$ – that is, that ${\mathfrak h}=\mathfrak {so}(2,m)$ is embedded in ${\mathfrak g}=\mathfrak {so}(2,n)$ in a way that the $m+1, m+2, \dotsc , n$ rows and columns are zero. We will consider the cases n even and n odd separately.

First assume $n-2=2k$ is even. Then we have

$$ \begin{align*} \Phi({\mathfrak w}) = \{ \alpha_1\pm \eta_i\}_{1 \leq i \leq k} \sqcup \{\alpha_2 \pm \eta_i\}_{1 \leq i \leq k}\sqcup \{\alpha_1+\alpha_2\}. \end{align*} $$

For $w\in W^P$ of length $\leq m-2=2k-(n-m)$ , the set of roots $\Phi (w) \subset \Phi ({\mathfrak w})$ has cardinality $\leq 2k-(n-m)$ , so that $\Phi ^+-\Phi (w)$ contains at least $n-m$ elements which belong to $\{\alpha _2\pm \eta _i\}_{1\leq i \leq k}$ , which has cardinality $2k$ . Since $\Phi ^+-\Phi (w)$ is closed under $\dot +$ and $\alpha _1-\alpha _2 \in \Phi ^+-\Phi (w)$ , we may choose sets $I_+$ and $I_-$ in $\{1,\dotsc , k\}$ such that the following are true:

1. 1. $I_+$ and $I_-$ are disjoint and $\lvert I_+\sqcup I_-\rvert =n-m$ .

2. 2. For $i \in I_+$ , we have $\{\alpha _1+\eta _i, \alpha _2+\eta _i\} \subset \Phi ^+-\Phi (w)$ .

3. 3. For $i \in I_-$ , we have $\{\alpha _1-\eta _i, \alpha _2-\eta _i\} \subset \Phi ^+-\Phi (w)$ .

(These sets are not unique, but any choice suffices for our purposes.) Let ${\mathfrak h}'$ be the copy of $\mathfrak {so}(2,m)$ given by the embedding of the subspace

$$ \begin{align*} \left(\bigoplus_{i \in I_+} {\mathbb R} e_{2i+2} \oplus \bigoplus_{i \in I_-} {\mathbb R} e_{2i+1}\right)^{\perp} \subset {\mathbb R}^{n+2}; \end{align*} $$

that is, row and column $2i+2$ are zero for $i \in I_+$ and row and column $2i+1$ are zero for $i \in I_-$ . Then the restriction of the harmonic representative $e_w= \wedge _{\alpha \in \Phi (w)} e_{-\alpha }$ in formula (4.2) to ${\mathfrak n}_{H'}$ is nonzero, and equals (up to a nonzero scalar) the harmonic representative of a similar class in ${\mathfrak n}_{H'}$ . Since the subspace is conjugate to the subspace $\left ({\mathbb R} e_{m+1} + \dotsb + {\mathbb R} e_n \right )^{\perp }$ by an element $m \in M({\mathbb C}) = Spin(n-2,{\mathbb C})$ , we see that ${\mathfrak h}'$ is conjugate to ${\mathfrak h}$ by $m \in M({\mathbb C})$ , and the highest weight vector $e_w$ restricts nontrivially to $\textrm{Ad}\left (m^{-1}\right )({\mathfrak h})$ . This proves the proposition in this case.

Next assume $n-2=2k+1$ is odd. Then

$$ \begin{align*} \Phi({\mathfrak w}) = \{\alpha_1\pm \eta_i\}_{1\leq i \leq k}\sqcup \{\alpha_2 \pm \eta_i\}_{1 \leq i \leq k} \sqcup \{\alpha_1,\alpha_2,\alpha_1+\alpha_2\}. \end{align*} $$

For $w\in W^P$ of length $\leq m-2=2k+1-(n-m)$ , the set of roots $\Phi (w) \subset \Phi ({\mathfrak w})$ has cardinality $\leq 2k+1-(n-m)$ , so that $\Phi ^+-\Phi (w)$ contains at least $n-m$ elements from the set $\{\alpha _2\pm \eta _i:1\leq i \leq k\} \sqcup \{\alpha _2\}$ of cardinality $2k+1$ . As in the previous case, $\alpha _1-\alpha _2 \in \Phi ^+-\Phi (w)$ , and $\Phi (w)$ is closed under $\dot +$ , so we may choose (not necessarily unique) sets of indices $I_+$ and $I_-$ such that the following are true:

1. 1. $I_+$ and $I_-$ are disjoint, $\lvert I_+\sqcup I_-\rvert =n-m-1$ if $\alpha _2 \in \Phi ^+-\Phi (w)$ , and $\lvert I_+\sqcup I_-\rvert =n-m$ if $\alpha _2 \notin \Phi ^+-\Phi (w)$ .

2. 2. For $i \in I_+$ , we have $\{\alpha _1+\eta _i, \alpha _2+\eta _i\} \subset \Phi ^+-\Phi (w)$ .

3. 3. For $i \in I_-$ , we have $\{\alpha _1-\eta _i, \alpha _2-\eta _i\} \subset \Phi ^+-\Phi (w)$ .

As before, if $\alpha _2 \notin \Phi ^+-\Phi (w)$ we can define the subspace

$$ \begin{align*} \left(\bigoplus_{i \in I_+} {\mathbb R} e_{2i+2} \oplus \bigoplus_{i \in I_-} {\mathbb R} e_{2i+1}\right)^{\perp} \subset {\mathbb R}^{n+2} \end{align*} $$

of dimension m, and the harmonic representative $e_w$ restricts nontrivially to the corresponding ${\mathfrak h}' = \mathfrak {so}(2,m)$ in ${\mathfrak g}$ . If $\alpha _2 \in \Phi ^+-\Phi (w)$ , then one adds on ${\mathbb R} e_n$ to the subspace and $e_2$ restricts nontrivially to the corresponding ${\mathfrak h}' = \mathfrak {so}(2,m)$ . In either case, since ${\mathfrak h}'$ is conjugate to ${\mathfrak h}$ by $m \in M({\mathbb C})$ , we have proved the proposition. $\Box $

4.4 Proof of Proposition 4.1

For $E={\mathbb C}$ , the injectivity in degrees $i\leq c/2$ except in the exceptional case is immediate from the fact that ${\mathfrak n}^*$ is the natural representation of the factor $SO(d-1)$ of L, and so $\textrm{H}^i({\mathfrak n},{\mathbb C}) = \wedge ^i{\mathfrak n}^*$ are irreducible. This can be easily generalized to the case of general E, but we give a computational proof using Kostant’s theorem, as we will have to verify slightly more.

We may assume ${\mathfrak g}=\mathfrak {so}(1,d) = \mathfrak {so}(J)$ , where

$$ \begin{align*} J= \begin{pmatrix} &&1\\ & I_{d-1}\\ 1 \end{pmatrix}. \end{align*} $$

Fix a Cartan subalgebra ${\mathfrak s}$ of $\mathfrak {so}(d-2)_{\mathbb C}$ and let ${\mathfrak t} \subset {\mathfrak g}_{\mathbb C}$ be the Cartan subalgebra defined by

$$ \begin{align*} {\mathfrak t} := \{diag(a,C,-a): a,d \in {\mathbb C}, C \in {\mathfrak s}\}. \end{align*} $$

Then ${\mathfrak t}$ is defined and maximally split over ${\mathbb R}$ , and the subspace ${\mathfrak a}_{\mathbb C} \subset \mathfrak {t}$ given by $C=0$ is the complexification of the Lie algebra ${\mathfrak a} \subset {\mathfrak g}$ of a maximal ${\mathbb R}$ -split subspace in ${\mathfrak g}$ . Let $\alpha \in \mathfrak {t}^*$ be defined by

(4.7) $$ \begin{align} \alpha(diag(a,C,-a))=a. \end{align} $$

We choose for ${\mathfrak s}$ the same Cartan subalgebra of block-diagonal matrices in $\mathfrak {so}(d-1)_{\mathbb C}$ specified earlier in equations (4.4) and (4.5) and use the same roots $\eta _i$ and the same positive system used there. An explicit computation shows that the positive roots are

$$ \begin{align*} \Phi^+({\mathfrak t},{\mathfrak g}_{\mathbb C}) = \Phi^+(\mathfrak{s},\mathfrak{so}(d-1)_{\mathbb C}) \sqcup \{\alpha\pm \eta_i: 1\leq i \leq k\} \end{align*} $$

for $d-1=2k$ even and

$$ \begin{align*} \Phi^+({\mathfrak t},{\mathfrak g}_{\mathbb C}) = \Phi^+(\mathfrak{s},\mathfrak{so}(d-1)_{\mathbb C}) \sqcup \{\alpha\pm \eta_i: 1 \leq i \leq k\} \sqcup \{\alpha\} \end{align*} $$

for $d-1=2k+1$ odd. We also have

$$ \begin{align*} \Phi({\mathfrak n}) = \{\alpha\pm \eta_i: 1\leq i \leq k\} \end{align*} $$

in the case $d-1=2k$ even and

$$ \begin{align*} \Phi({\mathfrak n}) = \{\alpha\pm \eta_i: 1\leq i \leq k\} \sqcup \{\alpha\} \end{align*} $$

in the case $d-1=2k+1$ odd.

We will list the relevant $w \in W^P$ and the sets $\Phi (w)$ . We will consider the even and odd cases separately.

First assume $d-1=2k$ is even. Let $\alpha _1,\alpha _2, \dotsc , \alpha _{k+1}$ be the set of simple roots of $\mathfrak {so}(1,d)_{\mathbb C}$ determined by the positive system already fixed – that is, $\alpha _1 := \alpha -\eta _1$ , $\alpha _i = \eta _{i-1}-\eta _i$ for $2\leq i \leq k$ , and $\alpha _{k+1}=\eta _{k-1}+\eta _k$ . The minimal length representatives in $W^P$ of length $\leq k$ are

$$ \begin{align*} \{s_0, s_1, \dotsc, s_k, t_k\}, \end{align*} $$

where $s_0:=1$ , $s_j := s_{\alpha _1} \dotsm s_{\alpha _j}$ for $ 1\leq j \leq k$ has length j, and $t_k = s_{k-1}s_{\alpha _{k+1}}$ has length k (compare [Reference Borel and Wallach17, Section VI.3.1]; as usual, $s_{\alpha _i}$ denotes the reflection in $\alpha _i$ ). The set $\Phi (w)$ is easily computed for these representatives: $\Phi (1)=\emptyset $ and $\Phi \left (s_j\right ) = \left \{\alpha _1, \alpha _1+\alpha _2, \dotsc , \alpha _1+\dotsb +\alpha _j\right \} = \left \{\alpha -\eta _1, \alpha _1-\eta _2, \dotsc , \alpha -\eta _j\right \}$ for $j\leq k$ , whereas $\Phi (t_k)=\{\alpha _1, \dotsc , \alpha _1+\dotsb +\alpha _{k-1}, \alpha _1+\dotsb +\alpha _{k-1}+\alpha _{k+1}\}= \{\alpha -\eta _1, \alpha _1-\eta _2, \dotsc , \alpha -\eta _{k-1}, \alpha +\eta _k\}$ .

Now suppose $d-1=2k+1$ is odd. Let $\alpha _1, \alpha _2, \dotsc , \alpha _{k+1}$ be the simple roots of $\mathfrak {so}(1,d)_{\mathbb C}$ determined by the positive system – that is, $\alpha _1 := \alpha -\eta _1$ , $\alpha _i = \eta _{i-1}-\eta _i$ for $2\leq i \leq k$ , and $\alpha _{k+1}=\eta _k$ . The minimal length representatives in $W^P$ of length $\leq k$ are

$$ \begin{align*} \{s_0, s_1, \dotsc, s_k\}, \end{align*} $$

where $s_0:=1$ , and $s_j := s_{\alpha _1} \dotsm s_{\alpha _j}$ for $ 1\leq j \leq k$ has length j (compare [Reference Borel and Wallach17, Section VI.4.4], where this set is denoted ${}^PW$ ). The set $\Phi \left (s_j\right )$ has the same description for $j\leq k$ as before.

Now consider the setup of the proposition. We have ${\mathfrak h}=\mathfrak {so}(1,c)$ for $c \leq d-1$ , embedded in the standard way – that is, using the subspace ${\mathbb R}^{c+1} \subset {\mathbb R}^{d+1}$ spanned by $e_1, e_2, \dotsc , e_c, e_{d+1}$ . To show that $\textrm{Res}_{\mathfrak n}$ is injective in a given degree i, it will suffice to show that the harmonic representative (i.e., L-highest weight vector) $e_w \otimes v_{w\lambda }$ restricts nontrivially in $\textrm{H}^i({\mathfrak n}_H,E_H)$ for each $w \in W^P$ of length i. In the case at hand, for $j<c/2$ there is a unique element in $W^P$ of length j, namely $s_j$ . The L-highest weight vector $e_{s_j} \otimes v_{s_j\lambda } = \bigwedge _{1\leq i \leq j} e_{-\left (\alpha -\eta _i\right )} \otimes v_{s_j\lambda }$ maps in $\wedge ^j{\mathfrak n}^*_H \otimes E_H$ to (a nonzero multiple of) the harmonic representative $\bigwedge _{1\leq i \leq j} e_{-\left (\alpha -\eta _i\right )}^H \otimes v^H_{s^H_j\lambda _H}$ , where $v^H_{s_j^H\lambda _H}$ is the $s_j^H(\lambda _H)$ -weight vector of $E_H$ , and hence is nonvanishing in cohomology. This proves that $\textrm{Res}_{\mathfrak n}(e_w \otimes v_{w\lambda }) \neq 0$ , and hence that $\textrm{Res}_{\mathfrak n}$ is injective for $j<c/2$ . The same proof works if $j\leq c/2$ , as long as we are not in the exceptional case $(d,c,i)=(2k+1,2k,k)$ .

In the remaining case we are considering $\textrm{Res}_{\mathfrak n}$ on $\textrm{H}^k({\mathfrak n},E)$ for $d=2k+1,c=2k,i=k$ . In this case there are two L-irreducible summands, with highest weight vectors $e_{s_k} \otimes v_{s_k\lambda }$ and $e_{t_k}\otimes v_{t_k\lambda }$ , respectively. In the embedding $\mathfrak {so}(2k+1,{\mathbb C})\subset \mathfrak {so}(2k+2,{\mathbb C})$ the weight space ${\mathfrak h}^{-\alpha _{k}}$ is embedded diagonally in the weight spaces ${\mathfrak g}^{-\alpha _k}$ and ${\mathfrak g}^{-\alpha _{k+1}}$ . Under the restriction from T to $T_H$ we have $\alpha _k\rvert _{T_H}=\alpha _{k+1}\rvert _{T_H} = \eta _{k-1}^H=\alpha ^H_{k}$ . Thus the vector $e_{s_k} \otimes v_{s_k\lambda } = \bigwedge _{1\leq i \leq k} e_{-\left (\alpha -\eta _i\right )} \otimes v_{s_k\lambda }$ goes to (a nonzero multiple of) the vector $\bigwedge _{1\leq i \leq k-1} e_{-\left (\alpha -\eta _i\right )}^H \wedge e_{-\eta _{k-1}}^H \otimes v^H_{s_k\lambda }$ , which is nonzero since $\eta _{k-1}$ is not one of $\alpha -\eta _i$ , $i\leq k-1$ . A similar argument applies to $e_{t_k}\otimes v_{t_k\lambda }$ . $\Box $

4.5 Proof of Proposition 4.2

This was proved for $E={\mathbb C}$ in [Reference Nair36, §1.6], and the elements of $W^P$ are explicitly listed there. The proof extends to general coefficients exactly as in the previous proof. $\Box $

4.6 Proof of Proposition 4.4

A tedious computational proof is possible, but we will argue differently. Recall the notation $P=MW$ for the parabolic in $SU(1,n)$ and $P_H= P\cap H = LN$ for the parabolic in $SO(1,n)$ . For $k<n$ we have a diagram

(4.8)

Here, as in §2.2, ${\mathfrak w}, {\mathfrak v}, \mathfrak {u}$ are the Lie algebras of the real points of $W({\mathbb R}), V({\mathbb R}), U({\mathbb R})$ . The second vertical map is induced by ${\mathfrak n} \subset {\mathfrak w}$ . The first vertical map is induced by the identification of ${\mathfrak v}$ with the $\chi $ -eigenspace for A in ${\mathfrak w}$ ; since $A \subset P_H$ and it acts by $\chi $ on $\mathfrak {u}$ , we have ${\mathfrak n} \subset {\mathfrak v}$ (compare §2.2 for notation). The top row comes from the long exact sequence for the boundary divisor in the toroidal compactification, and is exact in degrees $k<n$ and is a sequence of Hodge structures (see [Reference Nair36, Lemma 1.3]). The bigrading on $\wedge ^k{\mathfrak v}^*_{\mathbb C}$ comes from ${\mathfrak v}^*_{\mathbb C}=\left ({\mathfrak v}^*_{\mathbb C}\right )^{1,0}+\left ({\mathfrak v}^*_{\mathbb C}\right )^{0,1}$ , given by the complex structure on ${\mathfrak v}={\mathfrak w}/\mathfrak {u}$ (itself given by the central $U(1)\subset M({\mathbb R})$ ), and the $(k,0)$ -subspace $\wedge ^k \left ({\mathfrak v}^*_{\mathbb C}\right )^{1,0}$ maps isomorphically onto $\textrm{H}^{k,0}({\mathfrak w},{\mathbb C})$ . This is because the first term in the sequence has Hodge types $(k-1,1), \dotsc , (1,k-1)$ , because $\mathfrak {u}^*$ amounts to a Tate twist (see the proof of [Reference Nair36, Lemma 1.3]). Now the composition $\left ({\mathfrak v}^*_{\mathbb C}\right )^{1,0} \hookrightarrow {\mathfrak v}^*_{\mathbb C} \to {\mathfrak n}^*_{\mathbb C}$ is an isomorphism, since the kernel of ${\mathfrak v}^*_{\mathbb C} \to {\mathfrak n}^*_{\mathbb C}$ is the complexification of a real subspace, and hence does not meet $\left ({\mathfrak v}^*_{\mathbb C}\right )^{1,0}$ . Thus $\wedge ^k \left ({\mathfrak v}^*_{\mathbb C}\right )^{1,0} \to \wedge ^k {\mathfrak n}^*_{\mathbb C}$ is nonzero, and since $\textrm{H}^{k,0}({\mathfrak w},{\mathbb C})$ is M-irreducible (see [Reference Nair36, Remark 1.11]), $\textrm{Res}_{\mathfrak w}$ is injective on $\textrm{H}^{k,0}({\mathfrak w}_{\mathbb C},{\mathbb C})$ for $k<n$ . $\Box $