2.1. General notation

Throughout the paper F is a field of characteristic $0$ with algebraic closure $\bar F$ . (From §4 onward, F will be a number field.)

We denote by $\mathbf {G}_m$ the multiplicative group.

Whenever a group G acts on a set A, we denote by $A^G$ the set of fixed points of G in A.

We denote by $X_*(T)$ the lattice of F-rational cocharacters of an F-torus T.

For a linear algebraic group G over F, we use the following notation.

• • $G^\circ $ —the identity component of G.

• • $Z(G)$ —the center of G.

• • $G^{\operatorname {der}}$ —the derived group of G.

• • $G^{\operatorname {ab}}=G/G^{\operatorname {der}}$ —the abelianization of G.

• • $N_{G}$ —the unipotent radical of G.

• • $G^{\operatorname {lev}}=G/N_{G}$ —the Levi part of G, so that $G^{\operatorname {red}}:=(G^{\operatorname {lev}})^{\circ }=(G^\circ )^{\operatorname {lev}}$ is reductive.

• • $G^{\operatorname {ss}}=(G^{\operatorname {red}})^{\operatorname {der}}$ —the semisimple part of $G^\circ $ .

• • $G^{\operatorname {tor}}=(G^{\operatorname {red}})^{\operatorname {ab}}$ —the largest toric quotient of $G^\circ $ .

• • $G^{\operatorname {ssu}}=\operatorname {Ker} (G^\circ \rightarrow G^{\operatorname {tor}})$ —an extension of $G^{\operatorname {ss}}$ by $N_{G}$ .

• • —the Lie algebra of G (and similarly for other groups).

• • $\operatorname {Ad}=\operatorname {Ad}_G$ —the adjoint representation of G on .

• • $X^*(G)=X^*(G^{\operatorname {ab}})=X^*(G^{\operatorname {lev}})$ —the finitely generated abelian group of F-rational characters of G.

• • $\Delta _G=\det \operatorname {Ad}\in X^*(G)$ —the modular character of G.

• • $T_G$ —the maximal F-split torus of $Z(G^{\operatorname {red}})$ .

• • $\mathfrak {a}_G=X_*(T_G)\otimes _{\mathbb {Z}}\mathbb {R}$ —the $\mathbb {R}$ -vector space generated by $X_*(T_G)$ .

We often view $G^{\operatorname {lev}}$ as an F-subgroup (the Levi subgroup) of G, uniquely defined up to conjugation by $N_{G}(F)$ ([Reference Mostow27] or [Reference Hochschild15, Chapter VIII, Theorem 4.3]).

If G is connected, then $X^*(G)=X^*(G^{\operatorname {tor}})$ is torsion-free, i.e., it is a lattice.

If G is reductive, then $G^{\operatorname {ssu}}=G^{\operatorname {ss}}=G^{\operatorname {der}}$ .

If T is an F-split torus, then $X^*(T)$ can be identified with the dual lattice of $X_*(T)$ . In general, if $T'$ is the maximal F-split subtorus of T, then we can identify $X_*(T)$ with $X_*(T')$ , while the restriction map $X^*(T)\rightarrow X^*(T')$ is injective with finite cokernel. Thus, we have isomorphisms

$$\begin{align*}X^*(T)\otimes_{\mathbb{Z}}\mathbb{R}\simeq X^*(T')\otimes_{\mathbb{Z}}\mathbb{R}\simeq \mathfrak{a}_{T'}^*= \mathfrak{a}_T^*. \end{align*}$$

The F-split rank of T, i.e. the dimension of $T'$ , is the rank of $X^*(T)$ (and $X_*(T)$ ).

More generally, if G is connected, then the restriction map $X^*(G)=X^*(G^{\operatorname {red}})\rightarrow X^*(T_G)$ is injective with finite cokernel and hence yields an isomorphism

$$\begin{align*}X^*(G)\otimes_{\mathbb{Z}}\mathbb{R}\simeq X^*(T_G)\otimes\mathbb{R}=\mathfrak{a}_G^*. \end{align*}$$

For any finite subset A of a real vector space we denote by $\mathbb {R}_{\ge 0}A$ the polyhedral cone generated by A, i.e., the set of linear combinations $\sum _{a\in A}x_aa$ , where $x_a\ge 0$ for all $a\in A$ . Similarly, we use the notation $\mathbb {R}_{>0}A$ , $\mathbb {Z}_{\ge 0}A$ , $\mathbb {Z}_{>0}A$ where we require the $x_a$ to be strictly positive (resp., nonnegative integers, strictly positive integers) for all a.

Let G be an algebraic F-group and let H be an F-subgroup of G. We record a few basic results.

Proof. The first part follows from the fact that the homomorphism $H/[G,H]\rightarrow G^{\operatorname {ab}}$ has finite kernel and cokernel. In fact we have (a $\operatorname {Gal}(\bar F/F)$ -equivariant) exact sequence

$$\begin{align*}H_2(\Gamma,\mathbb{Z})\rightarrow H/[G,H]\rightarrow G^{\operatorname{ab}}\rightarrow\Gamma^{\operatorname{ab}}\rightarrow1, \end{align*}$$

[Reference Hilton and Stammbach18, Corollary VI.8.2]. The second part follows from the first part and the fact that if a finite group $\Gamma $ acts nontrivially on a lattice L, then $L^\Gamma $ is of infinite index in L.

We denote the Picard group of a variety X by $\operatorname {Pic} X=H^1(X,\mathcal {O}_X^*)$ . If X is smooth, then we can identify $\operatorname {Pic} X$ with the (Weil) divisor class group $\operatorname {Cl} X$ of X. (See [Reference Hartshorne14, §II.6] for basic facts about divisors and the Picard group.)

Lemma 2.3 ([Reference Raynaud33], Proposition VII.1.5; cf. also the proof of [Reference Popov31], Theorem 4)

Suppose that G is connected. Then, there is an exact sequence

$$\begin{align*}X^*(G)\rightarrow X^*(H)\rightarrow \operatorname{Pic} (G/H). \end{align*}$$

2.2

Recall that a prehomogeneous vector space (PVS) over F is a finite-dimensional F-representation

$$\begin{align*}\rho:G\rightarrow\operatorname{GL}(V), \end{align*}$$

of a connected linear algebraic F-group G on a finite-dimensional F-vector space V, for which there exists a (necessarily unique) open orbit X. Note that X is defined over F and $X(F)$ is $G(F)$ -invariant. If F is algebraically closed, then $X(F)$ is a $G(F)$ -orbit and the stabilizers $G_v$ , $v\in X(F)$ are conjugate by $G(F)$ . In general, given $u\in X(F)$ , the $G(F)$ -orbits in $X(F)$ are in bijection with the elements of the kernel of $H^1(F,G_u)\rightarrow H^1(F,G)$ (a map of pointed sets) [Reference Serre42, I §5.4, Corollary 1 of Proposition 36]. Correspondingly, the F-groups $G_v$ , $v\in X(F)$ are pure inner forms of $G_u$ .

Clearly, a quotient of a PVS by an invariant subspace is also a PVS.

We will write $H=G_v$ (the “generic stabilizer”) when $v\in X(F)$ is immaterial, e.g., when we refer to the group $X^*(H)$ or to properties pertaining to $H(\bar F)$ . Thus, we can identify X with $G/H$ . The F-subgroup ${G}^\star =G^{\operatorname {der}} H$ of G is well defined. Note that in general, H may be disconnected and the groups $G_v^\circ $ , $v\in X(F)$ are not necessarily in the same inner class. In fact, the lattice $X^*(G_v^\circ )$ may depend on the choice of $v\in X(F)$ . For this reason, we will refrain from using the notation $X^*(H^\circ )$ .

For $v\in X(F)$ , the linear map

(2.1)

is surjective, since it is the differential of the dominant map $G\rightarrow V$ , $g\mapsto \rho (g)v$ . The kernel of $D_v$ is the Lie algebra of $G_v$ [Reference Borel2, Lemma II.7.4]. We thus get a $G_v$ -equivariant isomorphism . In particular,

(2.2) $$ \begin{align} \Delta_G\det\rho^{-1}\big|_H=\Delta_H. \end{align} $$

A relative invariant with respect to a character $\chi \in X^*(G)$ is an F-rational function $f\not \equiv 0$ on V such that $f\circ \rho (g)=\chi (g)f$ for all $g\in G$ . We will write $f=\operatorname {RI}_\chi $ .

The following basic properties hold.

1. 1. $\operatorname {RI}_\chi $ , if it exists, is unique up to a nonzero scalar and it is regular and nonvanishing on X.

2. 2. The sublattice $X^*(G/{G}^\star )$ , which is the kernel of the restriction map $X^*(G)\rightarrow X^*(H)$ , consists of the characters with respect to which there exists a relative invariant.

3. 3. Let $\chi _1,\dots ,\chi _r$ be a basis of $X^*(G/{G}^\star )$ . Then $\operatorname {RI}_{\chi _1},\dots ,\operatorname {RI}_{\chi _r}$ are algebraically independent.

4. 4. The F-irreducible polynomials that are nowhere vanishing on X (if any) are relative invariants. They are called the fundamental relative invariants. The corresponding characters are called fundamental characters. They form a basis for $X^*(G/{G}^\star )$ . Denote the set of all fundamental characters by $\mathfrak {X}$ .

If G is reductive, then the following conditions are equivalent.

1. 1. $H^\circ $ is reductive.

2. 2. X is the complement of a hypersurface.

3. 3. X is the complement of the hypersurface defined by $\operatorname {RI}_{\chi _1}\cdots \operatorname {RI}_{\chi _r}$ where $\mathfrak {X}=\{\chi _1,\dots ,\chi _r\}$ .

4. 4. X is an affine variety.

5. 5. V is regular (see [Reference Kimura23, Definition 2.14]).

In this case, $(\det \rho )^2\in X^*(G/{G}^\star )$ . (As far as we know, it is unknown whether $\operatorname {RI}_{\det \rho ^2}$ is a polynomial in general.)

We give some more basic properties of PVSs.

Lemma 2.4. There is a short exact sequence

$$\begin{align*}0 \rightarrow X^*(G/{G}^\star) \rightarrow X^* (G)\rightarrow X^* (H) \rightarrow 0. \end{align*}$$

Thus, $\operatorname {rk} X^*(G)-r=\operatorname {rk} X^*(H)$ .

Let $\overline {\mathfrak {a}}_G^*$ be the subspace $X^*(G/{G}^\star )\otimes \mathbb {R}$ of $\mathfrak {a}_G^*$ . Thus, $\mathfrak {X}$ forms a basis for $\overline {\mathfrak {a}}_G^*$ .

For completeness, we also note the following fact, which can also be proved by a topological argument for $F=\mathbb {C}$ .

Proof. We may assume that F is algebraically closed. The condition on V means that $V\setminus X$ has codimension $>1$ in V. It follows from purity of the branch locus that X is simply connected [28, Exp. X, Corollaire 3.3], i.e., there is no nontrivial étale covering of X. On the other hand,

$$\begin{align*}G/H^\circ\rightarrow X, \end{align*}$$

is an étale covering. Thus, H is connected.

By Lemma 2.6, if H is connected, then V is nonisotropic.

The basic example of an isotropic PVS is the space V of binary quadratic forms with respect to the action of $\operatorname {GL}_2$ . Here, H is an orthogonal group in dimension $2$ ; $v\in X(F)$ is isotropic if and only it is isotropic in the usual sense of quadratic forms. We caution however, that the PVS of quadratic forms in $n>2$ variables (with respect to the $\operatorname {GL}_n$ -action) is nonisotropic.

In general, it is possible that all elements of $X(F)$ are isotropic (for instance, if V is isotropic and $F=\bar F$ ), although we expect that this is never the case if F is a number field.

Assume now that G is reductive, so that every representation of G is completely reducible. We single out two important classes of PVSs in the opposite extremes.

Note that every basic PVS is regular, since any connected nonreductive F-subgroup of G is contained in a proper parabolic F-subgroup of G [Reference Borel and Tits4]. Moreover, a basic PVS is necessarily nonisotropic. It is also clear that every subrepresentation of a basic PVS is also basic.

A distinguished PVS is regular and nonisotropic.

3.1

Let $G'$ be a reductive group over F and let $\lambda :\mathbf {G}_m\rightarrow G'$ be a one-parameter subgroup. Let . Decompose

This is a $\mathbb {Z}$ -gradation of . (If $G'$ is semisimple and adjoint, then every gradation of is of this form.) Let , $i\in \mathbb {Z}$ be the corresponding (decreasing) filtration of . The stabilizer P of $\mathcal {F}$ in G under $\operatorname {Ad}$ is the parabolic F-subgroup of G whose Lie algebra is $\mathcal {F}_0$ . The Lie algebra of the unipotent radical of P is $\mathcal {F}_1$ . The centralizer G of $\lambda $ is a Levi subgroup of P whose Lie algebra is . For every $i\ne 0$ , the vector space , which consists of nilpotent elements, comprises finitely many (geometric) orbits with respect to the adjoint action of G [Reference Vinberg49]. In particular, it is a PVS. In the terminology of [Reference Gyoja13], these are PVSs of V-type (for Vinberg). They are not necessarily regular. We also say that $\mathcal {F}$ is a filtration of of V-type. (Note that we may restrict here to the case $i=1$ , since the general case follows from it by considering the graded Lie subalgebra , which is reductive.)

As a special case, let P be a parabolic subgroup of $G'$ with Levi factor G and . Then, is a PVS with respect to the adjoint action of G. In fact, is a PVS with respect to the adjoint action of P by a well-known result of Richardson. We say that V is a PVS of R-type. We denote the nilpotent orbit containing the open orbit of P on (the Richardson orbit with respect to P) by $\operatorname {Rich}(P)$ . The number of irreducible components of V is the F-corank of P in G.

Other important special cases, this time of regular PVSs, arise from the Dynkin–Kostant classification of nilpotent orbits. In more detail, let $(e,h,f)$ be an -triple in . (By the Jacobson–Morozov theorem, every nilpotent is a part of an -triple.) It corresponds to an F-homomorphism $\operatorname {SL}_2\rightarrow G^{\prime \operatorname {der}}$ . Let be the grading induced by h. The corresponding (decreasing) filtration , $i\in \mathbb {Z}$ of depends only on e and in fact only on the P-orbit of e, where $P=P(e)$ is the stabilizer of $\mathcal {F}=(\mathcal {F}_i)_{i\in \mathbb {Z}}$ in $G'$ under the adjoint action. Recall that P is the parabolic subgroup of $G'$ with Lie algebra $\mathcal {F}_0$ . The Lie algebra of the unipotent radical of P is $\mathcal {F}_1$ . (That $\mathcal {F}$ depends only on e follows from the fact that two -triples $(e,h,f)$ and $(e,h',f')$ with the same first element e are conjugate by an element of $N_P(F) \cap G^{\prime }_e (F)$ [Reference Bourbaki3, Lemma VIII.11.1.4].)

Let G be the Levi quotient of P, so that . Let with the G-action $\rho $ induced by the adjoint representation. Let $\mathfrak {o}$ be the geometric $G'$ -orbit of e and let $\mathfrak {o}_{\mathcal {F}}=\mathfrak {o}\cap \mathcal {F}_2$ .

These are the PVSs of DK-type (Dynkin–Kostant) in the language of [Reference Gyoja13]. (We quickly sketch how to derive these assertions from the literature. Note first that is the open G-orbit X of V by [Reference Bourbaki3, Proposition VIII.11.3.6]. V is regular by [Reference Kac22], see below. In addition, the $N_P$ -orbit of e is $e + \mathcal {F}_3$ [Reference Ranga Rao34, Lemma 1]. This shows the second and third assertions. The fourth assertion easily reduces to the special case $G^{\prime }_e\subset P$ . Let $g \in G^{\prime }_e$ . Then $(e, \operatorname {Ad} (g) h, \operatorname {Ad} (g) f)$ is an -triple with the same first element as $(e,h,f)$ , and therefore conjugate to $(e,h,f)$ by an element $n \in N_P \cap G^{\prime }_e$ . Since $ng$ centralizes h, we have $ng \in G$ , and therefore $g \in P$ .)

If we fix in addition to e the -triple $(e,h,f)$ , then we can identify G with the centralizer of h in $G'$ and V with the subspace of . We have then $X=V\cap \mathfrak {o}$ . We may identify with the dual of via the Killing form. Fixing a vector space isomorphism between and , $\operatorname {RI}_{(\det \rho )^2}$ is given by the determinant of the linear map , a polynomial in $e\in V$ whose nonzero locus is X [Reference Kac22, Proposition 1.1, 1.2, 2.1].

Fix a minimal parabolic F-subgroup $P_0'$ of $G'$ and a maximal split F-torus $T_0'$ contained in $P_0'$ . Then, we may choose $(e,h,f)$ in its $G'(F)$ -orbit so that is dominant and $P=P(e)$ is standard. This is the canonical parabolic subgroup pertaining to $\mathfrak {o}$ . G is then identified with the standard Levi subgroup of P.

An orbit $\mathfrak {o}$ is called even if for all odd i (or equivalently, if ). For even orbits clearly $\mathfrak {o}=\operatorname {Rich}(P)$ . A particular case is given by distinguished orbits, where the connected stabilizer $(G^{\prime }_e)^{\circ }$ , $e\in \mathfrak {o}$ is modulo the center of $G'$ a unipotent group. In this case, the PVS V is distinguished (and the converse holds for even orbits $\mathfrak {o}$ ). In general, not every standard parabolic subgroup is the canonical parabolic subgroup of some nilpotent orbit, let alone an even one. (By [Reference Kac22, Proposition 2.1] and [Reference Gyoja13, Theorem 2.4], the Richardson orbit of a standard parabolic subgroup P has P as its canonical parabolic subgroup if and only if the associated PVS is regular and for all $i \ge 1$ , where denotes the descending central series of .)

Any PVS of DK-type arises from an even nilpotent orbit for the reductive Lie algebra . Hence, any PVS of DK-type is also of R-type, although not necessarily with the same $G'$ .

3.2

Let us give some concrete examples of DK-type arising from even nilpotent orbits. For each case we provide the following data:

• • The numbering in the Sato–Kimura classification (appendix of [Reference Kimura23]) in the irreducible case,

• • the group $G'$ ,

• • the (even) nilpotent orbit $\mathfrak {o}$ in (either the corresponding partition for classical groups or the notation in the Bala–Carter classification),

• • the canonical Levi G,

• • the regular PVS ,

• • the representation $\rho $ of G on V,

• • the fundamental relative invariant polynomials (FRIPs),

• • the generic stabilizer H.

• • If V is basic, distinguished, or isotropic, we indicate that.

For simplicity we only consider split $G'$ .

In the following $m,n,k$ are positive integers (possibly with some restrictions). We denote by $\operatorname {Mat}_{m,n}$ the space of $m\times n$ matrices and by $\operatorname {Mat}_n$ (resp., $\operatorname {Sym}_n$ , $\operatorname {Skew}_n$ ) the spaces of square (resp., symmetric, skew-symmetric) $n\times n$ -matrices. We denote by $\operatorname {Pf}$ the Pfaffian of a skew-symmetric matrix.

Other examples of basic PVSs of DK-type are summarized in Table 1 which refers to the numbering in the list in the appendix of [Reference Kimura23] whenever $\rho $ is reduced. (See also [Reference Igusa21].)

Table 1 Irreducible basic PVSs of DK-type for exceptional groups

Here, $\pi $ is the embedding $G_2\hookrightarrow \operatorname {SO}_7\hookrightarrow \operatorname {GL}_7$ and $\phi $ is its lifting to an embedding $G_2\hookrightarrow \operatorname {Spin}_7$ . Also, $\sigma $ is the embedding of $F_4$ as the fixed point subgroup of the outer involution of $E_6$ . Finally,

$$ \begin{align*} \iota_1(x,y)&=(\psi_{7,3}(\phi(x),y),y),\\ \iota_2(x,y)&=(\psi_{6,4}(\operatorname{Sym}^3(y),(x,y)),\operatorname{Sym}^2(y)),\\ \iota_3(x,y)&=\psi_{7,7}(\phi(x),\phi(y)), \end{align*} $$

where $\psi _{m,n}$ is the homomorphism $\operatorname {Spin}_m\times \operatorname {Spin}_n\rightarrow \operatorname {Spin}_{m+n}$ and we identify $\operatorname {Spin}_3\simeq \operatorname {SL}_2$ , $\operatorname {Spin}_6\simeq \operatorname {SL}_4$ , $\operatorname {Spin}_4\simeq \operatorname {SL}_2\times \operatorname {SL}_2$ , $\operatorname {SO}_4\simeq (\operatorname {SL}_2\times \operatorname {SL}_2)/\{\pm 1\}$ .

3.3

Another interesting set of examples of irreducible PVSs of DK-type arises from the Freudenthal-Tits magic square (see Table 2). In these cases, G is a maximal Levi subgroup of $G'$ corresponding to a simple root $\alpha $ which is a leaf in the Dynkin diagram (it is the unique simple root which is not orthogonal to the highest root $\tilde \alpha $ of $G'$ ). The highest weight $\mu $ of $\rho $ is $\tilde \alpha -\alpha $ and its restriction to $G^{\operatorname {der}}$ is the fundamental weight corresponding to the simple root $\beta $ adjacent to $\alpha $ (and of the same length) in the Dynkin diagram. Let $Q=LN'$ be the maximal parabolic subgroup of $G^{\operatorname {der}}$ corresponding to $\beta $ . Then, there exists a nontrivial element $w_0$ in the normalizer of L in $G^{\operatorname {der}}$ . We have $\alpha =w_0\mu $ , which is the lowest weight of V. Clearly, L stabilizes the root spaces $V_\alpha $ and $V_\mu $ . Remarkably, any $v\in V_\alpha +V_\mu $ outside $V_\alpha \cup V_\mu $ is regular. We have $H^\circ =L^{\operatorname {der}}$ and $[H:H^\circ ]=2$ with H containing a representative of $w_0$ . In particular, V is not basic. Since as a representation of H, we see that the restriction of $\rho $ to H is $\operatorname {Ind}_{H^\circ }^H(\rho '\oplus 1)$ where $\rho '$ is the representation of L on . It turns out that $N'$ is abelian. Therefore, is a PVS and in fact it is basic: the generic stabilizer (whose identity component is denoted by K in the table) is the centralizer of (a suitable choice of) $w_0$ in L, i.e., the stabilizer of an outer involution of L corresponding to $w_0$ . See drawings below.

Table 2 PVSs of DK-type pertaining to the Freudenthal–Tits magic square

The fundamental relative invariant polynomials in these cases (which are of degree 4) are called Freudenthal quartics (see [Reference Freudenthal12], [Reference Igusa20], [Reference Saito37]). They arise by considering $\operatorname {ad} (x)^4$ , as a map between the one-dimensional spaces and (cf. [Reference Gyoja13, Table II]). The orbit structure of V is described in [Reference Röhrle32].

What is more, the Richardson orbit with respect to the maximal parabolic subgroup corresponding to $\beta $ is also even. These orbits give rise to the PVSs #8 (Example 3.8), #12 (Example 3.9), #9 and #28, respectively, in the list of the appendix of [Reference Kimura23]. They all have a fundamental relative invariant polynomial of degree 12, obtained as the determinant of , $x \in V$ , where is two-dimensional. Alternatively, the PVS mentioned above has a cubic fundamental relative invariant polynomial, and the PVS V associated to $\beta $ is the direct sum of and of its isomorphic image under the simple reflection $w_\alpha $ , which enables one to construct the fundamental relative invariant of V as the discriminant of the cubic form associated to pairs of elements of .

Sato and Kimura classified the irreducible PVSs over $\bar F$ up to a suitable notion of equivalence weaker than isomorphism [Reference Sato and Kimura46].

There are 29 essential cases of regular PVSs, listed in the appendix of [Reference Kimura23].Footnote ³ All these examples are either of DK-type or restrictions thereof to suitable subgroups.Footnote ⁴ Most of these examples are non-isotropic. The exceptions are

• • The symmetric square representation of $\operatorname {GL}_2$ (Example 3.4 for $k=1$ , $n_1=2$ ).

• • Example 3.9.

• • The representation $\operatorname {GL}_2\times \operatorname {SO}_{k+2}$ on $\operatorname {Mat}_{2,k+2}$ (Example 3.5 for $k=1$ , $n_1=2$ ).

• • The restriction of the latter to $\operatorname {GL}_2\times \operatorname {Spin}_7$ (resp., $\operatorname {GL}_2\times G_2$ ) for $k=6$ (resp., $k=5$ ) via the Spin representation $\operatorname {Spin}_7\hookrightarrow \operatorname {SO}_8$ (resp., the $7$ -dimensional representation $G_2\hookrightarrow \operatorname {SO}_7$ ).

The non-regular irreducible PVSs over $\bar F$ are (up to castling equivalence) mostly of R-type or restrictions thereof. The notable exception is the representation $\rho $ of $\operatorname {SL}_2\times \operatorname {SL}_n$ on $\operatorname {Hom}_F(F^2,\operatorname {Skew}_n)$ given by $\rho (g_1,g_2)A(\xi )=g_2A(\xi g_1)g_2^t$ , which is a PVS for odd n but not of R-type for $n>7$ . Another curious example is the restriction of the representation of $\operatorname {GL}_3\times \operatorname {Sp}_{2n}$ (of R-type) on $\operatorname {Mat}_{3,2n}$ , $n>1$ given by $\rho (g_1,g_2)A=g_1Ag_2^{-1}$ to the image of the symmetric square representation of $\operatorname {GL}_2$ times $\operatorname {Sp}_{2n}$ . This irreducible, non-regular PVS has a relative invariant.

5.1

Recall that G is a reductive group over a number field F. Fix a maximal F-split torus $T_0$ of G and a minimal parabolic F-subgroup $P_0$ of G containing $T_0$ . We have a Levi decomposition $P_0=M_0\ltimes N_0$ where $M_0$ is the centralizer of $T_0$ in G and $N_0=N_{P_0}$ , which is a maximal unipotent F-subgroup of G.

If P is a standard parabolic subgroup of G, then we denote by $M_P$ its standard Levi subgroup.

For simplicity we write $\mathfrak {a}_0=\mathfrak {a}_{P_0}=\mathfrak {a}_{M_0}\simeq \mathfrak {a}_{T_0}$ . Let

$$\begin{align*}H_0:M_0(\mathbb{A})\rightarrow\mathfrak{a}_0, \end{align*}$$

be the group homomorphism defined by

$$\begin{align*}e^{\left\langle{\chi},{H_0(m)}\right\rangle}=\left|{\chi(m)}\right|,\ \ m\in M_0(\mathbb{A}),\ \chi\in X^*(M_0). \end{align*}$$

Let $\Delta _0\subset X^*(T_0)$ be the set of simple roots of $T_0$ on $\operatorname {Lie} N_0$ .

Fix a maximal compact subgroup K of $G(\mathbb {A})$ which is in good position with respect to $P_0$ .

For any $g\in G(\mathbb {A})$ , we write $m_0(g)\in M_0(\mathbb {A})/(M_0(\mathbb {A})\cap K)$ for the $M_0(\mathbb {A})$ -part in the Iwasawa decomposition $G(\mathbb {A})=N_0(\mathbb {A})M_0(\mathbb {A})K$ of g.

We extend $H_0$ to a left- $N_0(\mathbb {A})$ and right-K-invariant function on $G(\mathbb {A})$ . Thus,

$$\begin{align*}H_0(g)=H_0(m_0(g)). \end{align*}$$

Fix a Siegel set $\mathfrak {S}$ in $G(\mathbb {A})$ of the form

$$\begin{align*}\mathfrak{S}=\{g\in G(\mathbb{A})\mid\left\langle{\alpha},{H_0(g)}\right\rangle>c_0\ \forall\alpha\in\Delta_0\}, \end{align*}$$

for some fixed constant $c_0$ . Thus,

1. 1. $\mathfrak {S}$ is left $P_0(F)N_0(\mathbb {A})$ -invariant.

2. 2. For any $g\in G(\mathbb {A})$ , the set $\{\gamma \in P_0(F)\backslash G(F)\mid \gamma g\in \mathfrak {S}\}$ is finite.

3. 3. There exists a compact subset $\Omega \subset G(\mathbb {A})$ such that

   (5.1) $$ \begin{align} \mathfrak{S}\subset P_0(F)(T_0(\mathbb{A})\cap\mathfrak{S})\Omega. \end{align} $$

4. 4. $G(F)\mathfrak {S}=G(\mathbb {A})$ provided that $c_0$ is sufficiently negative.

Let $\delta _{P_0}$ be the modulus function of $P_0(\mathbb {A})$ .

5.2

Recall that V is a PVS. (For now, we do not assume that V is regular.) The first order of business is the following result, which will be proved in the rest of this section.

Proposition 5.1. There exists a continuous seminorm $\nu $ on $C_{\operatorname {rd}}(V(\mathbb {A}))$ such that

$$\begin{align*}\left|{\theta_\phi(g)}\right|\le\delta_{P_0}(m_0(g))\left|{\det\rho(g)}\right|\nu(\phi), \end{align*}$$

for all $\phi \in C_{\operatorname {rd}}(V(\mathbb {A}))$ and $g\in \mathfrak {S}$ .

Note that $\left |{\det \rho }\right |$ is the modulus function for the action of $G(\mathbb {A})$ on $V(\mathbb {A})$ , that is,

$$\begin{align*}\int_{V(\mathbb{A})}f(\rho(g)^{-1}v)\ dv=\left|{\det\rho(g)}\right|\int_{V(\mathbb{A})}f(v)\ dv, \end{align*}$$

for any $f\in L^1(V(\mathbb {A}))$ and $g\in G(\mathbb {A})$ .

5.3

Let $\Phi _G$ be the set of roots of $T_0$ on G and let $\Psi _V$ be the set of weights of $T_0$ on V. The set $\Phi _G$ is a (not necessarily reduced) root system and the co-roots are in $X_*(T_0)$ . We decompose and V according to the roots and weights:

(5.2a)

(5.2b) $$ \begin{align} V&=\oplus_{\beta\in\Psi_V}V_\beta. \end{align} $$

The vector spaces , $\alpha \in \Phi _G$ , are not necessarily one-dimensional unless G is split. Moreover, is not necessarily a commutative Lie subalgebra (namely, if $2\alpha \in \Phi _G$ ). The spaces and $V_\beta $ are invariant under $M_0$ . We write

$$\begin{align*}n_\beta=\dim V_\beta,\ \ \beta\in\Psi_V, \end{align*}$$

and

$$\begin{align*}\delta_V=\det\rho\big|_{T_0}=\sum_{\beta\in\Psi_V}n_\beta\beta\in X^*(T_0). \end{align*}$$

By convention, $V_\gamma =0$ if $\gamma \notin \Psi _V$ . Note that

(5.3)

where is the Lie algebra representation obtained by differentiating $\rho $ .

Let

where $\Phi _G^+\subset \Phi _G$ is the subset of positive roots with respect to $P_0$ , so that

$$\begin{align*}\delta_{P_0}(m)=e^{\left\langle{\delta_0},{H_0(m)}\right\rangle},\ \ m\in M_0(\mathbb{A}). \end{align*}$$

We enumerate the roots $\Phi _G\cup \{0\}$ (including multiplicities) as $\alpha _i$ , $i\in \mathcal {I}_G$ for some index set $\mathcal {I}_G$ , and fix a basis $\mathcal {B}_G=(x_i)_{i\in \mathcal {I}_G}$ for such that for all i. Similarly, we enumerate the weights $\Psi _V$ including multiplicities as $\beta _j$ , $j\in \mathcal {J}_V$ , and fix a basis $\mathcal {B}_V=(y_j)_{j\in \mathcal {J}_V}$ for V such that $y_j\in V_{\beta _j}$ for all j.

Let $\Gamma (V)$ be the (finite) set of subspaces of V that are sums of weight subspaces. Thus, every $U\in \Gamma (V)$ is of the form

$$\begin{align*}U=\oplus_{\beta\in\Psi_U}V_\beta, \end{align*}$$

where $\Psi _U$ is a subset of $\Psi _V$ . We will also write

(5.4) $$ \begin{align} \mathcal{J}_{V/U}=\{j\in\mathcal{J}_V\mid\beta_j\notin\Psi_U\}. \end{align} $$

Consider the set $\Gamma _{\operatorname {reg}}(V)$ of subspaces in $\Gamma (V)$ that contain a regular element:

$$\begin{align*}\Gamma_{\operatorname{reg}}(V)=\{U\in\Gamma(V)\mid U\cap X\ne\emptyset\}. \end{align*}$$

A connected algebraic F-subgroup R of G is called $T_0$ -saturated if its Lie algebra is of the form for some subset S of $\Phi _G\cup \{0\}$ . In this case, we write $\Phi _R = S \setminus \{0\}$ . Examples of $T_0$ -saturated subgroups are parabolic F-subgroups containing $T_0$ and their unipotent radicals, and semistandard Levi subgroups. If R is $T_0$ -saturated, then we write

(5.5)

For any subspace $U\subset V$ denote by $\operatorname {Stab}(U)=\{g\in G\mid \rho (g)U=U\}$ the stabilizer of U. The Lie algebra of $\operatorname {Stab}(U)$ is [Reference Borel2, Lemma II.7.4].

Let $U\in \Gamma (V)$ and let R be a $T_0$ -saturated subgroup of G. We denote by $R\star U$ the smallest R-stable space in $\Gamma (V)$ containing U.

The following is an immediate consequence of (5.3).

5.4

For any $U\in \Gamma (V)$ define the gauge

(5.6) $$ \begin{align} \lambda(U)=\delta_0+\sum_{\beta\in\Psi_V\setminus\Psi_U}n_\beta\beta=\delta_0+\delta_V-\sum_{\beta\in\Psi_U}n_\beta\beta\in X^*(T_0). \end{align} $$

It will turn out to be a key parameter governing the growth of the subsum of $\theta _\phi $ pertaining to U.

The following combinatorial-geometric lemma is crucial.

Lemma 5.6. Let $U\in \Gamma _{\operatorname {reg}}(V)$ . Then, there exists a one-to-one function

$$\begin{align*}\iota:\mathcal{J}_V\rightarrow\mathcal{I}_G, \end{align*}$$

such that

$$\begin{align*}\beta_j+\alpha_{\iota(j)}\in\Psi_U, \end{align*}$$

for all $j\in \mathcal {J}_V$ .

Note that we used that G is reductive in the proof, namely that $-\Phi _G=\Phi _G$ .

Corollary 5.7. Let $U\in \Gamma _{\operatorname {reg}}(V)$ . Then, for any $J\subset \mathcal {J}_V$ we have

$$\begin{align*}\delta_0+\sum_{j\in J}\beta_j\in\mathbb{R}_{\ge0}(\Psi_U\cup\Delta_0). \end{align*}$$

In particular, we have

(5.7) $$ \begin{align} \lambda(U)\in\mathbb{R}_{\ge0}(\Psi_U\cup\Delta_0). \end{align} $$

Proof. We write

$$\begin{align*}\delta_0+\sum_{j\in J}\beta_j=\delta_0-\sum_{j\in J}\alpha_{\iota(j)}+\sum_{j\in J} (\beta_j+\alpha_{\iota(j)}), \end{align*}$$

and observe that $\delta _0-\sum _{j\in J}\alpha _{\iota (j)}\in \mathbb {R}_{\ge 0}\Delta _0$ since $\iota $ is injective.

Taking $J=\mathcal {J}_{V/U}$ (cf. (5.4) and (5.6)) we obtain (5.7).

5.5

The proof of Proposition 5.1 is based on a standard bound for lattice sums.

For any $\xi \in V$ denote by $\xi _\beta \in V_\beta $ , $\beta \in \Psi _V$ the components of $\xi $ with respect to the decomposition (5.2b).

We write $x_\pm =\max (0,\pm x)$ for the positive and negative parts of a real number x, so that $x=x_+-x_-$ .

For simplicity, we write $A(\cdot ,\phi )\ll _{N,\phi }^{\operatorname {rd}} B_N(\cdot )$ if for every $N\ge 0$ there exists a continuous seminorm $\nu _N$ on $C_{\operatorname {rd}}(V(\mathbb {A}))$ such that $\left |{A(\cdot ,\phi )}\right |\le B_N(\cdot )\nu _N(\phi )$ for every $\phi \in C_{\operatorname {rd}}(V(\mathbb {A}))$ .

Lemma 5.8. Let $\Omega $ be a compact subset of $G(\mathbb {A})$ . Let $U\in \Gamma (V)$ . Then,

$$\begin{align*}\sum_{\xi\in U: \, \xi_{\beta} \neq 0\, \forall\beta \in\Psi_U} \left|{\phi(\rho(g)^{-1}\xi)}\right| \ll_{N,\phi}^{\operatorname{rd}} e^{\sum_{\beta\in\Psi_U} (n_\beta\left\langle{\beta},{H_0(g)}\right\rangle_+ - N\left\langle{\beta},{H_0(g)}\right\rangle_-)},\ \ g \in T_0(\mathbb{A})\Omega. \end{align*}$$

Equivalently,

$$\begin{align*}e^{\left\langle{\lambda(U)},{H_0(g)}\right\rangle}\left|{\det\rho(g)}\right|{}^{-1}\sum_{\xi\in U: \, \xi_{\beta} \neq 0\, \forall\beta \in\Psi_U} \left|{\phi(\rho(g)^{-1}\xi)}\right| \ll_{N,\phi}^{\operatorname{rd}} \delta_{P_0}(m_0(g))e^{-N\sum_{\beta\in\Psi_U}\left\langle{\beta},{H_0(g)}\right\rangle_-}. \end{align*}$$

In particular, for $U=V$

$$\begin{align*}\left|{\det\rho(g)}\right|{}^{-1}\sum_{\xi\in V: \, \xi_{\beta} \neq 0\, \forall\beta \in\Psi_V} \left|{\phi(\rho(g)^{-1}\xi)}\right| \ll_{N,\phi}^{\operatorname{rd}} \min(1,\left|{\det\rho(g)}\right|)^N. \end{align*}$$

Proof. The first two statements are clearly equivalent by the definition (5.6) of $\lambda (U)$ . Since $\rho $ acts continuously on $C_{\operatorname {rd}}(V(\mathbb {A}))$ , we may assume without loss of generality that $g\in T_0(\mathbb {A})$ . (In fact, since both sides are left $T_0(F)$ -invariant, we may assume that $g\in T_0(\mathbb {R})$ where $\mathbb {R}$ is viewed as a subring of $\mathbb {A}=\mathbb {A}_{\mathbb {Q}}\otimes F$ via $x\mapsto x\otimes 1$ .)

Upon replacing $\phi $ by $\prod _{\beta \in \Psi _V}(1+\lVert v_{\beta ,\infty }\rVert )^{-n}\mathbf {1}_{K_\beta }(v_{\beta ,\operatorname {fin}})$ , $n\gg 0$ where $K_\beta $ is a compact open subgroup of $V_\beta (\mathbb {A}_{\operatorname {fin}})$ for every $\beta \in \Psi _V$ , we may also assume that $\phi \big |_{U (\mathbb {A})}$ factors as a product $\phi (v) = \prod _{\beta \in \Psi _U} \phi _\beta (v_\beta )$ , $v \in U (\mathbb {A})$ , where each factor $\phi _\beta \in C_{\operatorname {rd}}(V_\beta (\mathbb {A}))$ is nonnegative real. Since $T_0$ acts by scalars on each space $V_\beta $ , we are reduced to the assertion (for $t\in \mathbb {A}^\times $ )

$$\begin{align*}\sum_{\xi_{\beta} \neq 0} \phi_\beta (t^{-1}\xi_{\beta})\ll_{N,\phi_\beta}^{\operatorname{rd}} \begin{cases}\left|{t}\right|{}^{n_\beta}&\left|{t}\right| \ge 1,\\ \left|{t}\right|{}^{N}&\left|{t}\right| \le 1.\end{cases} \end{align*}$$

In turn, this immediately reduces to the following elementary statement for $\varphi \in C_{\operatorname {rd}}(\mathbb {A})$ :

$$\begin{align*}\sum_{\xi\in F^\times}\varphi(t^{-1}\xi)\ll_{N,\varphi}^{\operatorname{rd}}\begin{cases}\left|{t}\right|&\left|{t}\right|\ge1\\\left|{t}\right|{}^N&\left|{t}\right|\le1.\end{cases} \end{align*}$$

To prove this, note that given a compact subset K of $\mathbb {A}$ , we have

$$\begin{align*}\#(F^\times\cap tK)\ll\left|{t}\right|. \end{align*}$$

In particular, $F^\times \cap tK=\emptyset $ if $\left |{t}\right |$ is sufficiently small. Thus, given a compact subset $K_{\operatorname {fin}}$ of $\mathbb {A}_{\operatorname {fin}}$ ,

$$\begin{align*}\#\{\xi\in F^\times\mid (t^{-1}\xi)_{\operatorname{fin}}\in K_{\operatorname{fin}}\text{ and }\lVert(t^{-1}\xi)_\infty\rVert\le R\}\ll R^{[F:\mathbb{Q}]}\left|{t}\right|,\ \ R\ge0. \end{align*}$$

The statement above follows easily from this estimate and the fact that $\varphi \in C_{\operatorname {rd}}(\mathbb {A})$ .

Proof of Proposition 5.1

Define the support of an element $\xi \in V$ to be

$$\begin{align*}\operatorname{supp}\xi=\{\beta\in\Psi_V\mid\xi_\beta\ne0\}, \end{align*}$$

so that for any $U\in \Gamma (V)$ , $\xi \in U\iff \operatorname {supp}\xi \subset \Psi _U$ .

We decompose $\theta _\phi $ according to the support of the summands, i.e., we write

(5.8) $$ \begin{align} \theta_\phi(g)=\sum_{U\in\Gamma(V)}\Theta_U(g)\text{ where } \Theta_U(g)=\sum_{\xi\in X(F):\,\operatorname{supp}\xi=\Psi_U}\phi(\rho(g)^{-1}\xi). \end{align} $$

Of course, $\Theta _U$ is no longer $G(F)$ -invariant or even $P_0(F)$ -invariant, but it is $T_0(F)$ -invariant.

At any rate, for the proof of Proposition 5.1 we may assume by (5.1) that $g\in (T_0(\mathbb {A})\cap \mathfrak {S})\Omega $ , which we will do throughout.

Fix $U\in \Gamma (V)$ . By Lemma 5.8, we have

(5.9) $$ \begin{align} \Theta_U(g)\ll_{N,\phi}^{\operatorname{rd}} \delta_{P_0}(m_0(g))\left|{\det\rho(g)}\right|e^{-\left\langle{\lambda(U)},{H_0(g)}\right\rangle - N \sum_{\beta\in\Psi_U} \left\langle{\beta},{H_0(g)}\right\rangle_-}. \end{align} $$

Moreover, we may assume that $U\in \Gamma _{\operatorname {reg}}(V)$ , for otherwise $\Theta _U(g)=0$ . By (5.7), we have

$$\begin{align*}\lambda(U)=\sum_{\beta\in\Psi_U}c_\beta\beta+\sum_{\alpha\in\Delta_0}d_\alpha\alpha, \end{align*}$$

with suitable coefficients $c_\beta ,d_\alpha \ge 0$ . Since $g\in (T_0(\mathbb {A})\cap \mathfrak {S})\Omega $ , we deduce that

$$\begin{align*}\Theta_U(g)\ll_{N,\phi}^{\operatorname{rd}} \delta_{P_0}(m_0(g))\left|{\det\rho(g)}\right|e^{-\sum_{\beta\in\Psi_U}(c_\beta \left\langle{\beta},{H_0(g)}\right\rangle_+ +N\left\langle{\beta},{H_0(g)}\right\rangle_-)}. \end{align*}$$

In particular,

(5.10) $$ \begin{align} \Theta_U(g)\ll_\phi^{\operatorname{rd}} \delta_{P_0}(m_0(g))\left|{\det\rho(g)}\right|. \end{align} $$

The proposition follows.

6.1

We start with the following elementary fact from convex geometry.

In the situation of Lemma 6.1, we will say that $v_i$ , $i\in I'$ is the positive envelope of $\lambda $ with respect to $v_i$ , $i\in I$ .

Fix $U\in \Gamma _{\operatorname {reg}}(V)$ . Recall that $\lambda (U) \in X^*(T_0)$ was defined in (5.6) and that $\lambda (U)\in \mathbb {R}_{\ge 0}(\Delta _0\cup \Psi _U)$ by (5.7). Let $\mathcal {C}_U \subset \mathfrak {a}_0$ be the cone dual to $\mathbb {R}_{\ge 0}(\Delta _0\cup \Psi _U)$ , that is,

(6.1) $$ \begin{align} \mathcal{C}_U=\{x\in\mathfrak{a}_{0,+}\mid\left\langle{\beta},{x}\right\rangle\ge0\ \forall\beta\in\Psi_U\}. \end{align} $$

Let $\mathcal {F}_U$ be the face of $\mathcal {C}_U$ defined by $\lambda (U)$ , i.e.,

$$\begin{align*}\mathcal{F}_U=\{x\in\mathcal{C}_U\mid\left\langle{\lambda(U)},{x}\right\rangle=0\}. \end{align*}$$

Roughly speaking, $\mathcal {F}_U$ describes the directions in $T_0(F)\backslash (T_0(\mathbb {A})\cap \mathfrak {S})$ where the bounded function $\delta _{P_0}^{-1}\left |{\det \rho }\right |{}^{-1}\Theta _U$ (see (5.8)) does not decay. To analyze $\mathcal {F}_U$ , let $A\subset \Delta _0\cup \Psi _U$ be the positive envelope of $\lambda (U)$ with respect to $\Delta _0\cup \Psi _U$ . Let $I=A\cap \Delta _0$ and $J=A\cap \Psi _U$ .

(Note that we do not assume at the outset that $\Delta _0$ and $\Psi _U$ are disjoint.)

Then,

$$\begin{align*}\mathcal{F}_U=\{x\in\mathcal{C}_U\mid\left\langle{\alpha},{x}\right\rangle=\left\langle{\beta},{x}\right\rangle=0\ \forall\alpha\in\Delta_0^{\mathbf{E}(U)},\beta\in J\}. \end{align*}$$

Remark 6.3. Let $U\in \Gamma _{\operatorname {reg}}(V)$ and let $\iota $ be as in Lemma 5.6. Then, the image under $\iota $ of $\mathcal {J}_{V/U}$ contains $\mathcal {I}_{N_{\mathbf {E}(U)}}$ and is contained in $\mathcal {I}_{\mathbf {E}(U)}$ .

Indeed, as in the proof of Corollary 5.7 we write

$$\begin{align*}\lambda (U) =\sum_{j\in \mathcal{J}_{V/U}} (\beta_j+\alpha_{\iota(j)}) + \delta_0-\sum_{j\in \mathcal{J}_{V/U}}\alpha_{\iota(j)}. \end{align*}$$

It follows from the definition of $\mathbf {E}(U)$ that $\delta _0-\sum _{j\in \mathcal {J}_{V/U}}\alpha _{\iota (j)}$ can be written as a linear combination of $\Delta _0^{\mathbf {E}(U)}$ . This precisely means that $\mathcal {I}_{N_{\mathbf {E}(U)}}\subset \iota (\mathcal {J}_{V/U})\subset \mathcal {I}_{\mathbf {E}(U)}$ .

6.2

To go further, we need a follow-up of Lemma 5.6. It will play a key role.

Proof. Write for the Lie algebra of S. Let $N=N_S$ , and let $\bar {N}$ be the unipotent radical of the parabolic subgroup of G opposite to S. Denote the corresponding Lie algebras by and .

Let $v_0\in U\cap X$ . The composition of the surjection $D_{v_0}$ (2.1) with the projection $V\rightarrow V/U$ factors as a surjective map

We have for all $\alpha \in \Phi _{\bar {N}}=-\Phi _N$ . The existence of $\iota $ follows as in the proof of Lemma 5.6. Moreover,

$$\begin{align*}\lambda(U)=\sum_{\beta\in\Psi_V\setminus\Psi_U}n_\beta\beta+\delta_0= \sum_{j\in\mathcal{J}_{V/U}}(\beta_j+\alpha_{\iota(j)})+\delta_0-\sum_{j\in\mathcal{J}_{V/U}}\alpha_{\iota(j)}. \end{align*}$$

Clearly,

where $\mu =\sum _{i\in \mathcal {I}_N\setminus \iota (\mathcal {J}_{V/U})}\alpha _i$ is a (possibly empty) sum of roots in N. Therefore, $S\subset \mathbf {E}(U)$ , and the inclusion is proper if $\mu \ne 0$ .

Note that the following statements are equivalent: $\mu =0$ ; $\iota $ is a bijection; $\dim V/U=\dim G/S$ ; $\tilde D_{v_0}$ is an isomorphism of vector spaces.

Observe that the kernel of $\tilde {D}_{v_0}$ is and we have a short exact sequence

Thus, $\tilde {D}_{v_0}$ is an isomorphism if and only if and . Since $\operatorname {Ker} D_{v_0}=\operatorname {Lie} G_{v_0}$ , the first condition means that $G_{v_0}^\circ \subset S$ , while the second condition means that the S-orbit of $v_0$ is dense in U. This proves that conditions (33a)–(33d) are equivalent and that the validity of (33d) does not depend on the choice of $v\in U\cap X$ . Therefore, under these conditions, $U\cap X$ is a single S-orbit, since $U\cap X$ is irreducible. Since $U\cap X=\rho (G_{v_0,U})^{-1}v_0$ , this also implies that $G_{v_0,U}=G_{v_0}S$ , which is a finite union of left S-cosets since $G_{v_0}^\circ \subset S$ .

Finally, if $G_{v_0,U}=\bigcup _{i=1}^n \eta _i S$ is the union of finitely many left S-cosets, then

$$\begin{align*}U\cap X=\rho(G_{v_0,U})^{-1}v_0=\bigcup_{i=1}^n \rho(S)\rho(\eta_i)^{-1}v_0, \end{align*}$$

is the union of finitely many S-orbits, at least one of which must be dense, while $G_{\rho (\eta _i)^{-1}v_0}^\circ \subset S$ for every $i = 1, \ldots , n$ , since $G_{\rho (\eta _i)^{-1}v_0}=\eta _i^{-1}G_{v_0}\eta _i\subset \eta _i^{-1}G_{v_0,U}$ is contained in a finite union of left S-cosets. Thus, condition (e) is also equivalent to the other ones. This concludes the proof of the proposition.

As we shall see, the spaces considered in the proposition above deserve special attention.

In principle, using Proposition 6.4 one can enumerate $\Gamma _{\operatorname {spcl}}(V)$ , at least in small rank cases or when H is contained in only a few parabolic subgroups.

6.3

For $\alpha \in \Phi _G$ denote by $N_{(\alpha )}$ the unipotent subgroup of G with Lie algebra if $2\alpha \in \Phi _G$ and otherwise.

The following general fact about representations is probably well-known.

This follows from the representation theory of $\operatorname {SL}_2$ . To explain this, we need a standard consequence of the Jacobson–Morozov Theorem.

Proof of Lemma 6.9

For any integer k let $V_k\in \Gamma (V)$ be such that

$$\begin{align*}\Psi_{V_k}=\{\beta\in\Psi_V\mid\left\langle{\beta},{\alpha^\vee}\right\rangle=k\}. \end{align*}$$

Fix . Using Lemma 6.10 and basic facts about representation theory of $\operatorname {SL}_2$ , it follows that for any integer k, the map

$$\begin{align*}d\rho(e):V_{k-1}\rightarrow V_{k+1}\text{ is } \begin{cases}\text{surjective}&\text{if }k\ge0,\\\text{injective}&\text{if }k\le0,\end{cases} \end{align*}$$

Thus, for every $\beta \in \Psi _V$ ,

$$\begin{align*}d\rho(e):V_\beta\rightarrow V_{\beta+\alpha}\text{ is } \begin{cases}\text{surjective}&\text{if }\left\langle{\beta},{\alpha^\vee}\right\rangle+1\ge0,\\\text{injective}&\text{if }\left\langle{\beta},{\alpha^\vee}\right\rangle+1\le0.\end{cases} \end{align*}$$

In particular, $d\rho (e)(V_\beta )\ne 0$ if $V_{\beta +\alpha }\ne 0$ . The lemma follows.

Proof. Let $U=U_1\cap U_2$ , $S=\operatorname {Stab}(U)$ , $N=N_S$ , $S_k=\operatorname {Stab}(U_k)$ , $N_k=N_{S_k}$ , $M_k=M_{S_k}$ , $k=1,2$ . Clearly, $\Psi _U=\Psi _{U_1}\cap \Psi _{U_2}$ , $P_0$ stabilizes U and $S\supset S_1\cap S_2$ . In particular, $\Phi _N\subset \Phi _{N_1}\cup \Phi _{N_2}$ .

By Proposition 6.4, there exists a one-to-one function

$$\begin{align*}\iota:\mathcal{J}_{V/U}\rightarrow\mathcal{I}_N, \end{align*}$$

such that $\beta _j+\alpha _{\iota (j)}\in \Psi _U$ for all $j\in \mathcal {J}_{V/U}$ .

We claim that if $\beta _j\in \Psi _V\setminus \Psi _{U_1}$ , then $\alpha _{\iota (j)}\in \Phi _{N_1}$ . Indeed, suppose otherwise. Then, $\alpha _{\iota (j)}\in \Phi _{M_1}$ (since $\alpha _{\iota (j)}>0$ as $\alpha _{\iota (j)}\in \Phi _N$ ). Therefore, using Lemma 6.9, $\beta _j=(\beta _j+\alpha _{\iota (j)})-\alpha _{\iota (j)}\in \Psi _{U_1}$ since $U_1$ is $S_1$ -stable, in contradiction to the assumption that $\beta _j\notin \Psi _{U_1}$ .

Thus, the restriction of $\iota $ to $\mathcal {J}_{V/U_1}$ is a bijection with $\mathcal {I}_{N_1}$ (since $U_1\in \Gamma _{\operatorname {spcl}}(V)$ ). Similarly, the restriction of $\iota $ to $\mathcal {J}_{V/U_2}$ is a bijection with $\mathcal {I}_{N_2}$ .

It follows that $\iota $ is onto and $\Phi _N=\Phi _{N_1}\cup \Phi _{N_2}$ . Thus, $U \in \Gamma _{\operatorname {spcl}}(V)$ and $N=N_1N_2$ , which implies that $S=S_1\cap S_2$ .

Proof. Clearly $\tilde U\in \Gamma _{\operatorname {reg}}(V)$ , since $\tilde U\supset U$ , and $\operatorname {Stab}(\tilde U)\supset \mathbf {E}:=\mathbf {E}(U)$ . In view of Proposition 6.4, in order to prove the lemma we only need to show that $\tilde {\mathbf {E}}:=\mathbf {E}(\tilde U)\subset \mathbf {E}$ . We have $\Psi _{\tilde U}\supset \Psi _U$ . Write

$$\begin{align*}\lambda(\tilde U)= \sum_{\beta\in\Psi_{\tilde U}} c_\beta \beta +\sum_{\alpha\in \Delta_0^{\tilde{ \mathbf{E}}}} d_\alpha\alpha, \end{align*}$$

with $c_\beta \ge 0$ for all $\beta \in \Psi _{\tilde U}$ and $d_\alpha>0$ for all $\alpha \in \Delta _0^{\tilde {\mathbf {E}}}$ . Then,

$$\begin{align*}\lambda(U)= \sum_{\beta\in\Psi_U} c_\beta\beta + \sum_{\beta\in\Psi_{\tilde U}\setminus\Psi_U} (c_\beta+n_\beta)\beta + \sum_{\alpha\in \Delta_0^{\tilde {\mathbf{E}}}} d_\alpha \alpha. \end{align*}$$

Using Lemma 5.4 for each $\beta \in \Psi _{\tilde U}\setminus \Psi _U$ , we can rewrite the above relation as

$$\begin{align*}\lambda(U)= \sum_{\beta\in\Psi_U} c^{\prime}_\beta \beta +\sum_{\alpha\in \Delta_0} d^{\prime}_\alpha \alpha, \end{align*}$$

where $c^{\prime }_\beta \ge 0$ for all $\beta \in \Psi _U$ , $d^{\prime }_\alpha \ge 0$ for $\alpha \in \Delta _0\setminus \Delta _0^{\mathbf {E}}$ and $d^{\prime }_\alpha \ge d_\alpha>0$ for all $\alpha \in \Delta _0^{\tilde {\mathbf {E}}}\setminus \Delta _0^{\mathbf {E}}$ . By Lemma 6.1 we deduce that $d^{\prime }_\alpha =0$ for all $\alpha \in \Delta _0\setminus \Delta _0^{\mathbf {E}}$ . This implies that $\tilde {\mathbf {E}}\subset \mathbf {E}$ , as required.

Example 6.14. Consider Example 3.2 with $k=3$ and $n_1<n_2$ . Then, it is easy to see that $\Gamma _{\operatorname {spcl}}(V)=\{V,V_1,V_2\}$ where

$$ \begin{align*} V_1&=\{(x_1,x_2)\mid\text{the first }n_2-n_1\text{ columns of }x_1\text{ vanish}\},\\ V_2&=\{(x_1,x_2)\mid\text{the last }n_2-n_3\text{ rows of }x_2\text{ vanish}\}. \end{align*} $$

The general case of Example 3.2 is far more complicated and will not be considered here.

Example 6.15. Consider Example 3.8 pertaining to $F_4$ . Write for simplicity $(i_1i_2i_3i_4)$ instead of $i_1\alpha _1+i_2\alpha _2+i_3\alpha _3+i_4\alpha _4\in X^*(T_0)$ .

We explicate $\Psi _V$ in a way that demonstrates its realization as pairs of $3\times 3$ -symmetric matrices as follows:

$$\begin{align*}\Psi_V=\left(\begin{smallmatrix}(0100)&(0110)&(0111)\\(0110)&(0120)&(0121)\\(0111)&(0121)&(0122)\end{smallmatrix}\right),\left(\begin{smallmatrix}(1100)&(1110)&(1111)\\(1110)&(1120)&(1121)\\(1111)&(1121)&(1122)\end{smallmatrix}\right). \end{align*}$$

Consider the subspaces $U_1,U_2,U_3$ in $\Gamma (V)$ given by

$$ \begin{align*} \Psi_{U_1}&=\Psi_V\setminus \{(0100),(1100)\},\\ \Psi_{U_2}&=\Psi_V\setminus \{(0100),(0110),(0120)\},\\ \Psi_{U_3}&=\Psi_V\setminus \{(0100),(0110),(0111)\}. \end{align*} $$

In matrices,

$$\begin{align*}U_1=\left(\begin{smallmatrix} 0&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right), \left(\begin{smallmatrix} 0&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right),\ \ U_2=\left(\begin{smallmatrix} 0&0&* \\ 0&0&* \\ *&*&* \end{smallmatrix}\right), \left(\begin{smallmatrix} *&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right),\ \ U_3=\left(\begin{smallmatrix} 0&0&0 \\ 0&*&* \\ 0&*&* \end{smallmatrix}\right), \left(\begin{smallmatrix} *&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right). \end{align*}$$

Let $U_4=U_1\cap U_2$ , $U_5=U_2\cap U_3$ so that

$$\begin{align*}U_4=\left(\begin{smallmatrix} 0&0&* \\ 0&0&* \\ *&*&* \end{smallmatrix}\right), \left(\begin{smallmatrix} 0&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right),\ \ U_5=\left(\begin{smallmatrix} 0&0&0 \\ 0&0&* \\ 0&*&* \end{smallmatrix}\right), \left(\begin{smallmatrix} *&*&* \\ *&*&* \\ *&*&* \end{smallmatrix}\right). \end{align*}$$

Then, it is not difficult to check that

$$\begin{align*}\Gamma_{\operatorname{spcl}}(V)=\{V,U_1,U_2,U_3,U_4,U_5\}. \end{align*}$$

Moreover,

$$ \begin{align*} &\operatorname{Stab}(U_1)=\mathbf{E}(U_1)=(*00*),\ \operatorname{Stab}(U_2)=\mathbf{E}(U_2)=(00{*}0),\ \operatorname{Stab}(U_3)=\mathbf{E}(U_3)=(000*),\\ &\operatorname{Stab}(U_4)=\operatorname{Stab}(U_5)=\mathbf{E}(U_5)=(0000),\ \ \mathbf{E}(U_4)=(00{*}0). \end{align*} $$

7.1

Let $\mathcal {C}$ be a closed cone in a finite-dimensional real vector space W.

Let f be a continuous real-valued function on W and let $d>0$ . We say that f is positively-d-homogeneous if $f(rv)=r^df(v)$ for all $r>0$ , $v\in W$ . (In the cases at hand, $d=1$ .) In this case, f is positive on $\mathcal {C}\setminus \{0\}$ if (and of course, only if) it is positive on the intersection of $\mathcal {C}$ with the unit sphere of W with respect to a prescribed norm.

The space of continuous positively-d-homogeneous functions on W is a Banach space with respect to the supremum norm on the unit ball and the positivity on $\mathcal {C}\setminus \{0\}$ is an open condition.

By integration using spherical coordinates, it is clear that if f is positively-d-homogeneous and positive on $\mathcal {C}\setminus \{0\}$ , then the integral of $e^{-f}$ over $\mathcal {C}$ converges.

Suppose that f is positively- $1$ -homogeneous and let $\lambda _j$ , $j\in J$ be a finite set of vectors in $W^*$ . Then, the positively- $1$ -homogeneous function $f+N\sum _{j\in J}\left \langle {\lambda _j},{v}\right \rangle _-$ is positive on $\mathcal {C}\setminus \{0\}$ for $N\gg 0$ if and only if f is positive on the subcone $\{v\in \mathcal {C}:\left \langle {\lambda _j},{v}\right \rangle \ge 0\ \forall j\in J\}\setminus \{0\}$ .

These facts yield the following elementary convergence result.