2.1 Statement of the problem

Throughout the paper, we let denote the space of complex $n \times d$ matrices, with . By the (complex) classical groups, we mean:

• • the general linear group ;

• • the orthogonal group ;

• • the symplectic group , where $J = \left [\begin {smallmatrix} 0 & I \\ -I & 0 \end {smallmatrix}\right ]$ .

That is, $\operatorname {O}_n$ preserves the symmetric bilinear form b on $\mathbb C^n$ given by $b(v,w)=v^Tw$ , while $\operatorname {\mathrm {Sp}}_{2n}$ preserves the skew-symmetric bilinear form $\omega $ on $\mathbb C^{2n}$ given by $\omega (v,w) = v^T J w$ .

In the context of each classical group G, we will write V to denote its defining representation, on which it acts naturally by matrix multiplication: hence, for $\operatorname {\mathrm {GL}}_n$ and $\operatorname {O}_n$ , we have $V = \mathbb C^n$ , while for $\operatorname {\mathrm {Sp}}_{2n}$ , we have $V=\mathbb C^{2n}$ . For $G = \operatorname {O}_n$ or $\operatorname {\mathrm {Sp}}_{2n}$ , we let denote the space of polynomial functions on V; for $G = \operatorname {\mathrm {GL}}_n$ , we let . In each case, $\Psi $ is a representation of G in the natural way, via $g \cdot \psi (v) = \psi (g^{-1}v)$ , for $g \in G$ , $\psi \in \Psi $ , and $v \in V$ (or $V \oplus V^*$ ). This, in turn, induces a representation of G on $\mathcal P(\Psi )$ , the space of polynomial functions on $\Psi $ (explained below), again via $g \cdot f(\psi ) = f(g^{-1} \cdot \psi )$ for $f \in \mathcal P(\Psi )$ . Our goal is to describe, as explicitly as possible, the invariant ring

2.2 Notation

Let . A Greek letter $\alpha = (\alpha _1,\ldots ,\alpha _n) \in \mathbb N^n$ will denote a multi-index corresponding to the exponent vector of the monomial $x_1^{\alpha _1} \cdots x_n^{\alpha _n}$ . It will be natural to regard $x_i$ as the linear functional on V dual to the ith standard basis vector. We will use these exponent vectors $\alpha $ to pick out coefficients of monomials; our desired invariants will then be polynomials in these coefficients. Below we organize the details, which vary slightly for each of the three classical groups:

• • For $G = \operatorname {\mathrm {GL}}_n$ :

  1. – We have We can regard $\partial _i$ as the ith standard basis element of the double dual $V^{**}$ , although the notation $\partial _i$ is indeed meant as an abbreviation for the differential operator $\partial /\partial x_i$ . (See Section 6 for the motivation from the Weyl algebra, i.e., the algebra of polynomial-coefficient differential operators. Note, however, that the analysis in this paper will not use the non-commutative algebra structure of the Weyl algebra, which we view only as a representation of $\mathrm {GL}_n$ .)

  2. – For $\alpha ,\beta \in \mathbb N^n$ , we write .

  3. – We have the monomial basis $\{\mathbf x^\alpha \boldsymbol {\partial }^\beta \mid \alpha ,\beta \in \mathbb N^n\}$ for $\Psi $ .

  4. – We define the linear functional $ c_{\alpha ,\beta } : \Psi \longrightarrow \mathbb C$ , which takes the value 1 on the basis vector $\mathbf x^\alpha \boldsymbol {\partial }^\beta $ and 0 on all other basis vectors.

  5. – Hence, we have $\mathcal P(\Psi ) = \mathbb C[c_{\alpha ,\beta }]_{\alpha ,\beta \in \mathbb N^n}$ .

• • For $G = \operatorname {O}_n$ :

  1. – We have .

  2. – We have the monomial basis $\{\mathbf x^\alpha \mid \alpha \in \mathbb N^n\}$ for $\Psi $ , where .

  3. – We define the linear functional $c_\alpha : \Psi \longrightarrow \mathbb C$ , which takes the value 1 on the basis vector $\mathbf x^\alpha $ and 0 on all other basis vectors.

  4. – Hence we have $\mathcal P(\Psi ) = \mathbb C[c_\alpha ]_{\alpha \in \mathbb N^n}$ .

• • For $G = \operatorname {\mathrm {Sp}}_{2n}$ :

  1. – We have .

  2. – We have the monomial basis $\{\mathbf x^\alpha \mid \alpha \in \mathbb N^{2n}\}$ for $\Psi $ .

  3. – We define the linear functional $c_\alpha :\Psi \longrightarrow \mathbb C$ , just as for $\operatorname {O}_n$ above.

  4. – Hence, we have $\mathcal P(\Psi ) = \mathbb C[c_\alpha ]_{\alpha \in \mathbb N^{2n}}$ .

The span of the c’s is the graded dual , where $\Psi _i$ is the ith homogeneous graded component of $\Psi $ with respect to polynomial degree. Note that $\Psi ^\circ $ is not the same as the full dual space $\Psi ^*$ , which includes infinite linear combinations of the c’s. The “c” notation stands for “coefficient,” since the c’s extract the coefficient of the corresponding monomial. Hence, our desired invariants will be polynomials in these c’s, i.e., elements of $\mathcal P(\Psi )\cong S(\Psi ^\circ )$ . As is customary in the literature, we define scaled coefficients $\widehat {c}$ as follows, writing :

We will appeal to the graded isomorphism of G-modules $\Psi \cong \Psi ^\circ $ given by

(2.1) $$ \begin{align} \begin{cases} \mathbf x^\alpha\boldsymbol{\partial}^\beta \longmapsto \widehat{c}_{\alpha,\beta}, & G = \operatorname{\mathrm{GL}}_n,\\ \phantom{\mathbf x^\alpha}\mathbf x^\alpha \longmapsto \widehat{c}_\alpha, & G = \operatorname{O}_n \text{ or }\operatorname{\mathrm{Sp}}_{2n}. \end{cases} \end{align} $$

(See [Reference WallachWal17, p. 117] and [Reference DolgachevDol03, pp. 7–8].) From a different perspective, it is a standard fact (see [Reference DolgachevDol03, Remark 1.2]) that the differential operator $\boldsymbol {\partial }^\alpha $ and the monomial $\mathbf x^\alpha $ transform dually under a classical group action; hence, in each graded component, the isomorphism in (2.1) can equivalently be written as

$$ \begin{align*} \begin{cases} \mathbf x^\alpha\boldsymbol{\partial}^\beta \longmapsto \boldsymbol{\partial}^\alpha \mathbf x^\beta, & G = \operatorname{\mathrm{GL}}_n,\\ \phantom{\mathbf x^\alpha}\mathbf x^\alpha \longmapsto \boldsymbol{\partial}^\alpha,& G = \operatorname{O}_n \text{ or }\operatorname{\mathrm{Sp}}_{2n}. \end{cases} \end{align*} $$

Here, we regard $\boldsymbol {\partial }^\alpha \mathbf x^\beta $ as the operator acting via $\boldsymbol {\partial }^\alpha \mathbf x^\beta (\mathbf x^\gamma \boldsymbol {\partial }^\delta ) = \boldsymbol {\partial }^\alpha (\mathbf x^\gamma )\cdot \boldsymbol {\partial }^\delta (\mathbf x^\beta )$ . For example, if $G=\operatorname {\mathrm {GL}}_2$ and $\Psi _{4,3} = \operatorname {span}\{\mathbf x^\alpha \boldsymbol {\partial }^\beta : |\alpha |=4, \: |\beta |=3\}$ , then $\partial _1^3 \partial _2^{\phantom {1}} x_1^{\phantom {1}} x_2^2$ equals the functional $\widehat {c}_{(3,1),(1,2)}$ , since it takes the value $3!\cdot 1! \cdot 1! \cdot 2!$ on the monomial $x_1^3 x_2^{\phantom {1}} \partial _1^{\phantom {1}}\partial _2^2 \in \Psi _{4,3}$ , and takes the value 0 on all other monomials in $\Psi _{4,3}$ .

In order to obtain finite-dimensional graded components, we impose a bigradation on $\mathcal P(\Psi )$ . First is the natural gradation by degree; hence, $\mathcal P^d(\Psi )$ is the space of homogeneous polynomials of degree d in the c’s. The second gradation is by weight. To make this clear, we write , and then we define the weight of a monomial in the c’s as follows:

As we will see, the factor $\frac {1}{2}$ arises due to the fact that the fundamental invariants are quadratics. Now we define $\mathcal P_k(\Psi )$ to be the space of polynomials that are homogeneous (i.e., “isobaric,” in the classical language) of weight k. Finally, we set .

2.3 Weyl’s fundamental theorems

In his monumental book [Reference WeylWey39], Weyl determined generators and relations for the ring of invariants, in the setting where a classical group G acts naturally on an arbitrary number d of vectors (and covectors, when $G = \operatorname {\mathrm {GL}}_n$ ). For all three classical groups, the first fundamental theorem (FFT) states that the generators are certain quadratic functions $r_{ij}$ . The second fundamental theorem (SFT) states the relations as the determinants (or Pfaffians) of certain minors in the $r_{ij}$ .

Combining the FFT and the SFT, one obtains an algebra isomorphism $\mu ^\sharp $ between the ring of invariants, on one hand, and the coordinate ring of a certain determinantal variety, on the other hand. (The notation $\mu ^\sharp $ is typical in the literature, because this map is the comorphism induced by a matrix multiplication map $\mu $ .) We will use the notation $\operatorname {\mathrm {SM}}_d \subset \operatorname {\mathrm {M}}_d$ for the subspace of symmetric matrices, and $\operatorname {\mathrm {AM}}_d \subset \operatorname {\mathrm {M}}_d$ for the subspace of alternating (i.e., skew-symmetric) matrices; we let $z_{ij}$ denote the natural coordinate functions on $\operatorname {\mathrm {M}}_d$ . The aforementioned determinantal varieties are denoted by

For our purposes in this paper, we present a version of the combined FFT and SFT below (once for each classical group) in terms of matrix coordinates $x_{ij}$ , rather than the coordinate-free presentation used by Weyl.

Theorem 2.1 (FFT and SFT for $G = \operatorname {\mathrm {GL}}_n$ )

Let $G = \operatorname {\mathrm {GL}}_n$ , acting on $\operatorname {\mathrm {M}}_{nd} \oplus \operatorname {\mathrm {M}}_{nd} \cong V^{\oplus d} \oplus (V^*)^{\oplus d}$ via $g \cdot (X,Y) = (gX, (g^{-1})^TY)$ . Set

(2.2)

Then $R^G$ is generated by the quadratics

(2.3)

Moreover, we have the following isomorphism of algebras:

(2.4) $$ \begin{align} \begin{split} \mu^{\sharp}:\mathcal P(\operatorname{\mathrm{M}}_d^{\leq n}) &\longrightarrow R^G,\\ z_{ij} & \longmapsto r_{ij}. \end{split} \end{align} $$

Theorem 2.2 (FFT and SFT for $G = \operatorname {O}_n$ )

Let $G = \operatorname {O}_n$ , acting by matrix multiplication on $\operatorname {\mathrm {M}}_{nd} \cong V^{\oplus d}$ . Set

(2.5)

Then $R^G$ is generated by the quadratics

(2.6)

Moreover, we have the following isomorphism of algebras:

(2.7) $$ \begin{align} \begin{split} \mu^{\sharp}:\mathcal P(\operatorname{\mathrm{SM}}_d^{\leq n}) &\longrightarrow R^G,\\ z_{ij} & \longmapsto r_{ij}. \end{split} \end{align} $$

Theorem 2.3 (FFT and SFT for $G = \operatorname {\mathrm {Sp}}_{2n}$ )

Let $G = \operatorname {\mathrm {Sp}}_{2n}$ , acting by matrix multiplication on $\operatorname {\mathrm {M}}_{2n,d} \cong V^{\oplus d}$ . Set

(2.8)

Then $R^G$ is generated by the quadratics

(2.9)

Moreover, we have the following isomorphism of algebras:

(2.10) $$ \begin{align} \begin{split} \mu^{\sharp}:\mathcal P(\operatorname{\mathrm{AM}}_d^{\leq 2n}) &\longrightarrow R^G,\\ z_{ij} & \longmapsto r_{ij}. \end{split} \end{align} $$

3.1 General algorithm

Our first result is the following algorithm, where the input is a certain type of graph with d vertices and k edges, and the output is a basis element in $\mathcal P^d_k(\Psi )^G$ . The algorithm takes the same form for all three classical groups. (See also the Appendix, where we include Mathematica code to implement the algorithm.) Recall that to each classical group G, we have already associated (in Theorems 2.1–2.3) a polynomial ring R and the quadratics $r_{ij} \in R$ . In the following three subsections, we will specify the set $\mathcal {G}^d_k$ of graphs for each group. The key to the algorithm is a G-equivariant linear operator $\varphi :R \longrightarrow \mathcal P^d(\Psi )$ , which (following [Reference SturmfelsStu08, p. 174]) we call the umbral operator. Again, in the following three subsections, we will explicitly define $\varphi $ for each group.

3.2 Details: the general linear group

For $G = \operatorname {\mathrm {GL}}_n$ , we define $\mathcal G^d_k$ to be the set of directed multigraphs with loops, with d vertices and k edges, up to isomorphism. The quadratics $r_{ij}$ are defined in (2.3). Define the umbral operator $\varphi : R \longrightarrow \mathcal P^d(\Psi )$ by

(3.1) $$ \begin{align} \varphi: \prod_{i,j} x_{ij}^{p_{ij}}y_{ij}^{q_{ij}} \longmapsto \prod_{j=1}^d \widehat{c}_{p_{_{\bullet j,}}\: q_{_{\bullet j}}}, \end{align} $$

where $p_{_{\bullet j}} = \left (p_{1j},\ldots ,p_{nj}\right )$ and $q_{_{\bullet j}} = \left (q_{1j},\ldots ,q_{nj}\right )$ , and extend by linearity.

3.3 Details: the orthogonal group

For $G = \operatorname {O}_n$ , we define $\mathcal G^d_k$ to be the set of undirected multigraphs with loops, with d vertices and k edges, up to isomorphism. The quadratics $r_{ij}$ are defined in (2.6). Define the umbral operator $\varphi : R \longrightarrow \mathcal P^d(\Psi )$ by

(3.2) $$ \begin{align} \varphi:\prod_{i,j} x_{ij}^{p_{ij}} \longmapsto \prod_{j=1}^d \widehat{c}_{p_{_{\bullet j}}}, \end{align} $$

where $p_{\bullet j} = \left (p_{1j},\ldots ,p_{nj}\right )$ , and extend by linearity.

3.4 Details: the symplectic group

For $G = \operatorname {\mathrm {Sp}}_{2n}$ , the graphs in $\mathcal G^d_k$ are more delicate to define. Let $\Gamma $ be an undirected multigraph with no loops, having d vertices. Arbitrarily choose vertex labels $1, \ldots , d$ ; then each element $\sigma $ of the symmetric group ${\mathfrak S_d}$ determines another labeled graph $\sigma \cdot \Gamma $ , obtained by permuting the original labels. The stabilizer $\operatorname {stab}_{{\mathfrak S_d}}(\Gamma )$ contains all elements $\sigma \in {\mathfrak S_d}$ such that $\Gamma $ is isomorphic to $\sigma \cdot \Gamma $ as a labeled graph. If $\{i,j\}$ is an edge of $\Gamma $ with $i < j$ , then we say that $\sigma $ inverts $\{i,j\}$ if $\sigma (i)> \sigma (j)$ . We now define the following set of graphs for the case $G=\operatorname {\mathrm {Sp}}_{2n}$ :

(3.3)

This set is independent of the initial choice of labeling for $\Gamma $ , since ${\mathfrak S_d}$ acts d-transitively on the vertex labels. Below we present examples of the graphs in $\mathcal G^d_k$ for $d \leq 3$ . (For $d=1$ , by definition, $\mathcal G^1_k = \varnothing $ since loops are not allowed.)

Example 3.5 Now suppose $d=3$ . Below we list the elements of $\mathcal G^3_k$ for the first few values of k. The reader can check that we have included precisely the graphs defined in (3.3):

As for the remaining details in Algorithm 3.1 for $G=\operatorname {\mathrm {Sp}}_{2n}$ : the quadratics $r_{ij}$ are defined in (2.9), and we define the umbral operator $\varphi : R \longrightarrow \mathcal P^d(\Psi )$ by

(3.4) $$ \begin{align} \varphi:\prod_{i,j} x_{ij}^{p_{ij}} \longmapsto \prod_{j=1}^d \widehat{c}_{p_{_{\bullet j}}}, \end{align} $$

where $p_{\bullet j} = \left (p_{1j},\ldots ,p_{2n,j}\right )$ , and extend by linearity.

5.1 Combinatorial difficulty outside the stable range

Inside the stable range (1.1), one could scarcely hope for a nicer combinatorial understanding of $\mathcal P(\Psi )^G$ than that given by Algorithm 3.1: specifically, we can obtain the dimension of each graded component $\mathcal P^d_k(\Psi )^G$ simply by counting the graphs in $\mathcal {G}^d_k$ . Outside the stable range, however, the graph-counting approach overshoots the true dimension, due to the complicated linear dependencies arising among the images $\varphi \circ s(\Gamma )$ .

Taking $G = \operatorname {\mathrm {GL}}_n$ , for example, it is straightforward to give a linear basis for $\mathcal P^k(\operatorname {\mathrm {M}}_d^{\leq n})$ consisting of the degree-k monomials of width at most n; the term “width” follows the sense of Sturmfels [Reference SturmfelsStu90], which we now summarize. A pair $(T,U)$ of semistandard Young tableaux with the same shape, each having (say) $\ell $ columns, and filled with entries from the set , can be viewed as a product of $\ell $ determinants in $\mathcal P(\operatorname {\mathrm {M}}_d)$ , as follows: the ith determinant is the minor of matrix coordinates whose rows (resp. columns) are given by the ith column of T (resp. U). It follows that the size of T (equivalently, of U) is the degree of the corresponding function in $\mathcal P(\operatorname {\mathrm {M}}_d)$ . The set of all such pairs (often called bitableaux in the literature) thus yields a linear basis for $\mathcal P(\operatorname {\mathrm {M}}_d)$ . Via the straightening law of [Reference Doubilet, Rota and SteinDRS74], it can be shown that by restricting to those bitableaux with size k and at most n rows, one obtains a linear basis for $\mathcal P^k(\operatorname {\mathrm {M}}^{\leq n}_d)$ . Sturmfels showed that these somewhat complicated basis elements (an example of standard monomials) can be replaced by certain ordinary monomials in the matrix coordinates $z_{ij}$ , by applying the Robinson–Schensted–Knuth (RSK) correspondence. The RSK correspondence, described by Knuth in [Reference KnuthKnu70, Section 3], is a bijection between the set of bitableaux and the set $\operatorname {\mathrm {M}}_d(\mathbb N)$ . In particular, the sum of the entries in the matrix $\mathrm {RSK}(T,U)$ equals the size of T; moreover, when the support of $\mathrm {RSK}(T,U)$ is viewed as a subset of the poset $[d] \times [d]$ , the width of that support (i.e., the size of the largest antichain) equals the number of rows in T. By viewing $\mathrm {RSK}(T,U)$ as the degree matrix of a monomial in the variables $z_{ij}$ , we can thus associate each bitableau to an ordinary monomial rather than to a standard monomial. We show this below in an example where $d=5$ and $n \geq 3$ :

In this way, a basis for $\mathcal P^k(\operatorname {\mathrm {M}}_d^{\leq n})$ is given by those ordinary monomials whose degree matrices have entries summing to k and width at most n. Finally, by viewing the degree matrix as an adjacency matrix, one can represent the basis monomials by digraphs with k edges on d (labeled) vertices. Variations of the RSK correspondence (see [Reference BurgeBur74] or [Reference ConcaCon94]) can be used to obtain similar graphical bases for $\mathcal P^k(\operatorname {\mathrm {SM}}_d^{\leq n})$ and $\mathcal P^k(\operatorname {\mathrm {AM}}_d^{\leq 2n})$ , corresponding to the groups $\operatorname {O}_n$ and $\operatorname {\mathrm {Sp}}_{2n}$ .

The difficulty in the present paper, however, arises from our symmetrization, whereby one “forgets” the vertex labels in a graph. In particular, when $n < d$ and $n < k$ , it is no longer clear how to parametrize a basis for $\mathcal P^k(\operatorname {\mathrm {M}}_d^{\leq n})^{{\mathfrak S_d}}$ . After all, the sense of “width” defining the labeled graphs is inherited from the poset of matrix coordinates, and symmetrization (by its very nature) nullifies the partial order. Outside the stable range, therefore (and with the exception of Section 5.3), we leave it as an open problem to exhibit a combinatorial realization of a basis of $\mathcal P^d_k(\Psi )^G$ . (We have had some success in the two border cases $n = d-1$ and $n=d-2$ , where it is possible to obtain a basis by deleting those graphs containing certain subgraphs; but the choice of these subgraphs is hardly canonical, and moreover this method seems to be hopeless when $d - n> 2$ .)

In this section, we remedy this combinatorial defect by taking a representation-theoretic approach. Note that our goal can be restated more elegantly as follows: we wish to understand the bigraded Hilbert series of the invariant ring $\mathcal P(\Psi )^G$ , namely

5.2 Branching multiplicities

The facts in this section are standard in any general reference on representation theory, such as [Reference Goodman and WallachGW09, Section 3.2]. A polynomial representation of $\operatorname {\mathrm {GL}}_d$ is a group homomorphism $\rho : \operatorname {\mathrm {GL}}_d \longrightarrow \operatorname {\mathrm {GL}}_m$ for some m, such that the matrix coordinates $\rho (g)_{ij}$ are polynomials in the entries of $g \in \operatorname {\mathrm {GL}}_d$ . (As is typical, we will also use the term “representation” for the complex m-dimensional vector space on which $\rho (\operatorname {\mathrm {GL}}_n)\subset \operatorname {\mathrm {GL}}_m$ acts.) A partition is a finite, weakly decreasing sequence of positive integers (parts), typically denoted by a lowercase Greek letter. If d is the sum of the parts of $\lambda $ , then we write $\lambda \vdash d$ . The number of parts of $\lambda $ is called its length, denoted by $\ell (\lambda )$ . We write $(d)$ to denote the length-1 partition of size d. We also define the set

We will speak of the rows and columns of a partition, by which we mean the rows and columns of its associated Young diagram.

By the theorem of the highest weight, the irreducible polynomial representations of $\operatorname {\mathrm {GL}}_d$ are indexed by the partitions $\lambda $ such that $\ell (\lambda )\leq d$ . We write $F^\lambda _d$ for the irreducible representation of $\operatorname {\mathrm {GL}}_d$ with highest weight $\lambda $ . Now consider the restriction of $\operatorname {\mathrm {GL}}_d$ to its subgroup ${\mathfrak S_d}$ , realized as permutation matrices. If $\mu \vdash d$ , we write $Y^\mu _d$ for the irreducible representation of ${\mathfrak S_d}$ which corresponds to $F^\mu _d$ via Schur–Weyl duality. Upon restriction to ${\mathfrak S_d}$ , the representation $F^\lambda _d$ decomposes into a direct sum of irreducible representations $Y^\mu _d$ ; we write

(5.1)

to denote the branching multiplicity of $Y^\mu _d$ in $F^\lambda _d$ . Hence, we have

(5.2) $$ \begin{align} \mathrm{Res}^{\operatorname{\mathrm{GL}}_d}_{{\mathfrak S_d}}F^\lambda_d \cong \bigoplus_{\mu \vdash d} b^\lambda_\mu\: Y^\mu_d. \end{align} $$

Proof (general linear group)

Recall from the two thick arrows in diagram (4.1) that we have a linear isomorphism $\mathcal P^k(\operatorname {\mathrm {M}}_d^{\leq n})^{\mathfrak S_d} \cong \mathcal P^d_k(\Psi )^G$ . Hence, it will suffice to show that $\dim \mathcal P^k(\operatorname {\mathrm {M}}_d^{\leq n})^{\mathfrak S_d}$ equals the sum in the theorem.

The determinantal variety $\operatorname {\mathrm {M}}_d^{\leq n}$ admits an action by $\operatorname {\mathrm {GL}}_d \times \operatorname {\mathrm {GL}}_d$ , via $(g,h)\cdot X = gXh^{-1}$ , which extends to $\mathcal P(\operatorname {\mathrm {M}}_d^{\leq n})$ in the usual way. Weyl’s SFT (from Section 2.3) can be applied to obtain the decomposition below [Reference Goodman and WallachGW09, Theorem 12.2.12.3]:

$$\begin{align*}\mathcal P^k(\operatorname{\mathrm{M}}_d^{\leq n}) \cong \bigoplus_{\lambda\in \operatorname{\mathrm{Par}}(k,\:\min\{d,n\})} \left(F^\lambda_d\right)^* \otimes F^\lambda_d. \end{align*}$$

Restricting to ${\mathfrak S_d} \times {\mathfrak S_d}$ , we use (5.2) to write

$$ \begin{align*} \mathcal P^k(\operatorname{\mathrm{M}}_d^{\leq n}) &\cong \bigoplus_{\lambda} \left(\bigoplus_{\mu \vdash d} b^\lambda_\mu \:Y^\mu_d\right)^* \otimes \left(\bigoplus_{\nu \vdash d} b^\lambda_\nu\: Y^\nu_d\right)\\ & \cong \bigoplus_{\lambda,\mu,\nu} b^\lambda_\mu b^\lambda_\nu \left(Y^\mu_d \otimes Y^\nu_d\right), \end{align*} $$

since the irreducible representations of ${\mathfrak S_d}$ are self-dual. Further restricting to the diagonal subgroup $\Delta ({\mathfrak S_d}) = \{(\sigma ,\sigma ) \mid \sigma \in {\mathfrak S_d}\}$ , we note that the $\Delta ({\mathfrak S_d})$ -action is just by conjugation on $\operatorname {\mathrm {M}}_d^{\leq n}$ . Therefore, we are interested in the subspace of $\Delta ({\mathfrak S_d})$ -invariants:

$$ \begin{align*}\mathcal P^k(\operatorname{\mathrm{M}}_d^{\leq n})^{\mathfrak S_d} \cong \left[\bigoplus_{\lambda,\mu,\nu} b^\lambda_\mu b^\lambda_\nu \left(Y^\mu_d \otimes Y^\nu_d\right)\right]^{\Delta({\mathfrak S_d})} = \bigoplus_{\lambda,\mu,\nu} b^\lambda_\mu b^\lambda_\nu \left( Y^\mu_d \otimes Y^\nu_d\right)^{\Delta({\mathfrak S_d})}. \end{align*} $$

But by Schur’s lemma, we have $\dim (Y^\mu _d \otimes Y^\nu _d)^{\Delta ({\mathfrak S_d})} = \dim \operatorname {\mathrm {Hom}}_{{\mathfrak S_d}}(Y^\mu _d,Y^\nu _d) = \delta _{\mu ,\nu }$ . Therefore, we keep only the summands in which $\mu =\nu $ , and we obtain

$$ \begin{align*}\mathcal P^k(\operatorname{\mathrm{M}}_d^{\leq n})^{\mathfrak S_d} \cong \bigoplus_{\lambda,\mu} \left(b^\lambda_\mu\right)^2 \underbrace{\left(Y^\mu_d \otimes Y^\mu_d\right)^{\Delta({\mathfrak S_d})}}_{\text{one-dimensional}}, \end{align*} $$

which proves Part 1 of the theorem.

Proof (orthogonal group)

In the $\operatorname {O}_n$ -analogue of the diagram (4.1), we have an isomorphism $\mathcal P^k(\operatorname {\mathrm {SM}}_d^{\leq n})^{\mathfrak S_d} \cong \mathcal P^d_k(\Psi )^G$ . Hence, it will suffice to show that $\dim \mathcal P^k(\operatorname {\mathrm {SM}}_d^{\leq n})^{\mathfrak S_d}$ equals the sum in the theorem.

The determinantal variety $\operatorname {\mathrm {SM}}_d^{\leq n}$ admits an action by $\operatorname {\mathrm {GL}}_n$ , via $g \cdot X = g^{-T}Xg^{-1}$ , which extends to $\mathcal P(\operatorname {\mathrm {SM}}_d^{\leq n})$ in the usual way. The SFT can be applied to obtain the decomposition below [Reference Goodman and WallachGW09, Theorem 12.2.14.3], which we decompose under the restriction to ${\mathfrak S_d}$ using (5.2):

$$ \begin{align*} \mathcal P^k(\operatorname{\mathrm{SM}}_d^{\leq n}) &\cong \bigoplus_{\lambda} F^{\lambda}_d\\ &\cong \bigoplus_\lambda \bigoplus_{\mu \vdash d} b^{\lambda}_\mu \: Y^\mu_d, \end{align*} $$

where the sum ranges over all $\lambda \in \operatorname {\mathrm {Par}}(2k,\: \min \{d,n\})$ with even row lengths. The partition $(d)$ labels the one-dimensional trivial representation of ${\mathfrak S_d}$ , and therefore the ${\mathfrak S_d}$ -invariant subspace above is the sum of the trivial representations $Y^{(d)}_d$ in the sum:

This proves Part 2 of the theorem.

Proof (symplectic group)

As in the previous parts, from the $\operatorname {\mathrm {Sp}}_{2n}$ -analogue of the diagram (4.1), we have an isomorphism $\mathcal P^k(\operatorname {\mathrm {AM}}_d^{\leq 2n})^{\mathfrak S_d} \cong \mathcal P^d_k(\Psi )^G$ . Hence, we will find the dimension of $\mathcal P^k(\operatorname {\mathrm {AM}}_d^{\leq 2n})^{\mathfrak S_d}$ .

The determinantal variety $\operatorname {\mathrm {AM}}_d^{\leq 2n}$ admits an action by $\operatorname {\mathrm {GL}}_d$ , just as in the $\operatorname {O}_n$ case. The SFT can be applied to obtain the decomposition below [Reference Goodman and WallachGW09, Theorem 12.2.15.3], which we decompose under the restriction to ${\mathfrak S_d}$ , using (5.2):

$$ \begin{align*} \mathcal P^k(\operatorname{\mathrm{AM}}_d^{\leq 2n}) &\cong \bigoplus_{\lambda} F^\lambda_d \\ & \cong \bigoplus_\lambda \bigoplus_{\mu \vdash d} b^{\lambda}_\mu \: Y^\mu_{d}, \end{align*} $$

where the sum ranges over all $\lambda \in \operatorname {\mathrm {Par}}(2k,\:\min \{d,2n\})$ with even column lengths. Therefore, as in Part 2, the ${\mathfrak S_d}$ -invariant subspace decomposes as

which proves Part 3 of the theorem.

5.3 Bases when $n=1$

The theme of this section – and the motivation for looking at branching multiplicities – has been the extreme difficulty in parametrizing a linear basis for $\mathcal P^d_k(\Psi )^G$ outside the stable range. One exception to this is actually the farthest extreme from the stable range, namely the case when $\dim V = 1$ . This occurs when $G= \operatorname {\mathrm {GL}}_1$ or $\operatorname {O}_1$ . In this case, the quadratics $r_{ij}$ are monomials, and thus so too are the expressions $s(\Gamma )$ and $\varphi \circ s(\Gamma )$ . It is fairly easy to see from the definitions that when $G = \operatorname {\mathrm {GL}}_1$ , we have

$$\begin{align*}\varphi \circ s(\Gamma) = \prod_{i = 1}^d \widehat{c}_{\operatorname{indeg}(i), \operatorname{outdeg}(i)}, \end{align*}$$

where indeg and outdeg denote the in-degree and out-degree of a vertex of $\Gamma $ . Likewise, when $G = \operatorname {O}_1$ , we have

$$\begin{align*}\varphi \circ s(\Gamma) = \prod_{i=1}^d \widehat{c}_{\operatorname{deg}(i)}, \end{align*}$$

where deg denotes the degree of a vertex. In other words, for both $\operatorname {\mathrm {GL}}_1$ and $\operatorname {O}_1$ , the invariant $\varphi \circ s(\Gamma )$ is determined by the degree sequence of $\Gamma $ ; hence, we can actually describe a basis for each component $\mathcal P^d_k(\Psi )^G$ by replacing the graphs in our method by simpler combinatorial objects. For $\operatorname {\mathrm {GL}}_1$ , we can parametrize a basis by the sets $\{(a_1,b_1), \ldots , (a_d, b_d)\}$ with $a_i,b_i \in \mathbb N$ such that $\sum _i a_i = \sum _i b_i = k$ . For $\operatorname {O}_1$ , we can parametrize a basis by the partitions of $2k$ with at most d parts.

For the symplectic group, the parameter $n=1$ corresponds to $\operatorname {\mathrm {Sp}}_{2} \cong \operatorname {\mathrm {SL}}(2, \mathbb C)$ . Since the defining representation V has dimension 2, a combinatorial parametrization of linear bases seems to be much more difficult than for $\operatorname {\mathrm {GL}}_1$ and $\operatorname {O}_1$ above. Nonetheless, thanks to Theorem 5.1, we do have an especially nice expression for the dimensions in terms of a single branching multiplicity:

$$\begin{align*}\dim \mathcal P^d_k(\mathbb C[x,y])^{\operatorname{\mathrm{SL}}(2, \mathbb C)} = b^{(k,k)}_{(d)} \end{align*}$$

for all d and k. This is because when $n=1$ , there is just one element $(k,k)$ in the set of all partitions of $2k$ with length at most 2, and with even column lengths.

6.1 The general linear group

For $\operatorname {\mathrm {GL}}_n$ , we have $\Psi = \mathcal P(V\oplus V^*)$ , which is isomorphic (as a $\operatorname {\mathrm {GL}}_n$ -module) to the associated graded algebra $\mathbb C[ x_1,\ldots ,x_n,\partial _1,\ldots ,\partial _n]$ of the Weyl algebra, i.e., the algebra of polynomial-coefficient differential operators on $\mathcal P(V)$ . In this context, $x_i$ is the operator that multiplies by $x_i$ , and $\partial _i$ is the differential operator $\partial /\partial x_i$ . (Of course, the Weyl algebra itself is not commutative, since $\partial _i x_i = x_i\partial _i+1$ , but here we are interested only in the G-module structure.) Explicitly, we have $\operatorname {span}\{x_1,\ldots ,x_n\} \cong V^*$ and $\operatorname {span}\{\partial _1,\ldots ,\partial _n\} \cong V$ as $\operatorname {\mathrm {GL}}_n$ -modules. (See the detailed exposition in [Reference ProcesiPro07, p. 39].) Thus, $\Psi $ is of interest because the $\operatorname {\mathrm {GL}}_n$ -orbits are precisely the equivalence classes of linear partial differential equations with polynomial coefficients, under linear change of coordinates. In turn, the invariants $\mathcal P(\Psi )^{\operatorname {\mathrm {GL}}_n}$ are those polynomial functions constant on the $\operatorname {\mathrm {GL}}_n$ -orbits. For example ( $n=3$ ), if $\omega (x,y)$ is a polynomial, then the two-dimensional time-dependent Schrödinger equation

$$\begin{align*}-i \frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} + \omega(x,y)u \end{align*}$$

corresponds to the Weyl algebra element $i \partial _3 + \partial _1^2 + \partial _2^2 + \omega (x_1, x_2) \in \Psi $ , upon setting $x = x_1$ , $y = x_2$ , and $t = x_3$ . See also Example 6.5 below, in which we view vector fields as elements of $\Psi $ .

Example 6.3 ( $n=d=k=2$ )

In this example, we will compute a basis for $\mathcal P^2_2(\Psi )^{\operatorname {\mathrm {GL}}_2}$ . Since $G=\operatorname {\mathrm {GL}}_2$ , we write x and y in place of $x_1$ and $x_2$ , so that $\Psi = \mathbb C[x,y,\partial _x,\partial _y]$ . As in the previous example, we write coefficients as $[x^a y^b \partial _x ^c \partial _y^d]$ rather than $c_{(a,b),(c,d)}$ .

Note that our parameters $n=d=k=2$ lie in the stable range, and so there is a one-to-one correspondence between $\mathcal G^2_2$ and our desired basis. There are six directed graphs with two vertices and two edges, and thus $\#\mathcal G^2_2 = 6 = \dim \mathcal P^2_2(\Psi )^{\operatorname {\mathrm {GL}}_2}$ . In Table 1, we display each of these graphs, its adjacency matrix (up to simultaneous permutation of rows and columns), and its corresponding basis element in $\mathcal P^2_2(\Psi )^{\operatorname {\mathrm {GL}}_2}$ .

Table 1: A basis for $\mathcal P^2_2(\Psi )^{\operatorname {\mathrm {GL}}_2}$ , computed using Algorithm 3.1.

Remark 6.4 We observe a general phenomenon from Table 1, which follows directly from the definition of the umbral operator in (3.1). Let $\Gamma \in \mathcal G^d_k$ , and choose an adjacency matrix $A^\Gamma $ for any vertex labeling. Then, in each term

$$ \begin{align*}c_{\alpha^1,\beta^1} \cdots c_{\alpha^d,\beta^d} \quad (\alpha^i,\beta^i \in \mathbb N^n) \end{align*} $$

of the corresponding invariant $\varphi \circ s(\Gamma )$ , the d factors $c_{\alpha ^i,\beta ^i}$ can be rearranged so that

(6.1) $$ \begin{align} \begin{split} |\alpha^i| &= i\text{th column sum of }A^\Gamma,\\ |\beta^i| &= i\text{th row sum of }A^\Gamma \end{split} \end{align} $$

simultaneously for each $i=1,\ldots , d$ .

Now recall that an element $\psi \in \Psi $ is a vector field if it takes the form

$$\begin{align*}\psi = \sum_{i=1}^n f_i(\mathbf x) \:\partial_i \end{align*}$$

with each $f_i(\mathbf x) \in \mathbb C[\mathbf x]$ . In this case, $c_{\alpha ,\beta }(\psi ) \neq 0$ implies that $|\beta |=1$ . By the observation (6.1), then, $\varphi \circ s(\Gamma )$ vanishes on $\psi $ unless $A^\Gamma $ contains exactly one 1 in each row, with 0’s elsewhere; equivalently, every vertex in $\Gamma $ has out-degree 1. For fixed d, each such digraph can be regarded naturally as the equivalence class of a finite dynamical system on a d-element set.

Example 6.5 (Quadratic invariants on two-dimensional vector fields)

Let $G = \operatorname {\mathrm {GL}}_2$ , and define the vector field

$$ \begin{align*}\psi = \big(x^2 + y^2 + 2xy + 2x + 2y + 1\big)\: \partial_x + \big(x^2 + y^2 - 2xy + 4x - 4y+4\big)\: \partial_y, \end{align*} $$

which (since all coefficients are real) we depict in Figure 1a. Let $g = \left [\begin {smallmatrix} 0 & -2 \\ 1 & \phantom {-}0 \end {smallmatrix}\right ] \in G$ . Then

$$ \begin{align*}g \cdot \psi = -\left(\frac{1}{2}x^2 + 2y^2 + 2xy + 4x + 8y + 8\right)\partial_x + \left(\frac{1}{4} x^2 + y^2 - xy - x + 2y + 1\right)\partial_y, \end{align*} $$

which we depict in Figure 1b. We will evaluate the quadratic G-invariant functions at both $\psi $ and $g \cdot \psi $ . The only graphs in Table 1 that correspond to nonvanishing invariants on $\psi $ are the finite dynamical systems, presented in rows 1, 4, and 6. We compute these invariants in Table 2.

Figure 1: The vector fields from Example 6.5.

Table 2: Nonvanishing quadratic invariants on the vector fields $\psi $ and $g \cdot \psi $ from Example 6.5.

6.2 The orthogonal group

Example 6.6 We consider the familiar example of conic sections in $\mathbb R^2$ . The general form of a conic section is

$$ \begin{align*}Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0,\end{align*} $$

where $A,\ldots ,F \in \mathbb R$ . (We remark that $\operatorname {O}_n$ is defined over $\mathbb R$ , and its real points constitute the subgroup $\operatorname {O}(n,\mathbb R)$ .) In Table 3, we list the well-known invariants on conic sections, under the action of $\operatorname {O}(2,\mathbb R)$ ; the right-hand column presents the corresponding graphs from Algorithm 3.1, where $G = \operatorname {O}_2$ is the complexification of $\operatorname {O}(2,\mathbb R)$ .

Table 3: Invariants of conic sections under the action of $\operatorname {O}(2,\mathbb R)$ .

6.3 The symplectic group

Let $n=1$ . In this case, since $G = \operatorname {\mathrm {Sp}}_2 \cong \operatorname {\mathrm {SL}}_2$ , our Algorithm 3.1 actually finds $\operatorname {\mathrm {SL}}_2$ -invariants on $\Psi \cong \mathbb C[x,y]$ . Whereas early classical invariant theory focused on $\operatorname {\mathrm {SL}}_2$ -invariants on binary m-forms, i.e., on the space $S^m(\mathbb C^2)$ spanned by $\{x^m, x^{m-1}y, \ldots , xy^{m-1}, y^m\}$ , our approach comprises the $\operatorname {\mathrm {SL}}_2$ -invariants on the direct sum of all these spaces, namely the entire polynomial ring $\mathbb C[x,y] \cong \bigoplus _{m=0}^\infty S^m(\mathbb C^2)$ .

In [Reference Olver and ShakibanOS89], and subsequently in [Reference OlverOlv99, Chapter 7], Olver describes a graphical method for invariants (and more generally, covariants) of binary forms, which can be regarded as a special case of our Algorithm 3.1. Specifically, when $G = \operatorname {\mathrm {Sp}}_2$ and $\Gamma $ is m-regular (i.e., every vertex has degree m), our corresponding invariant $\varphi \circ s(\Gamma )$ restricts to an invariant on the space of binary m-forms, which coincides (up to a constant multiple) with the invariant obtained from the same graph in Olver’s method. (Although Olver’s graphs are directed, reversing an arrow just multiplies the corresponding covariant by $-1$ .)

Example 6.8 (Invariants of binary forms)

In Table 4, we list the well-known fundamental invariants of the binary quadric, cubic, and quartic ( $m=2,3,4$ ), along with their corresponding graphs from Algorithm 3.1, up to a scale factor. In place of our c notation, we write the coefficients as capital letters $A,B,\ldots $ in descending order of degree in x. The symbol $\Delta _m$ denotes the discriminant of the binary m-form. In the case of the quartic, we follow [Reference OlverOlv99, p. 29] in denoting the two fundamental invariantsFootnote ¹ by i and j; then the discriminant is $\Delta _4 = i^3 - 27j^2$ . Graphically, the product of invariants corresponds to the union of their corresponding graphs. Note that the graphs in Table 4 are indeed m-regular, with the number of vertices giving the degree of the invariant in the coefficients.

Table 4: Fundamental invariants of the binary quadric, cubic, and quartic.

Example 6.9 In Olver’s method, for fixed m, a graph that is not m-regular corresponds to a covariant on binary m-forms (i.e., an $\operatorname {\mathrm {SL}}_2$ -invariant polynomial in not just the coefficients of the form, but also in the variables x and y). For us, however, all graphs correspond to true invariants on $\mathbb C[x,y]$ . Consequently, certain syzygies among Olver’s graphs do not carry over into our graphs, specifically Olver’s “Rule #2”; see illustration (7.3) in [Reference OlverOlv99, p. 135]. As a specific example, take the following graph $\Gamma \in \mathcal G^3_4$ , which for Olver in [Reference OlverOlv99, p. 135] corresponds to the zero covariant on a binary form of degree $m \geq 3$ :

For us, however, $\Gamma $ corresponds to a nontrivial degree- $3$ invariant on $\mathbb C[x,y]$ :

$$ \begin{align*}\varphi \circ s(\Gamma) = 16c_{12}^2 c^{}_{20}-8c^{}_{11}c^{}_{12}c^{}_{21}-48c^{}_{03}c^{}_{20}c^{}_{21}+16c^{}_{02}c^2_{21}+72c^{}_{03}c^{}_{11}c^{}_{30}-48c^{}_{02}c^{}_{12}c^{}_{30,} \end{align*} $$

where we have suppressed the parentheses around the ordered pairs in the subscripts. Observe that the vertices of $\Gamma $ have degrees $2$ , $3$ , and $3$ ; this is reflected in the fact that each term in the corresponding invariant takes the form $c_\alpha c_\beta c_\gamma $ , where $\alpha ,\beta ,\gamma \in \mathbb N^2$ and

$$ \begin{align*}|\alpha|=2,\quad |\beta|=3, \quad |\gamma|=3. \end{align*} $$

A.1 General linear group

A.2 Orthogonal group

A.3 Symplectic group