2.1. Boundary trace, capacitary setting and main results

Throughout this paper, we assume that

(2.1)\begin{equation} 0 \leq k \leq N-2, \quad \mu \leq H^2 \quad \text{and} \quad \lambda_{\mu,\Sigma}>0. \end{equation}

Under assumption (2.1), a theory for linear equations involving $L_\mu$ was established in [Reference Barbatis, Gkikas and Tertikas3, Reference Marcus and Nguyen29], which forms a basis for the study of equation (E _±). We also note that the first and second inequalities in (2.1) imply that ${\alpha _{+}} \geq H \geq 1$. Moreover ${\alpha _{+}}=1$ if and only if $k=N-2$ and $\mu =1$; in this case, we have ${\alpha _{-}}=1$.

First we focus on the equation with an absorption power nonlinearity

(E ₊)\begin{equation} -L_\mu u+|u|^{p-1}u=0 \quad \text{in } \Omega. \end{equation}

Before stating the main results for equation (E ₊), we introduce some notations. For any $\beta >0,$ we set

(2.2)\begin{equation} \Sigma_\beta:=\{x\in\mathbb{R}^N\setminus \Sigma:\,d_\Sigma(x)<\beta\} \quad \text{and} \quad \Omega_{\beta}:=\{ x \in \Omega: d_{\partial \Omega}(x)<\beta \}. \end{equation}

It is well known that (see appendix A.1) there is a small enough number $\beta _0>0$ such that for any $x\in \Omega _{\beta _0}$ there exists a unique $\xi _x\in \partial \Omega$ satisfying $d_{\partial \Omega }(x)=|x-\xi _x|$. Now set

(2.3)\begin{equation} \tilde{d}_{\Sigma}(x):=\sqrt{|{\rm dist}^{\partial\Omega}(\xi_x,\Sigma)|^2+|x-\xi_x|^2}, \end{equation}

where $\mathrm {dist}\,^{\partial \Omega }$ denotes the geodesic distance on $\partial \Omega$.

Let $\beta _3>0$ be the constant in proposition A.1. (One may choose $\beta _3<\beta _0$.) Let $\eta _{\beta _3}$ be a smooth cut-off function such that $0\leq \eta _{\beta _3}\leq 1$ such that $\eta _{\beta _3}=1$ in $\overline {{\Sigma }_{\frac {\beta _3}{4}}}$ with compact support in $\Sigma _{\frac {\beta _3}{2}}$. We define

\[ W(x):=\left\{ \begin{array}{@{}ll} (d_{\partial \Omega}(x)+\tilde d_\Sigma(x)^2)\tilde d_{\Sigma}(x)^{-{\alpha_{+}}}, & \text{if}\;\mu < H^2, \\ (d_{\partial \Omega}(x)+\tilde d_\Sigma(x)^2)\,d_{\Sigma}(x)^{{-}H}|\ln \tilde d_{\Sigma}(x)|, & \text{if}\;\mu =H^2, \end{array} \right. \quad x \in \Omega \cap \Sigma_{\beta_3}, \]

where ${\alpha _{+}}$ is defined in (1.5), and define

(2.4)\begin{equation} \tilde W(x):=(1-\eta_{\beta_3}(x))+\eta_{\beta_3}(x)W(x) , \quad x \in \Omega. \end{equation}

In the particular case $\mu =0$ and $\Sigma =\partial \Omega$, we have ${\alpha _{+}}=1$, whence $\tilde W(x)\approx 1$. We note that $\tilde W$ is an appropriate function to describe the boundary behaviour in a normalization sense of solutions to ($E_{\pm }$). For more detail, see (4.1) and (4.2) (see also [Reference Barbatis, Gkikas and Tertikas3, lemma 6.8]).

Our first theorem provides a removability result when $p\geq \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$ in the sense that if a nonnegative solution ‘vanishes’ on $\partial \Omega \setminus \Sigma$ as in (2.5), then it must be identically zero.

Theorem 2.1 Assume $\mu \leq H^2$ if $k< N-2$ or $\mu < H^2$ if $k=N-2$, and $p\geq \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$. We additionally assume that $\Omega$ is a $C^3$ open bounded domain. If $u \in C^2(\Omega )$ is a nonnegative solution of (E ₊) such that

(2.5)\begin{equation} \lim_{x\in\Omega,\;x\rightarrow\xi}\frac{u(x)}{\tilde W(x)}=0, \quad\forall \xi\in \partial\Omega\setminus \Sigma, \end{equation}

locally uniformly in $\partial \Omega \setminus \Sigma$, then $u\equiv 0$ in $\Omega$.

We remark that if $k=0$ the result in the theorem 2.1 coincides with the result in [Reference Chen and Véron8, theorem J with $h=0$]. In addition, when $p\geq \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$, boundary behaviour of solutions on $\Sigma$ is not imposed. However, when $\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}\leq p< \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$, zero boundary condition on $\Sigma$ is additionally required for the removability of isolated singularities, as stated in the following theorem.

Theorem 2.2 Assume $k \geq 1$, $\mu \leq H^2$, $z\in \Sigma$ and $\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}\leq p< \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$ if ${\alpha _{+}}>1$ or $\frac {N}{N-2} \leq p$ if ${\alpha _{+}}=1$. We additionally assume that $\Omega$ is a $C^3$ bounded domain. If $u\in C^2(\Omega )$ is a nonnegative solution of (E ₊) such that

(2.6)\begin{equation} \lim_{x\in\Omega,\;x\rightarrow\xi}\frac{u(x)}{\tilde W(x)}=0\quad\forall \xi\in \partial\Omega\setminus \{z\}, \end{equation}

locally uniformly in $\partial \Omega \setminus \{z\}$, then $u\equiv 0$.

Next, we discuss existence results for a boundary value problem for (E ₊). In order to formulate the boundary value problem for (E ₊), we use a notion of boundary trace, introduced in [Reference Barbatis, Gkikas and Tertikas3], the definition of which is recalled below.

A family $\{\Omega _n\}$ is called a $C^2$ exhaustion of $\Omega$ if $\Omega _n$ is a $C^2$ bounded domain, $\Omega _n \Subset \Omega _{n+1} \Subset \Omega$ for any $n \in {\mathbb {N}}$ and $\cup _{n \in {\mathbb {N}}}\Omega _n = \Omega$.

Let $x_0 \in \Omega$ be a fixed reference point.

Definition 2.3 Boundary trace

We say that a function $u\in W^{1,\kappa }_{\mathrm {loc}}(\Omega )$ ($\kappa >1$) possesses a boundary trace if there exists a measure $\nu \in \mathfrak M(\partial \Omega )$ such that for any $C^2$ exhaustion $\{ \Omega _n \}$ of $\Omega$ containing $x_0$, there holds

\[ \lim_{n\rightarrow\infty}\int_{ \partial \Omega_n}\phi u\,\mathrm{d} \omega_{\Omega_n}^{x_0}=\int_{\partial \Omega} \phi \,\mathrm{d} \nu \quad\forall \phi \in C(\overline{\Omega}). \]

The boundary trace of $u$ is denoted by $\mathrm {tr}_{\mu,\Sigma}(u)$. Here $\omega _{\Omega _n}^{x_0}$ is the $L_\mu$-harmonic measure on $\partial \Omega _n$ relative to $x_0$ (see § 4.1).

It is known by [Reference Barbatis, Gkikas and Tertikas3, lemmas 8.1 and 8.2] that

(2.7)\begin{equation} \mathrm {tr}_{\mu,\Sigma}(\mathbb{K}_{\mu}[\nu])=\nu \quad \forall \nu \in \mathfrak M(\partial \Omega ) \quad \text{and} \quad \mathrm {tr}_{\mu,\Sigma}(\mathbb{G}_\mu[\tau])=0 \quad \forall \tau \in \mathfrak M(\Omega;\phi_{\mu,\Sigma}). \end{equation}

For $q \in [1,+\infty )$, denote by $L^q(\Omega ;\phi _{\mu,\Sigma })$ the weighted Lebesgue space

$$\begin{align*} L^q(\Omega;\phi_{\mu,\Sigma}) & := \left\{\vphantom{\left.\left(\int_\Omega|u|^2\phi_{\mu,\Sigma}^2 \,\mathrm{d}x+\int_\Omega|\nabla u|^2\phi_{\mu,\Sigma}^2 \, \mathrm{d}x \right)^{\frac{1}{2} }<{+}\infty \right\}} u: \Omega \to {\mathbb{R}} \text{ measurable such that } \| u \|_{L^q(\Omega;\phi_{\mu,\Sigma})} \right. \\ & := \left.\left( \int_{\Omega} |u|^q \phi_{\mu,\Sigma}\,\mathrm{d}x \right)^{\frac{1}{q} } <{+}\infty \right\}. \end{align*}$$

Let $H^1(\Omega ;\phi _{\mu,\Sigma }^2)$ be the weighted Sobolev space

$$\begin{align*} H^1(\Omega;\phi_{\mu,\Sigma}^2) & :=\left\{\vphantom{\left.\left(\int_\Omega|u|^2\phi_{\mu,\Sigma}^2 \,\mathrm{d}x+\int_\Omega|\nabla u|^2\phi_{\mu,\Sigma}^2 \, \mathrm{d}x \right)^{\frac{1}{2} }<{+}\infty \right\}}u \in H^1_{\mathrm{loc}}(\Omega):\;\left \|u\right \|_{H^1(\Omega;\phi_{\mu,\Sigma}^2)}\right.\\ & := \left.\left(\int_\Omega|u|^2\phi_{\mu,\Sigma}^2 \,\mathrm{d}x+\int_\Omega|\nabla u|^2\phi_{\mu,\Sigma}^2 \, \mathrm{d}x \right)^{\frac{1}{2} }<{+}\infty \right\}. \end{align*}$$

We also denote by $H^1_0(\Omega ;\phi _{\mu,\Sigma }^2)$ the closure of $C_0^\infty (\Omega )$ with respect to the norm $\left \|\cdot \right \|_{H^1(\Omega ;\phi _{\mu,\Sigma }^2)}.$ It is worth mentioning here that $H^1_0(\Omega ;\phi _{\mu,\Sigma }^2)=H^1(\Omega ;\phi _{\mu,\Sigma }^2)$ (see [Reference Barbatis, Gkikas and Tertikas3, theorem 4.5]).

Weak solutions of the boundary value problem for (E ₊) with prescribed boundary trace are defined below.

Definition 2.4 Let $p>1.$ We say that $u$ is a weak solution of

(P₊)\begin{equation} \left\{ \begin{array}{@{}ll} - L_\mu u + |u|^{p-1}u=0 & \text{in }\;\Omega,\\ \mathrm {tr}_{\mu,\Sigma}(u)=\nu. \end{array} \right. \end{equation}

if $u\in L^1(\Omega ;\phi _{\mu,\Sigma })$, $|u|^p \in L^1(\Omega ;\phi _{\mu,\Sigma })$ and

\[ - \int_{\Omega}u L_{\mu }\zeta \, \mathrm{d} x + \int_{\Omega}|u|^{p-1}u\zeta \, \mathrm{d} x={-} \int_{\Omega} \mathbb{K}_{\mu}[\nu]L_{\mu }\zeta \,\mathrm{d} x\quad\forall \zeta \in\mathbf{X}_\mu(\Omega), \]

where

(2.8)\begin{equation} {\bf X}_\mu(\Omega):=\{ \zeta \in H_{\mathrm{loc}}^1(\Omega): \phi_{\mu,\Sigma}^{{-}1} \zeta \in H^1(\Omega;\phi_{\mu,\Sigma}^{2}), \, \phi_{\mu,\Sigma}^{{-}1}L_\mu \zeta \in L^\infty(\Omega) \}. \end{equation}

We remark that in light of [Reference Barbatis, Gkikas and Tertikas3, theorem 2.12], a function $u$ is a weak solution to problem (P₊) if and only if

\[ u + \mathbb{G}_\mu[|u|^{p-1}u] = \mathbb{K}_\mu[\nu] \quad \text{a.e. in } \Omega. \]

It was known by [Reference Barbatis, Gkikas and Tertikas3, theorem B.4 (b)] (see also [Reference Marcus and Nguyen29, theorem 1.18]) that, in the subcritical case $1< p<$$\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$, problem (P₊) admits a unique weak solution for any $\nu \in \mathfrak {M}(\partial \Omega )$ with support in $\Sigma$. The supercritical case $p \geq$$\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$ is more challenging. In order to treat this case, we will make use of appropriate capacities.

For $\theta \in {\mathbb {R}}$, we define the Bessel kernel of order $\alpha$ in ${\mathbb {R}}^d$ by ${\mathcal {B}}_{d,\theta }(\xi ):={\mathcal {F}}^{-1}((1+|.|^2)^{-\frac {\theta }{2}})(\xi )$, where ${\mathcal {F}}$ is the Fourier transform in the space ${\mathcal {S}}'({\mathbb {R}}^d)$ of moderate distributions in $\mathbb {R}^d$. For $\kappa >1$, the Bessel space $L_{\theta,\kappa }(\mathbb {R}^d)$ is defined by

\[ L_{\theta,\kappa}(\mathbb{R}^d):=\{f={\mathcal{B}}_{d,\theta} \ast g:g\in L^{\kappa}(\mathbb{R}^d)\}, \]

with norm

\[ \|f\|_{L_{\theta,\kappa}}:=\|g\|_{L^\kappa}=\|{\mathcal{B}}_{d,-\theta}\ast f\|_{L^\kappa}. \]

The Bessel capacity is defined for compact set $A \subset \mathbb {R}^d$ by

\[ \mathrm{Cap}^{\mathbb{R}^d}_{\theta,\kappa}(A):=\inf\{\|f\|^\kappa_{L_{\theta,\kappa}}:\; g\in L^\kappa_+(\mathbb{R}^d),\,f={\mathcal{B}}_{d,\theta} \ast g\geq {\mathbb 1}_A \}, \]

and is extended to open sets and arbitrary sets in ${\mathbb {R}}^d$ in the standard way. Here ${\mathbb 1}_A$ denotes the indicator function of $A$.

We denote by $B^d(x,r)$ the open ball of centre $x \in {\mathbb {R}}^d$ and radius $r>0$ in ${\mathbb {R}}^d$.

Using the Bessel capacities, we are able to define capacities for subsets of $\partial \Omega$ as follows. If $\Gamma \subset \partial \Omega$ is a $C^2$ submanifold without boundary, of dimension $d$ with $1 \leq d \leq N-1$ then there exist open sets $O_1,\ldots,O_m$ in $\mathbb {R}^N$, diffeomorphisms $T_i: O_i \to B^{d}(0,1)\times B^{N-d-1}(0,1)\times (-1,1)$, $i=1,\ldots,m$, and compact sets $K_1,...,K_m$ in $\Gamma$ such that

1. (i) $K_i \subset O_i$, $1 \leq i \leq m$ and $\Gamma = \cup _{i=1}^m K_i$;

2. (ii) $T_i(O_i \cap \Gamma )=B^d(0,1) \times \{ (x_{d+1},\ldots,x_{N-1}) = 0_{\mathbb {R}^{N-d-1}} \}\times \{x_N=0\}$, $T_i(O_i \cap \Omega )={B^{d}(0,1)}\times B^{N-d-1}(0,1)\times (0,1)$;

3. (iii) For any $x \in O_i \cap \Omega$, there exists $y \in O_i \cap \Gamma$ such that $d_\Gamma (x)=|x-y|$ (here $d_\Gamma (x)$ denotes the distance from $x$ to $\Gamma$).

We then define the $\mathrm {Cap}_{\theta,\kappa }^{\Gamma }-$capacity of a compact set $E \subset \Gamma$ by

(2.9)\begin{equation} \mathrm{Cap}_{\theta,\kappa}^{\Gamma}(E):=\sum_{i=1}^m \mathrm{Cap}_{\theta,\kappa}^{\mathbb{R}^d}(\tilde T_i(E \cap K_i)), \end{equation}

where $T_i(E \cap K_i)=\tilde T_i(E \cap K_i) \times \{ (x_{d+1},\ldots,x_{N-1}) = 0_{\mathbb {R}^{N-d-1}} \}\times \{x_N=0\}$.

We remark that the definition of the capacities does not depend on $O_i$, $i=1,\ldots,m$.

In the sequel, we will say that $\nu \in \mathfrak M^+(\partial \Omega )$ is absolutely continuous with respect to a capacity $\mathscr {C}$ if

(2.10)\begin{equation} \forall \text{ Borel set } E \subset \partial \Omega \text{ such that } \mathscr{C}(E)=0\Longrightarrow \nu(E)=0. \end{equation}

Our next main result gives a sufficient condition in terms of appropriate capacities for the solvability of problem (P₊) in the range $\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}\leq p<\frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$ when the boundary trace is supported in $\Sigma$.

When $\nu$ has support in $\partial \Omega \setminus \Sigma$, we provide a necessary and sufficient condition on the boundary trace for the existence of a solution to problem (P₊) in the supercritical range $p \geq \frac {N+1}{N-1}$.

Next we investigate boundary value problem for ($E_-$) of the form

(P^σ_–)\begin{equation} \left\{ \begin{array} {@{}ll}- L_\mu u - |u|^{p-1}u=0 & \text{in }\;\Omega,\\ \mathrm {tr}_{\mu,\Sigma}(u)=\sigma \nu, \end{array} \right. \end{equation}

where $\sigma >0$ is a parameter and $\nu \in \mathfrak M^+(\partial \Omega )$.

Weak solutions to problem ($\mathrm{P}_-^{\sigma}$) are defined similarly as in definition 2.4 with obvious modifications.

In the following theorems, for any $\nu \in \mathfrak {M}^+(\partial \Omega )$, we extend it to be a measure defined on $\overline {\Omega }$ by setting $\nu (\Omega )=0$ and use the same notation $\nu$ for the extension.

When $\nu$ is concentrated on $\Sigma$, various equivalent criteria for the existence of a weak solution to problem ($\mathrm{P}_-^{\sigma}$) are described in the following result.

Theorem 2.7 Assume that $\mu < \frac {N^2}{4}$ and

(2.11)\begin{equation} 1< p<\frac{{\alpha_{-}}+1}{{\alpha_{-}}-1}\ \text{if}\ {\alpha_{-}}>1\quad \text{or}\quad p>1\ \text{if}\ {\alpha_{-}}\leq1. \end{equation}

Let $\nu \in \mathfrak M^+(\partial \Omega )$ with compact support in $\Sigma$. Then the following statements are equivalent.

1. 1. Problem ($\mathrm{P}_-^{\sigma}$) has a positive weak solution for $\sigma >0$ small.

2. 2. For any Borel set $E \subset \overline {\Omega }$, there holds

   (2.12)\begin{equation} \int_E \mathbb{K}_\mu[{\mathbb 1}_E\nu]^p \phi_{\mu,\Sigma} \,\mathrm{d} x \leq C\,\nu(E). \end{equation}

3. 3. The following inequality holds

   (2.13)\begin{equation} \mathbb{G}_\mu[\mathbb{K}_\mu[\nu]^p]\leq C\,\mathbb{K}_\mu[\nu]<{+}\infty\quad \text{a.e. in } \Omega. \end{equation}

   Assume, in addition, that $k\geq 1$ and

   (2.14)\begin{equation} \left\{ \begin{array}{@{}ll} \max\left\{1,\dfrac{N-k-{\alpha_{-}}+1}{N-1-{\alpha_{-}}} \right\}< p<\dfrac{{\alpha_{+}}+1}{{\alpha_{+}}-1} & \text{if } {\alpha_{+}}>1 \\ \text{or}\quad\max\left\{1,\dfrac{N-k}{N-2} \right\}< p & \text{if}\ {\alpha_{+}}=1 . \end{array} \right. \end{equation}
   Put
   (2.15)\begin{equation} \vartheta: = \frac{{\alpha_{+}}+1-p({\alpha_{+}}-1)}{p}. \end{equation}
   Then any of the above statements is equivalent to the following statement

4. 4. For any Borel set $E \subset \Sigma$, there holds

   \[ \nu(E)\leq C\, \mathrm{Cap}_{\vartheta,p'}^{\Sigma}(E). \]

We remark that the case $\Sigma =\{0\}$ and $\mu =\frac {N^2}{4}$ is treated in § 6.3 with slightly modified capacities; see in particular remark 6.15.

When $\nu$ is concentrated on $\partial \Omega \setminus \Sigma$, we obtain necessary and sufficient conditions for the existence of a weak solution of ($\mathrm{P}_-^{\sigma}$) for the whole range $\mu \leq \frac {N^2}{4}$.

We note that the case $p \geq \frac {{\alpha _{-}}+1}{{\alpha _{-}}-1}$ (if ${\alpha _{-}}>1$) is still open and requires a different method.

2.2. Proof strategies and comparison with relevant works in the literature

The distinctive feature of the problems (P₊) and ($\mathrm{P}_-^{\sigma}$) is characterized by the interplay between the concentration of $\Sigma$, the type of nonlinearity, the exponent $p$ and the parameter $\mu$. By employing a fine analysis in capacitary setting, we are able to obtain existence and nonexistence results in the supercritical ranges for $p$ and the critical case for the parameter $\mu$, which justifies the novelty of our paper in comparison with related works in the literature. This is discussed in more detail below.

To establish the removability results (theorems 2.1 and 2.2), we treat the cases $p \geq \frac {{\alpha _{-}}+1}{{\alpha _{-}}-1}$ and $p \geq \frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$ separately. When $p \geq \frac {{\alpha _{-}}+1}{{\alpha _{-}}-1}$, we provide a proof the heart of which is the assertion that all nonnegative solutions $u$ of problem (E ₊)–(2.5) are dominated by $\tilde W$ in light of Keller–Osserman type estimates (see proposition 5.2), hence are uniformly bounded in $L^p(\Omega ;\phi _{\mu,\Sigma })$. Consequently, thanks to the representation theorem (see theorem 3.3), these solutions admit boundary traces concentrated on $\Sigma$ with uniformly bounded total mass. Therefore, by contradiction, if there is a nontrivial nonnegative solution with positive boundary trace then there is a sequence of solutions whose total mass are unbounded, which clearly contradicts the above assertion. In the larger range $p \geq \frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$, the above assertion is no longer valid and we focus on solutions with possible isolated boundary singularities concentrated at a point on $\Sigma$ depicted by (2.6). We offer a proof, which relies on a combination of localization techniques, Keller–Osserman type estimates and weak formulation for nonhomogneous linear equations, to show the removability of isolated singularities. Our results are new and cover [Reference Marcus and Nguyen29, theorem 1.17].

We prove the solvability for problem (P₊) (theorems 2.5 and 2.6) by extending the method in [Reference Marcus and Véron31]. When the boundary trace $\nu$ has support in $\Sigma$, a crucial ingredient in the proof of theorem 2.5 is the equivalence between the quantities $\|\mathbb {K}_{\mu }[\nu ]\|_{L^p(\Omega ;\phi _{\mu,\Sigma })}$ and $\| \nu \|_{B^{-\vartheta,p}(\Sigma )}$, where $B^{-\vartheta,p}(\Sigma )$ is the dual of an appropriate Besov space (see theorem 5.4). This allows us to utilize an approximation argument to prove the existence of a (unique) weak solution to problem (P₊). When $\nu$ has support in $\partial \Omega \setminus \Sigma$, we construct the Poisson kernel associated to $-L_\mu$ and adapt the idea in [Reference Marcus and Véron32] to prove theorem 2.6. In this case, the effect of the potential $d_{\Sigma }^{-2}$ is not pivotal as it can be seen that the critical exponent and the involved capacities are the same as in the free potential case. To our knowledge, theorems 2.5 and 2.6 are the first existence results for problem (P₊) expressed in terms of capacities in case $1 \leq k \leq N-2$, which complements or extends the results in [Reference Gkikas and Véron21, Reference Marcus and Nguyen29, Reference Marcus and Véron32].

The source case is sharply different from the absorption case in several aspects due to the distinct effect of the source nonlinearity and hence require a completely different approach. Theorems 2.7 and 2.8 provide various necessary and sufficient conditions for the existence of a weak solution to problems with source nonlinearity ($\mathrm{P}_-^{\sigma}$) for $\mathrm {supp}\,\nu \subset \Sigma$ and $\mathrm {supp}\,\nu \subset \partial \Omega \setminus \Sigma$ respectively. The proofs are in the spirit of [Reference Bidaut-Véron, Hoang, Nguyen and Véron5], requiring several sharp estimates to adapt nontrivially an abstract result in [Reference Kalton and Verbitsky23] to our setting. Our theorems extend the existence results in [Reference Bidaut-Véron, Hoang, Nguyen and Véron5, Reference Bidaut-Véron and Vivier6, Reference Gkikas and Nguyen18, Reference Nguyen33] and can be regarded as a counterpart of the results in [Reference Gkikas and Nguyen20].

It is worth pointing out that the optimal Hardy constant $C_{\Omega,\Sigma }$ defined in (1.2), as well as the asymptotic behaviour of the first eigenfunction $\phi _{\mu,\Sigma }$ in (1.4), the Green function and the Martin kernel, are different from those in the case where the potentials blow up on subsets of $\Omega$. As a result, the critical exponents for the existence of a solution to (P₊) and ($\mathrm{P}_-^{\sigma}$) and the employed capacities are different from those in the [Reference Gkikas and Nguyen19, Reference Gkikas and Nguyen20].

Organization of the paper. In § 3, we quote two-sided estimates of the Green function and the Martin kernel from [Reference Barbatis, Gkikas and Tertikas3], recall the representation theorem and results for linear and semilinear equations with an absorption established in [Reference Barbatis, Gkikas and Tertikas3]. In § 4, we give the definition of the $L_\mu$-harmonic measures and show identities regarding the Poisson kernel and Martin kernel. Section 5 is devoted to the derivation of various results for equation (E ₊) such as a prior estimate, removable singularities (theorems 2.1 and 2.2) and existence results (theorems 2.5 and 2.6). In § 6, we demonstrate necessary and sufficient conditions for the existence of a weak solution to ($\mathrm{P}_-^{\sigma}$) (theorems 2.7 and 2.8). Finally, in appendix A, we provide the local representation of $\Sigma$ and $\Omega$ and construct a barrier function for solutions under assumption that $\Omega$ is a $C^3$ bounded domain.

Notations. We denote by $c,c_1,C...$ the constants which depend on initial parameters and may change from one appearance to another. The notation $A \gtrsim B$ (resp. $A \lesssim B$) means $A \geq c\,B$ (resp. $A \leq c\,B$) where $c$ is a positive constant depending on some initial parameters. If $A \gtrsim B$ and $A \lesssim B$, we write $A \approx B$. Throughout the paper, most of the implicit constants depend on some (or all) of the initial parameters such as $N,\Omega,\Sigma,k,\mu$ and we will omit these dependencies in the notations (except when it is necessary). For a set $D \subset {\mathbb {R}}^N$, ${\mathbb 1}_D$ denotes the indicator function of $D$.

3.1. Two-sided estimates on Green function and Martin kernel

In this subsection, we recall sharp two-sided estimates on the Green function $G_\mu$ and the Martin kernel $K_\mu$ associated to $-L_\mu$ in $\Omega$, as well as the representation formula for nonnegative $L_\mu$-harmonic functions.

Recall that the Green operator and Martin operator are respectively defined by

$$\begin{align*} & \mathbb{G}_\mu[\tau](x)=\int_{\Omega }G_{\mu}(x,y) \, \mathrm{d}\tau(y), \quad \tau \in \mathfrak M(\Omega;\phi_{\mu,\Sigma}), \\ & \mathbb{K}_\mu[\nu](x)=\int_{\partial\Omega }K_{\mu}(x,y)\,\mathrm{d}\nu(y), \quad \nu\in \mathfrak{M}(\partial\Omega). \end{align*}$$

A function $u \in L_{\mathrm {loc}}^1(\Omega )$ is called an $L_\mu$-harmonic function in $\Omega$ if $L_\mu u = 0$ in the sense of distributions in $\Omega$.

Next we state the representation theorem which provides a bijection between the class of positive $L_\mu$-harmonic functions in $\Omega$ and the measure space $\mathfrak M^+(\partial \Omega )$.

3.2. Boundary value problems for linear equations and semilinear equations

We recall the existence, uniqueness and Kato-type inequalities for solutions to boundary value problems for linear equations.

Theorem 3.4 [Reference Barbatis, Gkikas and Tertikas3, theorem 2.12]

Let $\tau \in \mathfrak {M}(\Omega ;\phi _{\mu,\Sigma })$ and $\nu \in \mathfrak {M}(\partial \Omega )$. Then there exists a unique weak solution $u\in L^1(\Omega ;\phi _{\mu,\Sigma })$ of

\[ \left\{ \begin{array}{@{}ll} - L_\mu u=\tau & \text{in }\;\Omega,\\ \mathrm {tr}_{\mu,\Sigma}(u)=\nu, \end{array} \right. \]

in the sense

\[ - \int_{\Omega}u L_{\mu }\xi \,\mathrm{d} x=\int_{\Omega } \xi \,\mathrm{d} \tau - \int_{\Omega} \mathbb{K}_{\mu}[\nu]L_{\mu }\xi \, \mathrm{d} x\quad\forall \xi \in\mathbf{X}_\mu(\Omega), \]

where ${\bf X}_\mu (\Omega )$ has been defined in (2.8). Furthermore

\[ u=\mathbb{G}_{\mu}[\tau]+\mathbb{K}_{\mu}[\nu] \quad \text{a.e. in } \Omega, \]

and for any $\zeta \in \mathbf {X}_\mu (\Omega ),$ there holds

\[ \|u\|_{L^1(\Omega;\phi_{\mu,\Sigma})} \leq \frac{1}{\lambda_\mu}\| \tau \|_{\mathfrak M(\Omega;\phi_{\mu,\Sigma})} + C \| \nu \|_{\mathfrak M(\partial\Omega)}, \]

where $C=C(N,\Omega,\Sigma,\mu )$. In addition, if $\mathrm {d} \tau =f\mathrm {d} x+\mathrm {d}\rho$ with $\rho \in \mathfrak {M}(\Omega ;\phi _{\mu,\Sigma })$ and $f\in L^1(\Omega ;\phi _{\mu,\Sigma })$, then, for any $0 \leq \zeta \in \mathbf {X}_\mu (\Omega )$, the following Kato-type inequalities are valid

(3.5)\begin{equation} \begin{aligned} & -\int_{\Omega}|u|L_{\mu }\zeta \, \mathrm{d} x\leq \int_{\Omega}\mathrm{sign}(u)f\zeta\, \mathrm{d} x +\int_{\Omega}\zeta \, \mathrm{d}|\rho|-\int_{\Omega}\mathbb{K}_{\mu}[|\nu|] L_{\mu }\zeta \, \mathrm{d} x,\\ & -\int_{\Omega}u^+L_{\mu }\zeta \, \mathrm{d} x\leq \int_{\Omega} \mathrm{sign}^+(u)f\zeta\, \mathrm{d} x +\int_{\Omega }\zeta\, \mathrm{d}\rho^+{-}\int_{\Omega}\mathbb{K}_{\mu}[\nu^+]L_{\mu }\zeta \,\mathrm{d} x. \end{aligned} \end{equation}

Here $u^+=\max \{u,0\}$.

Proposition 3.5 [Reference Barbatis, Gkikas and Tertikas3, theorem 9.7]

Let $\nu \in \mathfrak {M}(\partial \Omega )$ and $g\in C(\mathbb {R})$ be a nondecreasing function such that $g(0)=0$ and $g(\mathbb {K}_{\mu }[\nu _+]),g(\mathbb {K}_{\mu }[\nu _-])\in L^1(\Omega ;\phi _{\mu,\Sigma }).$ Then there exists a unique weak solution $u\in L^1(\Omega ;\phi _{\mu,\Sigma })$ of

\[ \left\{ \begin{array}{ll} - L_\mu u+g(u)=0 & \text{in }\;\Omega,\\ \mathrm {tr}_{\mu,\Sigma}(u)=\nu, & \end{array} \right. \]

in the sense that $g(u) \in L^1(\Omega ;\phi _{\mu,\Sigma })$ and

\[ - \int_{\Omega}u L_{\mu }\zeta \, \mathrm{d} x+ \int_{\Omega}g(u)\zeta \, \mathrm{d} x={-} \int_{\Omega} \mathbb{K}_{\mu}[\nu]L_{\mu }\zeta \, \mathrm{d} x\quad\forall \zeta \in\mathbf{X}_\mu(\Omega). \]

4.1. $L_\mu$-harmonic measures

Let $h\in C(\partial \Omega )$. Then by [Reference Barbatis, Gkikas and Tertikas3, lemma 6.8], there exists a unique solution $v_h$ of the Dirichlet problem

(4.1)\begin{equation} \left\{ \begin{array}{@{}ll} L_{\mu}v=0 & \text{in}\ \Omega\\ v=h\quad \text{on}\ \partial\Omega. \end{array} \right. \end{equation}

Let $\tilde W$ be as in (2.4). The boundary value condition in (4.1) is understood as

(4.2)\begin{equation} \lim_{\mathrm{dist}\,(x,F)\to 0}\frac{v(x)}{\tilde W(x)}=h \quad \text{for every compact set }\ F\subset \partial \Omega. \end{equation}

Let $z \in \Omega$ and set ${\mathcal {L}}_{\mu,z}(h):=u_h(z),$ then the mapping $h\mapsto {\mathcal {L}}_{\mu,z}(h)$ is a linear positive functional on $C(\partial \Omega )$. Thus, there exists a unique Borel measure on $\partial \Omega$, called $L_{\mu }$-harmonic measure on $\partial \Omega$ relative to $z$ and denoted by $\omega _{\Omega }^{z}$, such that

\[ v_{h}(z)=\int_{\partial\Omega}h(y) \, \mathrm{d}\omega_{\Omega}^{z}(y). \]

Let $x_0 \in \Omega$ be a fixed reference point. Let $\{\Omega _n\}$ be a $C^2$ exhaustion of $\Omega$, i.e. $\{\Omega _n\}$ is an increasing sequence of bounded $C^2$ domains such that

\[ \overline{\Omega}_n\subset \Omega_{n+1}, \quad \cup_n\Omega_n=\Omega, \quad \mathcal{H}^{N-1}(\partial \Omega_n)\to \mathcal{H}^{N-1}(\partial \Omega), \]

where $\mathcal {H}^{N-1}$ denotes the $(N-1)$-dimensional Hausdorff measure in ${\mathbb {R}}^N$.

Then $-L_\mu$ is uniformly elliptic and coercive in $H^1_0(\Omega _n)$ and its first eigenvalue $\lambda _{\mu,\Sigma }^{\Omega _n}$ in $\Omega _n$ is larger than its first eigenvalue $\lambda _{\mu,\Sigma }$ in $\Omega$.

For $h\in C(\partial \Omega _n)$, the following problem

\[ \left\{ \begin{array}{@{}ll} -L_{\mu } v=0 & \text{in } \Omega_n\\ v=h & \text{on } \partial \Omega_n, \end{array} \right. \]

admits a unique solution which allows to define the $L_{\mu }$-harmonic measure $\omega _{\Omega _n}^{x_0}$ on $\partial \Omega _n$ by

\[ v(x_0)={\displaystyle \int_{\partial \Omega_n}^{}}h(y) \,\mathrm{d}\omega^{x_0}_{\Omega_n}(y). \]

Let $G^{\Omega _n}_\mu (x,y)$ be the Green kernel of $-L_\mu$ on $\Omega _n$. Then $G^{\Omega _n}_\mu (x,y)\uparrow G_\mu (x,y)$ for $x,y\in \Omega, x \neq y$.

Proposition 4.1 [Reference Barbatis, Gkikas and Tertikas3, proposition 7.7]

Assume $x_0\in \Omega _1$. Then for every $Z\in C(\overline {\Omega })$,

\[ \lim_{n\rightarrow\infty}\int_{\partial \Omega_n} Z(x)\tilde{W}(x)\, \mathrm{d}\omega^{x_0}_{\Omega_n}(x)=\int_{\partial \Omega}Z(x)\, \mathrm{d}\omega^{x_0}_\Omega(x). \]

4.2. Poisson kernel

By the standard elliptic theory, we can easily show that for any $x \in \Omega$, $G_{\mu }(x,\cdot ) \in C^{1,\gamma }(\overline {\Omega } \setminus (\Sigma \cup \{x\})) \cap C^2(\Omega \setminus \{x\})$ for all $\gamma \in (0,1)$. Therefore, we may define the Poisson kernel associated to $-L_\mu$ in $\Omega \times (\partial \Omega \setminus \Sigma )$ as

(4.3)\begin{equation} P_\mu(x,y):={-}\frac{\partial G_\mu}{\partial{\bf n}}(x,y), \quad x \in \Omega , \; y \in \partial \Omega\setminus\Sigma, \end{equation}

where ${\bf n}$ is the unit outer normal vector of $\partial \Omega$. This kernel satisfies the following properties.

Proof.

1. (i) We note that $P_\mu (\cdot,y)$ is $L_\mu$-harmonic in $\Omega$ and

   \[ \lim_{x\in\Omega,\;x\rightarrow\xi}\frac{P_\mu(x,y)}{\tilde W(x)}=0\quad\text{for all } \xi\in \partial\Omega\setminus \{y\}\quad\text{and}\quad y\in\partial\Omega\setminus\Sigma. \]
   Hence, $\frac {P_\mu (x,y)}{P_\mu (x_0,y)}$ is a kernel function with pole at $y$ and basis at $x_0$ in the sense of [Reference Barbatis, Gkikas and Tertikas3, definition 2.7]. This, together with the fact that any kernel function with pole at $y$ and basis at $x_0$ is unique (see [Reference Barbatis, Gkikas and Tertikas3, proposition 7.3]), implies (4.4).

2. (ii) Let $\zeta \in C(\partial \Omega )$ with compact support in $\partial \Omega \setminus \Sigma$ such that $\mathrm {dist}\,(\mathrm {supp}\, \zeta,\Sigma )=r>0$. Let $Z\in C(\overline {\Omega })$ be such that $Z(y)=\zeta (y)$ for any $y\in \partial \Omega$ and $Z(y)=0$ in $\Sigma _{\frac {r}{2}}.$ Set $r_0=\frac {1}{4}\min \{\beta _2,r\}$ where $\beta _2$ is the constant in (A.7). We consider a decreasing sequence of bounded $C^2$ domains $\{\Sigma _n\}$ such that

   (4.6)\begin{equation} \Sigma \subset \Sigma_{n+1}\subset\overline{\Sigma}_{n+1}\subset \Sigma_{n}\subset\overline{\Sigma}_{n} \subset\Sigma_{\frac{r_0}{4}}, \quad \cap_n \Sigma_n=\Sigma. \end{equation}

   Let $\phi _*$ be the unique solution of

   (4.7)\begin{equation} \left\{ \begin{array}{@{}ll} -L_{\mu }\phi_*=0 & \text{in } \Omega\\ \phi_* =1 & \text{on } \partial\Omega, \end{array} \right. \end{equation}
   where the boundary condition in (4.7) is understood as
   \[ \lim_{\mathrm{dist}\,(x,F)\to 0}\frac{\phi_*(x)}{\tilde W(x)}=1 \quad \text{for every compact set } \; F\subset \partial \Omega. \]
   Then, by [Reference Barbatis, Gkikas and Tertikas3, lemma 6.8 and estimate (6.21)], there exist constants $c_1=c_1(\Omega,\Sigma,\Sigma _n,\mu )$ and $c_2=c_2(\Omega,\Sigma,N,\mu )$ such that $0< c_1 \leq \phi _*(x)\leq c_2 d_\Sigma (x)^{-{\alpha _{+}}}$ for all $x \in \Omega \setminus \Sigma _n$. By the standard elliptic theory, $\phi _*\in C^2(\Omega ) \cap C^{1,\gamma }(\overline {\Omega }\setminus \Sigma )$ for any $0<\gamma <1$.

Now, for any $\eta \in C(\partial \Omega ),$ we can easily show that $u_\eta$ is a solution of

\[ \left\{ \begin{array}{@{}ll} -L_{\mu }v=0 & \text{in } \Omega\setminus\Sigma_n\\ v=\eta & \text{on } \partial(\Omega\setminus \Sigma_n) \end{array} \right. \]

if and only if $w_\eta =\frac {u_\eta }{\phi _*}$ is a solution of

\[ \left\{ \begin{array}{@{}ll} -\text{div}(\phi_*^2\nabla w)=0 & \text{in } \Omega\setminus\Sigma_n\\ w=\dfrac{\eta}{\phi_*} & \text{on } \partial( \Omega\setminus\Sigma_n). \end{array} \right. \]

Since the operator $L_{\phi _*}w:=-\text {div}(\phi _*^2\nabla w)$ is uniformly elliptic and has smooth coefficients, we may deduce the existence of $L_\mu$-harmonic measure $\omega _n^{x}$ on $\partial ( \Omega \setminus \Sigma _n)$ and the Green kernel $G^n_{\mu }$ of $-L_\mu$ in $\Omega \setminus \Sigma _n$.

Let $v_n$ be the unique solution of

(4.8)\begin{equation} \left\{ \begin{array}{@{}ll} -L_{\mu }v=0 & \text{in } \Omega\setminus\Sigma_n\\ v=Z\tilde W & \text{on } \partial(\Omega\setminus \Sigma_n). \end{array} \right. \end{equation}

Then by the representation formula we have

\[ v_n(x)=\int_{\partial( \Omega\setminus\Sigma_n)}Z\tilde W\, \mathrm{d}\omega_n^x(y)=\int_{\partial\Omega\cap\mathrm{supp}\, \zeta}\zeta\tilde W\, \mathrm{d}\omega_n^x(y). \]

Proceeding as in the proof of [Reference Barbatis, Gkikas and Tertikas3, proposition 7.7], we may show that

\[ v_n(x)\rightarrow \int_{\partial\Omega\cap\mathrm{supp}\, \zeta } \zeta(y) \,\mathrm{d} \omega^{x}_{\Omega}(y)=:v(x) \quad \text{as } n \to \infty. \]

On the other hand, the Poisson kernel $P_\mu ^n$ of $-L_\mu$ in $\Omega \setminus \Sigma _n$ is well defined and given by

\[ P_\mu^n(x,y)={-}\frac{\partial G_\mu^n}{\partial {\bf n}^n}(x,y), \quad x \in \Omega\setminus \Sigma_n, \, y \in \partial (\Omega\setminus\Sigma_n), \]

where ${\bf n}^n$ is the unit outer normal vector to $\partial (\Omega \setminus \Sigma _n)$. Hence,

$$\begin{align*} v_n(x) & = \int_{\partial(\Omega\setminus\Sigma_n)}P_\mu^n(x,y)Z(y)\tilde W(y)\,\mathrm{d} S_{\partial \Omega}(y)\\ & =\int_{\partial\Omega\cap\mathrm{supp}\, \zeta }P_\mu^n(x,y)\zeta(y)\tilde W(y)\,\mathrm{d} S_{\partial \Omega}(y), \end{align*}$$

where $S_{\partial \Omega }$ is the $(N-1)$-dimensional surface measure on $\partial (\Omega \setminus \Sigma _n)$. Combining all above, we obtain

(4.9)\begin{equation} \int_{\partial \Omega\cap\mathrm{supp}\, \zeta}\zeta(y)\, \mathrm{d} \omega_n^{x}(y)=\int_{\partial \Omega\cap\mathrm{supp}\, \zeta}P_\mu^n(x,y)\tilde W(y)\zeta(y)\, \mathrm{d} S_{\partial \Omega}(y). \end{equation}

Put $\beta =\frac {1}{2}\min \{d_{\partial \Omega }(x), r_0\}$. Since $G^n_{\mu }(x,y)\nearrow G_{\mu }(x,y)$ for any $x\neq y$ and $x,y\in \Omega,$ $\{G_{\mu }^n(x,\cdot )\}_n$ is uniformly bounded in $W^{2,\kappa }(\Omega _\beta \setminus \Sigma _{r_0})$ for any $\kappa >1$. Thus, by the standard compact Sobolev embedding, there exists a subsequence, still denoted by index $n$, which converges to $G_{\mu }(x,\cdot )$ in $C^1(\overline {\Omega _\beta \setminus \Sigma _{r_0}})$ as $n \to \infty$. This implies that $P_\mu ^n(x,\cdot ) \to P_\mu (x,\cdot )$ uniformly on $\partial \Omega \setminus \Sigma _{r_0}$ as $n \to \infty$.

Therefore, by letting $n \to \infty$ in (4.9), we obtain

(4.10)$$\begin{align} \int_{\partial\Omega} \zeta(y) \,\mathrm{d} \omega^{x}_{\Omega}(y)& =\lim_{n\to\infty}\int_{\partial\Omega} \zeta(y) \tilde W(y) \,\mathrm{d} \omega^{x}_n(y) \nonumber\\ & =\lim_{n\to\infty}\int_{\partial\Omega} P_{\mu}^n(x,y)\zeta(y)\tilde W(y) \, \mathrm{d} S_{\partial \Omega}(y)\nonumber\\ & =\int_{\partial\Omega} P_{\mu}(x,y)\zeta(y)\tilde W(y) \,\mathrm{d} S_{\partial \Omega}(y). \end{align}$$

By (4.10) and the fact that $\inf _{y \in \partial \Omega \setminus \Sigma _{r}}P_\mu (x_0,y)>0\quad \forall \;r>0,$ we deduce that

\[ \omega^{x}_{\Omega}(E)=\mathbb{P}_\mu[{\mathbb 1}_E\tilde W](x) \]

for any Borel set $E\subset \overline {E}\subset \partial \Omega \setminus \Sigma$. This implies in particular that $\omega _{\Omega }^{x_0}$ and $S_{\partial \Omega }$ are mutually absolutely continuous with respect to compact subsets of $\partial \Omega \setminus \Sigma$.

Now, assume $0 \leq h \in L^1(\partial \Omega ; \mathrm {d}\omega _{\Omega }^{x_0})$ has compact support in $\partial \Omega \setminus \Sigma$ and $\mathrm {dist}\,(\mathrm {supp}\, h, \Sigma )=4r>0$. Then there exists a sequence of nonnegative functions $\{ h_n \} \subset C(\partial \Omega )$ with compact support in $\partial \Omega \setminus \Sigma$ such that $\mathrm {dist}\,(\mathrm {supp}\, h_n,\Sigma )=2r>0$ for any $n \in {\mathbb {N}}$ and $h_n \to h$ in $L^1(\partial \Omega ; \mathrm {d}\omega _{\Omega }^{x_0})$ as $n \to \infty$.

Applying (4.10) with $\zeta$ replaced by $|h_n-h_m|$ for $m, n \in {\mathbb {N}}$ and using the fact that $\inf _{y \in \Sigma _r \cap \partial \Omega }(P_\mu (x_0,y) \tilde W(y))>0$, we have

$$\begin{align*} \int_{\partial \Omega} |h_m(y)-h_n(y)|\, \mathrm{d}\omega_{\Omega}^{x_0}(y) & =\int_{\partial \Omega} P_{\mu}(x_0,y)|h_n(y)-h_n(y)| \tilde W(y) \, \mathrm{d} S_{\partial \Omega}(y) \\ & \geq c\int_{\partial \Omega} |h_n(y)-h_n(y)| \, \mathrm{d} S_{\partial \Omega}(y). \end{align*}$$

This implies that $\{h_n\}$ is a Cauchy sequence in $L^1(\partial \Omega )$. Therefore, there exists $\tilde h \in L^1(\partial \Omega )$ such that $h_n \to \tilde h$ in $L^1(\partial \Omega )$. Since $\omega _{\Omega }^{x_0}$ and $S_{\partial \Omega }$ are mutually absolutely continuous with respect to compact subsets of $\partial \Omega \setminus \Sigma$ and $h$ and $\tilde h$ have compact support in $\partial \Omega \setminus \Sigma$, we deduce that $h=\tilde h$ $\omega _{\Omega }^{x_0}$-a.e. and $S_{\partial \Omega }$-a.e. in $\partial \Omega$. In particular, $h \in L^1(\partial \Omega )$.

Applying (4.10) with $\zeta$ replaced by $h_n$, for any $n \in {\mathbb {N}}$, we have

(4.11)\begin{equation} \int_{\partial \Omega} h_n(y) \, \mathrm{d}\omega_{\Omega}^{x_0}(y)=\int_{\partial \Omega} P_{\mu}(x_0,y)h_n(y) \tilde W(y) \,\mathrm{d} S_{\partial \Omega}(y). \end{equation}

By letting $n \to \infty$ in (4.11), we obtain (4.5).

Next, we assume $h \in L^1(\partial \Omega ; \mathrm {d}\omega _{\Omega }^{x_0})$ and drop the assumption that $h \geq 0$, then we write $h=h_+ - h_-$ where $h_\pm \in L^1(\partial \Omega ; \mathrm {d}\omega _{\Omega }^{x_0})$. By applying (4.5) for $h_\pm$, we deduce that (4.5) holds true for $h \in L^1(\partial \Omega ; \mathrm {d}\omega _{\Omega }^{x_0})$. Moreover, we can show that $h_\pm \in L^1(\partial \Omega )$, which implies $h \in L^1(\partial \Omega )$.

5.1. Keller–Osserman estimates

In this section, we prove Keller–Osserman type estimates for nonnegative solutions of equation (E ₊).

Lemma 5.1 Assume $p>1$. Let $u\in C^2(\Omega )$ be a nonnegative solution of equation (E ₊). Then there exists a positive constant $C=C(\Omega,\Sigma,\mu,p)$ such that

(5.1)\begin{equation} 0 \leq u(x) \leq Cd_{\partial \Omega}(x)^{-\frac{2}{p-1}},\quad\forall x\in\Omega.\end{equation}

Proof. Let $\beta _0$ be as in § A.1 and $\eta _{\beta _0} \in C_c^\infty ({\mathbb {R}}^N)$ such that

\[ 0\leq\eta_{\beta_0}\leq1, \quad \eta_{\beta_0}=1 \text{ in } \overline{\Omega}_{\frac{\beta_0}{4}} \quad \text{and} \quad \mathrm{supp}\,\eta_{\beta_0} \subset \Omega_{\frac{\beta_0}{2}}, \]

where $\Omega _{\epsilon }$ is defined in (2.2). For $\varepsilon \in \left (0,\frac {\beta _0}{16}\right )$, we define

\[ V_\varepsilon:=1-\eta_{\beta_0}+\eta_{\beta_0}(d_{\partial \Omega}-\varepsilon)^{-\frac{2}{p-1}} \quad \text{in } \Omega \setminus \overline{\Omega}_\varepsilon. \]

Then $V_\varepsilon \geq 0$ in $\Omega \setminus \overline {\Omega }_\epsilon$. It can be checked that there exists $C=C(\Omega,\beta _0,\mu,p)>1$ such that the function $W_\varepsilon :=CV_\varepsilon$ satisfies

(5.2)\begin{equation} -L_\mu W_\varepsilon+W_\varepsilon^p = C({-}L_\mu V_\varepsilon + C^{p-1}V_\varepsilon^p) \geq 0 \quad \text{in } \Omega\setminus \overline{\Omega}_\varepsilon. \end{equation}

Combining (E ₊) and (5.2) yields

(5.3)\begin{equation} -L_\mu (u-W_\varepsilon) + u^{p} - W_\varepsilon^p \leq 0 \quad \text{in } \Omega \setminus \overline{\Omega}_\varepsilon. \end{equation}

We see that $(u-W_\varepsilon )^+\in H_0^1(\Omega \setminus \overline {\Omega }_\varepsilon )$ and $(u-W_\varepsilon )^+$ has compact support in $\Omega \setminus \overline {\Omega }_\varepsilon$. By using $(u-W_\varepsilon )^+$ as a test function for (5.3), we deduce that

$$\begin{align*} 0 & \geq \int_{\Omega\setminus \Omega_\varepsilon}|\nabla (u-W_\varepsilon)^+|^2 \mathrm{d} x - \mu\int_{\Omega\setminus \Omega_\varepsilon}\frac{[(u-W_\varepsilon)^+]^2}{d_\Sigma^2} \mathrm{d} x\\ & \quad +\int_{\Omega\setminus \Omega_\varepsilon}(u^{p}-W_\varepsilon^p)(u-W_\varepsilon)^+\,\mathrm{d} x\\ & \geq \int_{\Omega\setminus \Omega_\varepsilon}|\nabla (u-W_\varepsilon)^+|^2\,\mathrm{d} x - \mu\int_{\Omega\setminus \Omega_\varepsilon}\frac{[(u-W_\varepsilon)^+]^2}{d_\Sigma^2}\,\mathrm{d} x \\ & \quad \geq \lambda_{\mu,\Sigma}\int_{\Omega\setminus \Omega_\varepsilon}| (u-W_\varepsilon)^+|^2\,\mathrm{d} x. \end{align*}$$

This and the assumption $\lambda _{\mu,\Sigma } >0$ imply $(u-W_\varepsilon )^+ = 0$, whence $u\leq W_\varepsilon$ in $\Omega \setminus \overline {\Omega }_\varepsilon$. Letting $\varepsilon \to 0$, we obtain the desired result.

The following theorem is the main tool in the study of the boundary removable singularities for nonnegative solutions of equation (E ₊). We assume additionally that $\Omega$ is a $C^3$ domain which is needed to apply proposition A.2.

Theorem 5.2 Let $p>1,$ $F\subset \Sigma$ be a compact subset of $\Sigma$ and $d_F(x)=\mathrm {dist}\,(x,F)$. We additionally assume that $\Omega$ is a $C^3$ bounded domain. If $u\in C^2(\Omega )$ is a nonnegative solution of (E ₊) satisfying

(5.4)\begin{equation} \lim_{x\in\Omega,\;x\rightarrow\xi}\frac{u(x)}{\tilde W(x)}=0\quad\forall \xi\in \partial\Omega\setminus F,\quad \text{locally uniformly in } \partial\Omega\setminus F, \end{equation}

then there exists a positive constant $C=C(N,\Omega,\Sigma,\mu,p)$ such that

(5.5)$$\begin{align} & u(x)\leq Cd_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{-}}}d_F(x)^{-\frac{2}{p-1}+{\alpha_{-}}-1}\quad\forall x\in \Omega, \end{align}$$

(5.6)$$\begin{align} & |\nabla u(x)|\leq Cd_\Sigma(x)^{-{\alpha_{-}}} d_F(x)^{-\frac{2}{p-1}+{\alpha_{-}}-1}\quad\forall x\in \Omega. \end{align}$$

Proof. The proof is in the spirit of [Reference Marcus and Véron30, proposition 3.4.3]. Let $\beta _5$ be the positive constant defined in proposition A.2. Let $\xi \in (\Sigma _{\beta _5}\cap \partial \Omega )\setminus F$ and put $d_{F,\xi } = \frac {1}{16} d_F(\xi )<1$. Denote

\[ \Omega^\xi:= \frac{1}{d_{F,\xi}}\Omega = \{y\in\mathbb{R}^N:\;d_{F,\xi}\,y\!\in\!\Omega\}\quad\text{and}\quad \Sigma^\xi:=\frac{1}{d_{F,\xi}}\Sigma =\{y\!\in\!\mathbb{R}^N:\;d_{F,\xi}\,y\!\in\! \Sigma\}. \]

If $u$ is a nonnegative solution of (E ₊) in $\Omega$ then the function

\[ u^\xi (y): = d_{F,\xi}^{\frac{2}{p-1}} u(d_{F,\xi}y),\quad y\in\Omega^\xi, \]

is a nonnegative solution of

(5.7)\begin{equation} -\Delta u^\xi- \frac{\mu}{|\mathrm{dist}(y,\Sigma^\xi)|^2} u^\xi +\left(u^\xi\right)^p=0 \end{equation}

in $\Omega ^\xi.$

Now we note that $u^\xi$ is a nonnegative $L_\mu$ subharmonic function and satisfies [by (5.4)]

\[ \lim_{y\in\Omega^\xi,\;y\rightarrow P}\frac{u^\xi(y)}{\tilde W^\xi(y)}=0\quad\forall P\in B\left(\frac{1}{d_{F,\xi}}\xi,2\right) \cap \partial\Omega^\xi, \]

where

\[ \tilde W^\xi(y) = 1-\eta_{\frac{\beta_3}{d_{F, \xi}}}+\eta_{\frac{\beta_3}{d_{F, \xi}}}W^\xi(y) \quad \text{in } \Omega^\xi \setminus \Sigma^\xi, \]

$\eta _{\frac {\beta _3}{d_{F, \xi }}}$ is the scaled version of the function $\eta _{\beta _3}$ defined before (2.4), and

\[ W^\xi(y)=\left\{ \begin{array}{@{}ll}(d_{\partial \Omega^\xi}(y)+d_{\Sigma^\xi}(y))^{2}d_{\Sigma^\xi}(y)^{-{\alpha_{+}}} & \text{if}\ \mu < H^2, \\ (d_{\partial \Omega^\xi}(y)+d_{\Sigma^\xi}(y))^{2}d_{\Sigma^\xi}(y)^{{-}H}|\ln d_{\Sigma^\xi}(y)| & \text{if}\ \mu =H^2, \end{array} \right. \quad x \in \Omega^\xi \setminus \Sigma^\xi. \]

Set $R_0=\min \{\beta _5,1\}$. In view of the proof of [Reference Barbatis, Gkikas and Tertikas3, lemma 6.2 and estimate (6.7)], there exists a positive constant $c$ depending on $\Omega,\Sigma,\mu$ and

(5.8)\begin{equation} \int_{B\left(\frac{1}{d_F(\xi)}\xi,2R_0\right)\cap\Omega^\xi} u^\xi(y)\,d_{\partial \Omega^\xi}(y)\,d_{\Sigma^\xi}(y)^{-{\alpha_{-}}}\,\mathrm{d} y \end{equation}

such that

(5.9)\begin{equation} u^\xi(y) \leq c\,d_{\partial \Omega^\xi}(y)\,d_{\Sigma^\xi}(y)^{-{\alpha_{-}}}\quad \forall y\in B\left(\frac{1}{d_F(\xi)}\xi,R_0\right)\cap\Omega^\xi . \end{equation}

Let $r_0=\frac {R_0}{16}$ and let $w_{r_0,\xi }$ be the supersolution of (5.7) in $\mathscr {B}(\frac {1}{d_{F,\xi }}\xi,r_0)\cap \Omega ^\xi$ constructed in proposition A.2 with $R=r_0$ and $z=\frac {1}{d_{F,\xi }}\xi$.

Taking into account of (5.9) and using a similar argument as in the proof of lemma 5.1, we can show that

\[ u^\xi(y) \leq w_{r_0,\xi}(y)\quad \forall y\in \mathscr{B}\left(\frac{1}{d_{F,\xi}}\xi,r_0 \right)\cap\Omega^\xi. \]

By (5.8), (5.9) and the above inequality, we may deduce that

(5.10)\begin{equation} u^\xi(y) \leq c\,d_{\partial \Omega^\xi}(y)\,d_{\Sigma^\xi}(y)^{-{\alpha_{-}}}\quad \forall y\in B\left(\frac{1}{d_F(\xi)}\xi,\frac{r_0}{16}\right)\cap\Omega^\xi, \end{equation}

where $c$ depends only on $\Omega,\Sigma,\mu,p,$ the $C^3$ characteristic of $\Omega ^\xi$ and the $C^2$ characteristic of $\Sigma ^\xi$. As $d_{F, \xi } \leq 1$ the $C^3$ characteristic of $\Omega$ (respectively the $C^2$ characteristic of $\Sigma$) is also a $C^3$ characteristic of $\Omega ^\xi$ (respectively a $C^2$ characteristic of $\Sigma ^\xi$), therefore this constant $c$ can be taken to be independent of $\xi$. Thus, for any $\xi \in (\Sigma _{\beta _5}\cap \partial \Omega )\setminus F$ such that $d_{F}(x)\leq 16$, there holds

(5.11)\begin{equation} u(x)\leq c\,d_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{-}}}d_{F}(\xi)^{-\frac{2}{p-1}+{\alpha_{-}}-1}\quad\forall x\in B\left(\xi,r_1d_F(\xi)\right)\cap \Omega, \end{equation}

where $r_1=\frac {r_0}{16^2}.$

Now, we consider three cases.

Case 1: $x\in \Sigma _{\frac {r_1}{32}}\cap \Omega$ and $d_F(x) < 1$. If $d_{\partial \Omega }(x)\leq \frac {r_1}{8+r_1}d_F(x)$ then there exits a unique point in $\xi \in \partial \Omega \setminus F$ such that $|x-\xi |=d_{\partial \Omega }(x)$. Hence,

(5.12)\begin{equation} d_F(\xi)\leq d_{\partial \Omega}(x)+d_F(x) \leq 2\frac{4+r_1}{8+r_1}d_F(x)<16, \end{equation}

$d_F(x)\leq \frac {8+r_1}{8}d_{F}(\xi )$ and $d_{\partial \Omega }(x)\leq \frac {r_1}{8}d_{F}(\xi )$. This, combined with (5.11), (5.12) and the fact that $d_F(x)\approx d_{F}(\xi )$, yields

\[ u(x)\leq C d_\Sigma(x)^{-{\alpha_{-}}}d_{F}(\xi)^{-\frac{2}{p-1}+{\alpha_{-}}-1} \leq Cd_\Sigma(x)^{-{\alpha_{-}}}d_F(x)^{-\frac{2}{p-1}+{\alpha_{-}}-1}. \]

If $d_{\partial \Omega }(x)> \frac {r_1}{8+r_1}d_F(x)\geq \frac {r_1}{8+r_1}\,d_\Sigma (x)$ then by (5.1) and the fact that $d_{\partial \Omega }(x)\approx d_F(x)\approx d_\Sigma (x)$, we obtain

\[ u(x) \leq Cd_{\partial \Omega}(x)^{-\frac{2}{p-1}} \leq Cd_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{-}}}d_F(x)^{-\frac{2}{p-1}+{\alpha_{-}}-1}. \]

Thus, (5.5) holds for every $x\in \Sigma _{\frac {r_1}{4}}$ such that $d_F(x) <1$.

Case 2: $x\in \Sigma _{\frac {r_1}{32}}\cap \Omega$ and $d_F(x) \geq 1$. Let $\xi$ be the unique point in $\partial \Omega \setminus F$ such that $|x-\xi |=d_{\partial \Omega }(x)$. Since $u$ is an $L_\mu$-subharmonic function in $B(\xi,r_1) \cap \Omega$, in view of the proof of (5.10), we obtain

$$\begin{align*} u(x)& \leq Cd_{\partial \Omega}(x) d_\Sigma(x)^{-{\alpha_{-}}} \leq Cd_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{-}}} d_F(x)^{-\frac{2}{p-1}+{\alpha_{-}}-1}\\ & \quad\forall x\in B\left(\xi,\frac{\beta_3}{2}\right)\cap \Omega , \end{align*}$$

where $C$ depend only on $\Omega,\Sigma,\mu,p.$

Case 3: $x \in \Omega \setminus \Sigma _{\frac {r_1}{32}}$. The proof is similar to the one of [Reference Gkikas and Véron21, proposition A.3] and we omit it. (ii) Let $x_0\in \Omega$. Put $\ell =d_{\partial \Omega }(x_0)$ and

\[ \Omega^{\ell}:=\frac{1}{\ell}\Omega = \{y\in\mathbb{R}^N:\;\ell y\in\Omega\}\quad\text{and}\quad\Sigma^{\ell}:=\frac{1}{\ell}\Sigma = \{y\in\mathbb{R}^N:\;\ell y\in\Sigma\}. \]

If $x\in B\left (x_0,\frac {\ell }{2}\right )$ then $y=\ell ^{-1}x$ belongs to $B\left (y_0,\frac {1}{2}\right ),$ where $y_0=\ell ^{-1}x_0$. Also we have that $\frac {1}{2}\leq d_{\Omega ^\ell }(y)\leq \frac {3}{2}$ and $\frac {1}{2}\leq d_{\Sigma ^\ell }(y)$ for each $y\in B\left (y_0,\frac {1}{2}\right )$. Set $v(y)=u(\ell y)$ for $y\in B(y_0,\frac {1}{2})$ then $v$ satisfies

\[ -\Delta v- \frac{\mu}{d_{\Sigma^\ell}^2} v + \ell^2 \left|v\right|^p=0\quad\mathrm{in}\ B\left(y_0,\frac{1}{2}\right). \]

By the standard elliptic estimates and (5.1) we have

\[ \sup_{y\in B\left(y_0,\frac{1}{4}\right)}|\nabla v(y)|\leq C\sup_{y\in B\left(y_0,\frac{1}{3}\right)}|v(y)|\leq C v(y_0), \]

This, together with the equality $\nabla v(y)=\ell \nabla u(x)$, estimate (5.5) implies

$$\begin{align*} |\nabla u(x_0)|& \leq C\ell^{{-}1} d_{\partial \Omega}(x_0) d_\Sigma^{-{\alpha_{-}}}(x_0)\,d_F(x_0)^{-\frac{2}{p-1}+{\alpha_{-}}-1}\\ & \leq C d_\Sigma(x_0)^{-{\alpha_{-}}}d_F(x_0)^{-\frac{2}{p-1}+{\alpha_{-}}-1}. \end{align*}$$

Therefore, estimate (5.6) follows since $x_0$ is an arbitrary point. The proof is complete.

5.2. Removable boundary singularities

This subsection is devoted to the study of removable boundary singularities for nonnegative solutions of equation (E ₊) in the supercritical range. More precisely we will prove theorems 2.1 and 2.2.

Proof of theorem 2.1 We will only consider the case $\mu < H^2$ and $p=\frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$, since the proof in the other cases is very similar. Let $u$ be a nonnegative solution of (E ₊) satisfying (2.5). By (5.5) with $F=\Sigma$, there holds

(5.13)\begin{equation} u(x)\leq Cd_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{+}}}\quad\forall x\in \Omega, \end{equation}

for some constant $C$ independent of $u$.

Let $\{\Omega _n\}$ be a $C^2$ exhaustion of $\Omega$ and we write

\[ \mathbb{G}_\mu^{\Omega_n}\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](x)=\int_{\Omega_n}G^{\Omega_n}_\mu(x,y)u(y)^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\,\mathrm{d} y. \]

By the representation formula in $\Omega _n,$ we have that

(5.14)\begin{equation} u(x_0)+\mathbb{G}_\mu^{\Omega_n}\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](x_0)=\int_{\partial\Omega_n}u(y)\,\mathrm{d} \omega^{x_0}_{\Omega_n}(y). \end{equation}

By (5.13), the definition of $\tilde W$ in (2.4), estimate (A.9) and proposition 4.1, we deduce

(5.15)\begin{equation} \limsup_{n\to\infty}\int_{\partial\Omega_n}u(y)\,\mathrm{d} \omega^{x_0}_{\Omega_n}(y)\leq C\limsup_{n\to\infty}\int_{\partial\Omega_n}\tilde W(y)\,\mathrm{d} \omega^{x_0}_{\Omega_n}(y)=C\omega^{x_0}_{\Omega}(\partial\Omega). \end{equation}

Since $G^{\Omega _n}_\mu (x,y)\uparrow G_\mu (x,y)$ for $x,y\in \Omega, x \neq y,$ (5.14) and (5.15) yield

\[ \mathbb{G}_\mu[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}](x_0)\leq C\omega^{x_0}_{\Omega}(\partial\Omega). \]

Hence, there exists another positive constant $C$ independent of $u$ such that

(5.16)\begin{equation} \int_\Omega u(y)^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\phi_{\mu,\Sigma}(y)\,\mathrm{d} y\leq C. \end{equation}

Consequently, the function $v=u+\mathbb {G}_\mu [u^\frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}]$ is a nonnegative $L_\mu$-harmonic in $\Omega$. This, together with theorem 3.3, implies the existence of a measure $\nu \in \mathfrak M^+(\partial \Omega )$ such that

(5.17)\begin{equation} u+\mathbb{G}_\mu\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right]=\mathbb{K}_\mu[\nu] \quad\text{in}\;\Omega. \end{equation}

By proposition 4.1 and (5.13), we may deduce that $\nu$ has compact support in $\Sigma.$

Next we will show that $\nu \equiv 0$. Suppose by contradiction that $\nu \not \equiv 0$. Let $1< M\in \mathbb {N}$ and $v_{M,n}$ be the positive solution of

\[ \left\{ \begin{array}{@{}ll} -L_{\mu }^{\Omega_n}v_{M,n} + v_{M,n}^{\dfrac{\alpha_{+}+1}{\alpha_{+}-1}}=0 & \text{in } \Omega_n\\ v_{M,n}=M u & \text{on } \partial \Omega_n. \end{array} \right. \]

Since $M u$ is a supersolution of the above problem, we have that $0\leq v_{M,n}\leq Mu$ in $\Omega _n$ for any $n\in \mathbb {N}.$ As a result, there exists $v_M\in C^2(\Omega )$ such that $v_{M,n}\to v_M$ locally uniformly in $\Omega$ and $L_\mu v_M+ v_{M}^\frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}=0$ in $\Omega$. As $v_M\leq Mu$, it follows that $v_M$ satisfies (2.5) and hence thanks to theorem 5.2, estimate (5.5) holds for $v_M$ with $F=\Sigma$, namely

(5.18)\begin{equation} v_M(x)\leq Cd_{\partial \Omega}(x)\,d_\Sigma(x)^{-{\alpha_{+}}}\quad\forall x\in \Omega, \end{equation}

for some constant $C$ independent of $v_M$. By using an argument similar to the one leading to (5.16), we derive

(5.19)\begin{equation} \int_\Omega v_M(y)^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\phi_{\mu,\Sigma}(y)\,\mathrm{d} y\leq C_0, \end{equation}

for some constant $C_0$ independent of $v_M$. Also, by the representation formula we have

(5.20)\begin{equation} v_{M,n}(x)+\mathbb{G}_\mu^{\Omega_n}\left[v_{M,n}^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](x)=M\int_{\partial\Omega_n}u(y)\,\mathrm{d} \omega^{x}_{\Omega_n}(y), \quad \forall x \in \Omega. \end{equation}

From (5.17), we have

$$\begin{align*} \int_{\partial\Omega_n}u(y)\, \mathrm{d} \omega^{x}_{\Omega_n}(y)& =\int_{\partial\Omega_n}(\mathbb{K}_\mu[\nu](y)-\mathbb{G}_\mu\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](y))\,\mathrm{d} \omega^{x}_{\Omega_n}(y)\\ & =\mathbb{K}_\mu[\nu](x)-\int_{\partial\Omega_n}\mathbb{G}_\mu\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](y)\,\mathrm{d} \omega^{x}_{\Omega_n}(y). \end{align*}$$

By proposition 2.7 (ii), we find

\[ \lim_{n \to \infty}\int_{\partial\Omega_n}\mathbb{G}_\mu\left[u^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right](y)\, \mathrm{d} \omega^{x}_{\Omega_n}(y) = 0. \]

Hence, letting $n\to \infty$ in (5.20), we obtain

\[ v_M+\mathbb{G}_\mu\left[v_M^\frac{{\alpha_{+}}+1}{{\alpha_{+}}-1}\right]=M\mathbb{K}_\mu[\nu]\quad\text{in}\;\Omega, \quad\forall M>0. \]

We see that the above display contradicts with (5.18) and (5.19). The proof is complete.

Next we turn to

Proof of theorem 2.2 Without loss of generality, we may assume that $z=0.$ Let $\zeta :\mathbb {R}\to [0,\infty )$ be a smooth function such that $0\leq \zeta \leq 1,$ $\zeta (t)=0$ for $|t|\leq 1$ and $\zeta (t)=1$ for $|t|>2$. For $\varepsilon >0$, we set $\zeta _\varepsilon (x)=\zeta (\frac {|x|}{\varepsilon })$. Since $u\in C^2(\Omega ),$ there holds

\[ L_\mu (\zeta_\varepsilon u)=u\Delta\zeta_{\varepsilon}+\zeta_\varepsilon u^p + 2\nabla\zeta_\varepsilon \cdot \nabla u \quad\text{in }\;\Omega . \]

By (5.5) and (5.6) with $F=\{0\} \subset \Sigma$, (1.4) and the estimate $\int _{\Sigma _{\beta }}d_{\Sigma }(x)^{-\alpha }\,\mathrm {d} x \lesssim \beta ^{N-\alpha }$ for $\alpha < N-k$, we have

(5.21)$$\begin{align} & \int_\Omega \zeta_\varepsilon u^p \phi_{\mu,\Sigma} \,\mathrm{d} x \!\lesssim\! \varepsilon^{-\frac{2p}{p-1}+({\alpha_{-}}- 1)p}\int_{\Omega\cap\{|x|>\varepsilon\}}\,d_\Sigma(x)^{(p+1)(1-{\alpha_{-}})}\, \mathrm{d} x \lesssim \varepsilon^{-\frac{2p}{p-1}+({\alpha_{-}} -1)p}, \end{align}$$

(5.22)$$\begin{align} & \int_\Omega u|\Delta\zeta_{\varepsilon}| \phi_{\mu,\Sigma} \, \mathrm{d} x \leq \varepsilon^{-\frac{2}{p-1}+{\alpha_{-}}-3}\int_{\Omega\cap\{\varepsilon<|x|<2\varepsilon\}}\,d_\Sigma(x)^{2-2{\alpha_{-}}} \, \mathrm{d} x\nonumber\\ & \quad \lesssim \varepsilon^{N-\frac{2}{p-1}-{\alpha_{-}}-1} \lesssim 1, \end{align}$$

(5.23)$$\begin{align} & \int_\Omega |\nabla\zeta_\varepsilon||\nabla u|\phi_{\mu,\Sigma} \,\mathrm{d} x \lesssim \varepsilon^{-\frac{2}{p-1}+{\alpha_{-}}-2}\int_{\Omega\cap\{\varepsilon<|x|<2\varepsilon\}}\,d_\Sigma(x)^{{-}2{\alpha_{-}}+1} \,\mathrm{d} x\nonumber\\ & \quad \lesssim \varepsilon^{N-\frac{2}{p-1}-{\alpha_{-}}-1} \lesssim 1. \end{align}$$

The estimates in (5.21) hold because of the assumption $p>1$ if ${\alpha _{+}} = 1$ or $p<\frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$ if ${\alpha _{+}}>1$. For the last estimate in (5.22) and (5.23), we have used the assumption $p \geq \frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$.

Estimates (5.21)–(5.23) imply that $L_\mu (\zeta _{\varepsilon }u) \in L^1(\Omega ;\phi _{\mu,\Sigma })$. By [Reference Barbatis, Gkikas and Tertikas3, lemma 8.5], we have

\[ -\int_{\Omega} \zeta_\varepsilon u L_\mu\eta \,\mathrm{d} x={-}\int_{\Omega} \left(u\Delta\zeta_{\varepsilon}+\zeta_\varepsilon u^p + 2\nabla\zeta_\varepsilon \cdot \nabla u\right)\eta \,\mathrm{d} x,\quad\forall \eta \in \mathbf{X}_\mu(\Omega). \]

Taking $\eta =\phi _{\mu,\Sigma },$ we obtain

\[ \lambda_{\mu,\Sigma}\int_{\Omega} \zeta_\varepsilon u \phi_{\mu,\Sigma} \, \mathrm{d} x+\int_{\Omega} \zeta_\varepsilon u^p\phi_{\mu,\Sigma} \, \mathrm{d} x={-}\int_{\Omega} \left(u\Delta\zeta_\varepsilon+2\nabla\zeta_\varepsilon \cdot \nabla u\right)\phi_{\mu,\Sigma}\,\mathrm{d} x. \]

By (5.22)–(5.23), we have

\[ \lambda_{\mu,\Sigma}\int_{\Omega} \zeta_\varepsilon u\phi_{\mu,\Sigma} \,\mathrm{d} x+\int_{\Omega} \zeta_\varepsilon u^p\phi_{\mu,\Sigma} \,\mathrm{d} x\leq C\varepsilon^{N-\frac{2}{p-1}-{\alpha_{-}}-1}. \]

By letting $\varepsilon \to 0$ and Fatou's lemma, we deduce that

\[ \lambda_{\mu,\Sigma}\int_{\Omega} u\phi_{\mu,\Sigma} \,\mathrm{d} x+\int_{\Omega} u^p\phi_{\mu,\Sigma} \,\mathrm{d} x\lesssim\left\{ \begin{array}{@{}ll} 0 & \text{if}\ p>\dfrac{N+{\alpha_{-}}+1}{N-{\alpha_{-}}-1},\\ 1 & \text{if}\ p=\dfrac{N+{\alpha_{-}}+1}{N-{\alpha_{-}}-1}. \end{array}\right. \]

This implies that $u\equiv 0$ if $p>\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}$ or $u\in L^p(\Omega ;\phi _{\mu,\Sigma })$ if $p=\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}.$

The rest of the proof can proceed as in the proof of theorem 2.1 and we omit it.

5.3. Existence of solutions in the supercritical range

In this subsection, we discuss the existence of solutions for the following problem

(5.24)\begin{equation} \left\{\begin{aligned} -L_\mu u + \left | u\right |^{p-1}u = 0 & \text{in } \Omega ,\\ \mathrm {tr}_{\mu,\Sigma}(u) = \nu, \end{aligned} \right. \end{equation}

where $p>1$ and $\nu \in \mathfrak M(\partial \Omega )$. We will focus on the supercritical case $p\geq \min \{\frac {N+1}{N-1},\frac {N-{\alpha _{-}}+1}{N-{\alpha _{-}}-1}\}$. In particular, we will give various sufficient conditions for the existence of solutions to (5.24).

To this purpose, we use Besov space (see e.g. [Reference Adams and Hedberg1, Reference Stein35]). For $\sigma >0$, $1\leq \kappa <\infty$, we denote by $W^{\sigma,\kappa }(\mathbb {R}^d)$ the Sobolev space over $\mathbb {R}^d$. If $\sigma$ is not an integer the Besov space $B^{\sigma,\kappa }(\mathbb {R}^d)$ coincides with $W^{\sigma,\kappa }(\mathbb {R}^d)$. When $\sigma$ is an integer we denote $\Delta _{x,y}f:=f(x+y)+f(x-y)-2f(x)$ and

\[ B^{1,\kappa}(\mathbb{R}^d):=\left\{f\in L^\kappa(\mathbb{R}^d): {\displaystyle \frac{\Delta_{x,y}f}{|y|^{1+\frac{d}{\kappa}}} }\in L^\kappa(\mathbb{R}^d\times \mathbb{R}^d)\right\}, \]

with norm

\[ \|f\|_{B^{1,\kappa}}:=\left(\|f\|^\kappa_{L^\kappa}+\int_{\mathbb{R}^d} \int_{\mathbb{R}^d}\frac{|\Delta_{x,y}f|^\kappa}{|y|^{\kappa+d}}\,\mathrm{d} x \,\mathrm{d} y\right)^{\frac{1}{\kappa}}. \]

Then

\[ B^{m,\kappa}(\mathbb{R}^d):=\left\{f\!\in\! W^{m-1,\kappa}(\mathbb{R}^d): D_x^\alpha f\!\in\! B^{1,\kappa}(\mathbb{R}^d)\;\forall\alpha\!\in\! \mathbb{N}^d \text{ such that } |\alpha|=m\!-\!1\right\}, \]

with norm

\[ \|f\|_{B^{m,\kappa}}:=\left(\|f\|^\kappa_{W^{m-1,\kappa}}+\sum_{|\alpha|=m-1}\int_{\mathbb{R}^d} \int_{\mathbb{R}^d}\frac{|D_x^\alpha\Delta_{x,y}f|^\kappa}{|y|^{\kappa+d}}\,\mathrm{d} x \,\mathrm{d} y\right)^{\frac{1}{\kappa}}. \]

These spaces are fundamental because they are stable under the real interpolation method developed by Lions and Petree.

It is well known that if $1<\kappa <\infty$ and $\alpha >0$, $L_{\alpha,\kappa }(\mathbb {R}^d)=W^{\alpha,\kappa }(\mathbb {R}^d)$ if $\alpha \in \mathbb {N}.$ If $\alpha \notin \mathbb {N}$ then the positive cone of their dual coincide, i.e. $(L_{-\alpha,\kappa '}(\mathbb {R}^d))^+=(B^{-\alpha,\kappa '}(\mathbb {R}^d))^+$, always with equivalent norms.

Lemma 5.3 Let $k\geq 1$, $p$ be as in (2.14), and $\nu \in \mathfrak {M}^+(\mathbb {R}^k)$ with compact support in $B^k\left (0,\frac {R}{2}\right )$ for some $R>0$. Set

(5.25)\begin{equation} \vartheta: = \frac{{\alpha_{+}}+1-p({\alpha_{+}}-1)}{p}. \end{equation}

For $x \in \mathbb {R}^{k+1}$, we write $x=(x_1,x') \in \mathbb {R}\times \mathbb {R}^{k}$. Then there exists a constant $C=C(R,N,k,\mu,p)>1$ such that

$$\begin{align*} & C^{{-}1}\left \|\nu\right \|^p_{B^{-\vartheta,p}(\mathbb{R}^k)}\\ & \leq \int_{B^k(0,R)}\int_{0}^R x_{1}^{N-k-1-(p+1)({\alpha_{-}}-1)}\\ & \quad\times\left(\int_{B^k(0,R)}\left(x_1+|x'-y'|\right)^{-(N-2{\alpha_{-}})}\,\mathrm{d} \nu(y')\right)^p\,\mathrm{d} x_1\,\mathrm{d} x'\\ & \leq C \left \|\nu\right \|^p_{B^{-\vartheta,p}(\mathbb{R}^k)}. \end{align*}$$

Theorem 5.4 Let $k\geq 1$, $p$ be as in (2.14), and $\nu \in \mathfrak {M}^+(\partial \Omega )$ with compact support in $\Sigma$. Then there exists a constant $C=C(\Omega,\Sigma,\mu )>1$ such that

\[ C^{{-}1}\left \|\nu\right \|_{B^{-\vartheta,p}(\Sigma)}\leq \left \|\mathbb{K}_{\mu}[\nu]\right \|_{L^p(\Omega;\phi_{\mu,\Sigma})}\leq C \left \|\nu\right \|_{B^{-\vartheta,p}(\Sigma)}, \]

where $\vartheta$ is given in (5.25).

Proof. By (A.7), there exists $\xi ^j \in \Sigma$, $j=1,2,\ldots,m_2$ (where $m_2 \in {\mathbb {N}}$ depends on $N,\Sigma$), and $\beta _2 \in (0,\frac {\beta _0}{4})$ such that $\Omega \cap \Sigma _{\beta _2} \subset \cup _{j=1}^{m_2} \mathcal {V}_{\Sigma }(\xi ^j,\frac {\beta _0}{4})\cap \Omega$.

Assume $\nu \in \mathfrak {M}^+(\partial \Omega )$ with compact support in $\Sigma \cap \mathcal {V}_{\Sigma }(\xi ^j,\frac {\beta _0}{4})$ for some $j\in \{1,\ldots,m_2\}$.

On one hand, from (1.4), (3.3) and since $p< \frac {{\alpha _{+}}+1}{{\alpha _{+}}-1}$ and ${\alpha _{+}}\geq {\alpha _{-}}$, we have

$$\begin{align*} & \int_{\Omega\cap\mathcal{V}_{\Sigma}\left(\xi^j,\frac{\beta_0}{2}\right)}\mathbb{K}_\mu[\nu]^p\phi_{\mu,\Sigma} \,\mathrm{d}x \\ & \quad\gtrsim \nu(\partial\Omega)^p\int_{\Omega\cap\mathcal{V}_{\Sigma}(\xi^j,\frac{\beta_0}{2})}d_{\partial \Omega}(x)^{p+1}d_\Sigma(x)^{-(p+1){\alpha_{-}}} \,\mathrm{d}x\gtrsim\nu(\partial\Omega)^p. \end{align*}$$

On the other hand,

$$\begin{align*} & \int_{\Omega\setminus \mathcal{V}_{\Sigma}(\xi^j,\frac{\beta_0}{2})}\mathbb{K}_\mu[\nu]^p\phi_{\mu,\Sigma}\,\mathrm{d}x \\ & \quad\lesssim \nu(\partial\Omega)^p\int_{\Omega\setminus \mathcal{V}_{\Sigma}(\xi^j,\frac{\beta_0}{2})}d_{\partial \Omega}(x)^{p+1}d_\Sigma(x)^{-(p+1){\alpha_{-}}}\,\mathrm{d}x\lesssim \nu(\partial\Omega)^p. \end{align*}$$

Combining the above estimates, we obtain

(5.26)$$\begin{align} \int_{ \Omega}\phi_{\mu,\Sigma} \mathbb{K}_{\mu}[\nu]^p \,\mathrm{d} x & = \int_{\Omega \setminus \mathcal{V}_{\Sigma}(\xi^j,\frac{\beta_0}{2})}\phi_{\mu,\Sigma} \mathbb{K}_{\mu}[\nu]^p \,\mathrm{d} x+\int_{\Omega\cap\mathcal{V}_{\Sigma}\left(\xi^j,\frac{\beta_0}{2}\right) }\phi_{\mu,\Sigma} \mathbb{K}_{\mu}[\nu]^p \,\mathrm{d} x\nonumber\\ & \approx\int_{\Omega\cap\mathcal{V}_{\Sigma}\left(\xi^j,\frac{\beta_0}{2}\right) }\phi_{\mu,\Sigma} \mathbb{K}_{\mu}[\nu]^p \,\mathrm{d} x. \end{align}$$

For any $x \in {\mathbb {R}}^N$, we write $x=(x',x''',x_N)$ where $x'=(x_1,\ldots,x_k),$ $x'''=(x_{k+1},\ldots,x_{N-1})$ and define the $C^2$ function

\[ \Phi(x)\!:=\!(x',x_{k+1}\!-\!\Gamma_{k+1,\Sigma}^{\xi^j}(x'),\ldots,x_{N-1}-\Gamma_{N-1,\Sigma}^{\xi^j}(x'),x_N-\Gamma_{N,\partial\Omega}^{\xi^j}(x_1,\ldots,x_{N-1})). \]

Taking into account the local representation of $\Sigma$ and $\partial \Omega$ in § A.1, we may deduce that $\Phi :\mathcal {V}_\Sigma (\xi ^j,\beta _0)\to B^k(0,\beta _0)\times B^{N-1-k}(0,\beta _0)\times (-\beta _0,\beta _0)$ is $C^2$ diffeomorphism and $\Phi (x)=(x',0_{{\mathbb {R}}^{N-k}})$ for $x=(x',x''',x_N) \in \Sigma$. In view of the proof of [Reference Adams and Hedberg1, lemma 5.2.2], there exists a measure $\overline {\nu }\in \mathfrak M^+(\mathbb {R}^k)$ with compact support in $B^k\left (0,\frac {\beta _0}{4}\right )$ such that for any Borel $E\subset B^k\left (0,\frac {\beta _0}{4}\right ),$ there holds $\overline {\nu }(E)=\nu (\Phi ^{-1}(E\times \{0_{{\mathbb {R}}^{N-k}}\}))$.

Set $\psi =(\psi ',\psi ''',\psi _N)=\Phi (x)$ then

$$\begin{align*} & \psi'=x', \, \psi'''=(x_{k+1}-\Gamma_{k+1,\Sigma}^{\xi^j}(x'),\ldots,x_{N-1}-\Gamma_{N-1,\Sigma}^{\xi^j}(x'))\ \text{and}\ \psi_N\\ & =x_N-\Gamma_{N,\partial\Omega}^{\xi^j}(x_1,\ldots,x_{N-1}). \end{align*}$$

By (1.4), (A.6) and (3.3), we have

$$\begin{align*} & \phi_{\mu,\Sigma}(x)\approx \psi_N(\psi_N+|\psi'''|)^{-{\alpha_{-}}},\\ & K_{\mu}(x,y)\approx \psi_N(\psi_N+|\psi'''|)^{-{\alpha_{-}}}(\psi_N+|\psi'''|+|\psi'-y'|)^{-(N-2{\alpha_{-}})},\\ & \quad\forall x\in \mathcal{V}(\xi^j,\beta_0)\cap\Omega,\;\forall y=(y',y''',y_N) \in \mathcal{V}(\xi^j,\beta_0)\cap \Sigma. \end{align*}$$

Therefore,

(5.27)$$\begin{align} & \int_{ \Omega\cap \mathcal{V}(\xi^j,\beta_0/2) }\phi_{\mu,\Sigma} \mathbb{K}_{\mu}^p[\nu]\,\mathrm{d} x\nonumber\\ & \approx \int_{B^k\left(0,\frac{\beta_0}{2}\right)}\int_0^{\frac{\beta_0}{2}}\int_{B^{N-k-1}\left(0,\frac{\beta_0}{2}\right)}\psi_N^{p+1}(\psi_N+|\psi'''|)^{-(p+1){\alpha_{-}}}\nonumber\\ & \quad\left(\int_{B^k(0,\frac{\beta_0}{4})}(\psi_N+|\psi'''|+|\psi'-y'|)^{-(N-2{\alpha_{-}})}\,\mathrm{d} \overline{\nu}(y')\right)^p \,\mathrm{d}\psi'''\,\mathrm{d} \psi_N \,\mathrm{d}\psi'\\ & \approx\int_{B^k\left(0,\frac{\beta_0}{2}\right)}\int_{0}^{\frac{\beta_0}{2}}r^{N-k-1-(p+1)({\alpha_{-}}-1)}\nonumber\\ & \quad \left(\int_{B^k\left(0,\frac{\beta_0}{2}\right)}(r+|\psi'-y'|)^{-(N-2{\alpha_{-}}-2)}\,\mathrm{d} \overline{\nu}(y')\right)^p\,\mathrm{d} r \,\mathrm{d}\psi'. \nonumber \end{align}$$

Since $\nu \mapsto \nu \circ \Phi ^{-1}$ is a $C^2$ diffeomorphism between $\mathfrak M^+(\Sigma \cap \mathcal {V}_\Sigma (\xi ^j,\beta _0))\cap B^{-\vartheta,p}(\Sigma \cap \mathcal {V}_\Sigma (\xi ^j,\beta _0))$ and $\mathfrak M^+(B^k(0,\beta _0))\cap B^{-\vartheta,p}(B^k(0,\beta _0))$, using (5.26), (5.27) and lemma 5.3, we derive that

\[ C^{{-}1}\left \|\nu\right \|_{B^{-\vartheta,p}(\Sigma)}\leq \left \|\mathbb{K}_{\mu}[\nu]\right \|_{L^p(\Omega;\phi_{\mu,\Sigma})}\leq C \left \|\nu\right \|_{B^{-\vartheta,p}(\Sigma)}. \]

If $\nu \in \mathfrak {M}^+(\partial \Omega )$ has compact support in $\Sigma,$ we may write $\nu =\sum _{j=1}^{m_2}\nu _j$, where $\nu _j\in \mathfrak {M}^+(\partial \Omega )$ with compact support in $\mathcal {V}(\xi ^j,\frac {\beta _0}{4})\cap \Sigma$. By (5.26), we can show that

\[ \left \|\mathbb{K}_{\mu}[\nu]\right \|_{L^p(\Omega;\phi_{\mu,\Sigma})}\!\approx\! \sum_{j=1}^{m_2}\left \|\mathbb{K}_{\mu}[\nu_j]\right \|_{L^p(\Omega;\phi_{\mu,\Sigma})}\!\approx\! C\sum_{j=1}^{m_2} \left \|\nu_j\right \|_{B^{-\vartheta,p}(\Sigma)}\approx C m_2 \left \|\nu\right \|_{B^{-\vartheta,p}(\Sigma)}. \]

The proof is complete.

Using theorem 5.4 and proposition 3.5, we are ready to prove theorem 2.5.

Proof of theorem 2.5 If $\nu$ is a positive measure which vanishes on Borel sets $E\subset \Sigma$ with $\mathrm {Cap}^{\mathbb {R}^{k}}_{\vartheta,p'}$-capacity zero then there exists an increasing sequence $\{\nu _n\}$ of positive measures in $B^{-\vartheta,p}(\Sigma )$ which converges weakly to $\nu$ (see [Reference Dal Maso10, Reference Feyel and de la Pradelle16]). By theorem 5.4, we have $\mathbb {K}_{\mu }[\nu _n]\in L^p(\Omega ;\phi _{\mu,\Sigma })$, hence we may apply proposition 3.5 with $g(t)=|t|^{p-1}t$ to deduce that there exists a unique nonnegative weak solution $u_n$ of (5.24) with $\mathrm {tr}_{\mu,\Sigma}(u_n)=\nu _n.$

Since $\{ \nu _n \}$ is an increasing sequence of positive measures, by theorem 3.4, $\{u_n\}$ is increasing and its limit is denoted by $u$. Moreover,

(5.28)\begin{equation} - \int_{\Omega}u_n L_{\mu }\zeta \,\mathrm{d} x + \int_{\Omega} u_n^p \zeta\,\mathrm{d} x ={-} \int_{\Omega} \mathbb{K}_{\mu}[\nu_n]L_{\mu }\zeta \,\mathrm{d} x\quad\forall \zeta \in\mathbf{X}_\mu(\Omega). \end{equation}

By taking $\zeta =\phi _{\mu,\Sigma }$ in (5.28), we obtain

\[ \int_{\Omega} \left(\lambda_{\mu,\Sigma} u_{n}+u_n^p\right)\phi_{\mu,\Sigma} \,\mathrm{d} x=\lambda_{\mu,\Sigma}\int_{\Omega} \mathbb{K}_{\mu}[\nu_n]\phi_{\mu,\Sigma} \,\mathrm{d} x, \]

which implies that $\{u_n\}$ and $\{u_n^p\}$ are uniformly bounded in $L^1(\Omega ;\phi _{\mu,\Sigma })$. Therefore, $u_n \to u$ in $L^1(\Omega ;\phi _{\mu,\Sigma })$ and in $L^p(\Omega ;\phi _{\mu,\Sigma })$. By letting $n \to \infty$ in (5.28), we deduce

\[ \int_{\Omega}-uL_\mu\zeta \,\mathrm{d} x +\int_{\Omega} u^p\zeta \,\mathrm{d} x={-}\int_{\Omega} \mathbb{K}_{\mu}[\nu]L_\mu\zeta \,\mathrm{d} x\quad\forall \zeta\in {\bf X}_\mu(\Omega). \]

This means $u$ is the unique weak solution of (5.24) with $\mathrm {tr}_{\mu,\Sigma}(u)=\nu$.

6.1. Preparative results

For $\alpha \leq N$, set

\[ {\mathcal{N}}_{\alpha}(x,y):=\frac{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha}{|x-y|^{N-2}\max\{|x-y|,d_{\partial \Omega}(x),d_{\partial \Omega}(y)\}^2},\quad (x,y)\in\overline{\Omega}\times\overline{\Omega}, x \neq y, \]

(6.2)\begin{equation} \mathbb{N}_{\alpha}[\omega](x):=\int_{\overline{\Omega}} {\mathcal{N}}_{\alpha}(x,y)\,\mathrm{d}\omega(y), \quad \omega \in \mathfrak M^+(\overline \Omega). \end{equation}

Let $\alpha < N$, $b>0$, $\theta >-N+k$ and $s>1$. We define the capacity $\text {Cap}_{\mathbb {N}_{\alpha },s}^{b,\theta }$ by

$$\begin{align*} & \text{Cap}_{\mathbb{N}_{\alpha},s}^{b,\theta}(E) :=\inf\left\{\int_{\overline{\Omega}}d_{\partial \Omega}^b d^\theta_\Sigma\phi^s\,\mathrm{d}x:\ \phi \geq 0,\ \mathbb{N}_{\alpha}[ d_{\partial \Omega}^b d_{\Sigma}^\theta\phi ]\geq{\mathbb 1}_E\right\} \nonumber\\ & \quad \text{for any Borel set } E\subset\overline{\Omega}. \end{align*}$$

Here ${\mathbb 1}_E$ denotes the indicator function of $E$. By [Reference Adams and Hedberg1, theorem 2.5.1], we have

\[ (\text{Cap}_{\mathbb{N}_{\alpha},s}^{b,\theta}(E))^\frac{1}{s}=\sup\{\omega(E):\omega \in\mathfrak M^+(E), \|\mathbb{N}_{\alpha}[\omega]\|_{L^{s'}(\Omega;d_{\partial \Omega}^b d^\theta_\Sigma)} \leq 1 \}. \]

If $\mu <\frac {N^2}{4}$ and $\nu \in \mathfrak {M}^+(\partial \Omega )$, then, by (3.1) and (3.3), we can show that

(6.3)\begin{equation} G_\mu(x,y)\approx d_{\partial \Omega}(x)\,d_{\partial \Omega}(y)(d_{\Sigma}(x)\,d_{\Sigma}(y))^{-{\alpha_{-}}} {\mathcal{N}}_{2{\alpha_{-}}}(x,y) \quad \forall x,y \in \Omega, x \neq y \end{equation}

and

(6.4)\begin{equation} K_\mu(x,y)\approx d_{\partial \Omega}(x)\,d_{\Sigma}(x)^{-{\alpha_{-}}} {\mathcal{N}}_{2{\alpha_{-}}}(x,y) \quad \forall (x,y) \in \Omega\times\partial\Omega. \end{equation}

Therefore, if the integral equation

(6.5)\begin{equation} v=\mathbb{N}_{2{\alpha_{-}}}[(d_{\partial \Omega}d_{\Sigma}^{-{\alpha_{-}}})^{p+1}v^p]+\ell \mathbb{N}_{2{\alpha_{-}}}[\nu] \end{equation}

has a solution $v$ for some $\ell >0$ then $\tilde v=d_{\partial \Omega }d_{\Sigma }^{-{\alpha _{-}}}v$ satisfies

(6.6)\begin{equation} \tilde v\approx\mathbb{G}_\mu[\tilde v^p]+\ell \mathbb{K}_\mu[\nu]. \end{equation}

This, together with [Reference Bidaut-Véron, Hoang, Nguyen and Véron5, proposition 2.7], implies that equation (6.1) has a positive solution $u$ for some $\sigma >0$, whence problem ($\text{BVP}_-^{\sigma}$) has a weak positive solution $u$ for some $\sigma >0$.

In order to show that (6.5) possesses a solution, we will apply the results in [Reference Kalton and Verbitsky23] which we recall here for the sake of completeness.

Let $\mathbf {Z}$ be a metric space and $\omega \in \mathfrak M^+(\mathbf {Z})$. Let $J : \mathbf {Z} \times \mathbf {Z} \to (0,\infty ]$ be a Borel positive kernel such that $J$ is symmetric and $1/J$ satisfies a quasi-metric inequality, i.e. there is a constant $C>1$ such that for all $x, y, z \in \mathbf {Z}$,

(6.7)\begin{equation} \frac{1}{J(x,y)}\leq C\left(\frac{1}{J(x,z)}+\frac{1}{J(z,y)}\right). \end{equation}

Under these conditions, one can define the quasi-metric $\mathbf {d}$ by

\[ \mathbf{d}(x,y):=\frac{1}{J(x,y)} \]

and denote by $\mathfrak B(x,r):=\{y\in \mathbf {Z}:\; \mathbf {d}(x,y)< r\}$ the open $\mathbf {d}$-ball of radius $r > 0$ and centre $x$. Note that this set can be empty.

For $\omega \in \mathfrak M^+(\mathbf {Z})$ and a positive function $\phi$, we define the potentials $\mathbb {J}[\omega ]$ and $\mathbb {J}[\phi,\omega ]$ by

\[ \mathbb{J}[\omega](x):=\int_{\mathbf{Z}}J(x,y)\,\mathrm{d}\omega(y)\quad\text{and}\quad \mathbb{J}[\phi,\omega](x):=\int_{\mathbf{Z}}J(x,y)\phi(y)\,\mathrm{d}\omega(y). \]

For $t>1$ the capacity $\text {Cap}_{\mathbb {J},t}^\omega$ in $\mathbf {Z}$ is defined for any Borel set $E\subset \mathbf {Z}$ by

\[ \text{Cap}_{\mathbb{J},t}^\omega(E):=\inf\left\{\int_{\mathbf{Z}}\phi(x)^{t}\,\mathrm{d}\omega(x):\ \phi\geq0,\ \mathbb{J}[\phi,\omega] \geq{\mathbb 1}_E\right\}. \]

Proposition 6.2 ([Reference Kalton and Verbitsky23]) Let $p>1$ and $\lambda,\omega \in \mathfrak M^+(\mathbf {Z})$ such that

(6.8)$$\begin{align} \int_0^{2r}\frac{\omega\left(\mathfrak B(x,s)\right)}{s^2}\,\mathrm{d} s & \leq C\int_0^{r}\frac{\omega\left(\mathfrak B(x,s)\right)}{s^2}\,\mathrm{d} s , \end{align}$$

(6.9)$$\begin{align} \sup_{y\in \mathfrak B(x,r)}\int_0^{r}\frac{\omega\left(\mathfrak B(y,r)\right)}{s^2} \mathrm{d} s& \leq C\int_0^{r}\frac{\omega\left(\mathfrak B(x,s)\right)}{s^2} \mathrm{d} s, \end{align}$$

for any $r > 0,$ $x \in \mathbf {Z}$, where $C > 0$ is a constant. Then the following statements are equivalent.

1. 1. The equation $v=\mathbb {J}[|v|^p,\omega ]+\ell \mathbb {J}[\lambda ]$ has a positive solution for $\ell >0$ small.

2. 2. For any Borel set $E \subset \mathbf {Z}$, there holds $\int _E \mathbb {J}[{\mathbb 1}_E\lambda ]^p \mathrm {d} \omega \leq C\, \lambda (E)$.

3. 3. The following inequality holds $\mathbb {J}[\mathbb {J}[\lambda ]^p,\omega ]\leq C\mathbb {J}[\lambda ]<\infty \ \omega -a.e.$

4. 4. For any Borel set $E \subset \mathbf {Z}$ there holds $\lambda (E)\leq C\, {\rm Cap}_{\mathbb {J},p'}^\omega (E)$.

We will point out below that $\mathbb {N}_\alpha$ defined in (6.2) with $\mathrm {d} \omega =d_{\partial \Omega }(x)^b\,d_\Sigma (x)^\theta {\mathbb 1}_{\Omega }(x) \mathrm {d}x$ satisfies all assumptions of $\mathbb {J}$ in proposition 6.2, for some appropriate $b,\theta \in \mathbb {R}$. Let us first prove the quasi-metric inequality.

Lemma 6.3 Let $\alpha \leq N$. There exists a positive constant $C=C(\Omega,\Sigma,\alpha )$ such that

(6.10)\begin{equation} \frac{1}{{\mathcal{N}}_\alpha(x,y)}\leq C\left(\frac{1}{{\mathcal{N}}_\alpha(x,z)}+\frac{1}{{\mathcal{N}}_\alpha(z,y)}\right),\quad \forall x,y,z\in \overline{\Omega}. \end{equation}

Proof. Case 1: $0<\alpha \leq N$. We first assume that $|x-y|<2|x-z|$. Then by the triangle inequality, we have $d_\Sigma (z)\leq |x-z|+d_\Sigma (x)\leq 2\max \{|x-z|,d_\Sigma (x)\}$ hence

\[ \max\{|x-z|,d_\Sigma(x),d_\Sigma(z)\}\leq 2\max\{ |x-z|,d_\Sigma(x)\}. \]

If $|x-z|\geq d_\Sigma (x)$ then $|x-z|\geq d_{\partial \Omega }(x)$ which implies that $|x-z|\geq \frac {d_{\partial \Omega }(x)+|x-y|}{4}$. Now,

(6.11)$$\begin{align} & \frac{|x-z|^{N-2}\max\{|x-z|,d_{\partial \Omega}(x),d_{\partial \Omega}(z)\}^2}{\max\{|x-z|,d_\Sigma(x),d_\Sigma(z)\}^\alpha}\nonumber\\ & \quad \geq 2^{-\alpha}|x-z|^{N-\alpha}\geq 2^{{-}2N+\alpha}(|x-y|+d_{\partial \Omega}(x))^{N-\alpha}\nonumber\\ & \quad=2^{{-}2N+\alpha}\frac{(|x-y|+d_{\partial \Omega}(x))^{N}}{(|x-y|+d_{\partial \Omega}(x))^\alpha}\gtrsim \frac{|x-y|^{N-2}\max\{|x-y|,d_{\partial \Omega}(x),d_{\partial \Omega}(y)\}^2}{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha}, \end{align}$$

since $d_{\partial \Omega }(x)\leq d_\Sigma (x).$

If $|x-z|\leq d_\Sigma (x)$ then

(6.12)$$\begin{align} & \frac{|x-z|^{N-2}\max\{|x-z|,d_{\partial \Omega}(x),d_{\partial \Omega}(z)\}^2}{\max\{|x-z|,d_\Sigma(x),d_\Sigma(z)\}^\alpha}\nonumber\\ & \quad\geq 2^{-\alpha-2}\,d_\Sigma(x)^{-\alpha}|x-z|^{N-2}\max\{|x-y|,d_{\partial \Omega}(x)\}^2 \nonumber\\ & \quad\gtrsim\frac{|x-y|^{N-2}\max\{|x-y|,d_{\partial \Omega}(x),d_{\partial \Omega}(y)\}^2}{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha}, \end{align}$$

since $d_{\partial \Omega }(y)\leq |x-y|+d_{\partial \Omega }(x)\leq 2\max \{|x-y|,d_\Sigma (x)\}.$ Combining (6.11)–(6.12), we obtain (6.10).

Next we consider the case $2|x-z|\leq |x-y|$. Then $\frac {1}{2} |x-y|\leq |y-z|,$ thus by symmetry we obtain (6.10).

Case 2: $\alpha \leq 0$. Let $b\in [0,2],$ since $d_\Sigma (x)\leq |x-y|+d_\Sigma (y)$, it follows that

\[ \max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}\leq |x-y|+\min\{d_\Sigma(x),d_\Sigma(y)\}. \]

Using the above estimate, we obtain

(6.13)$$\begin{align} & |x-y|^{N-b}\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^{-\alpha}\nonumber\\ & \lesssim |x-y|^{N-b-\alpha}+\min\{d_\Sigma(x),d_\Sigma(y)\}^{-\alpha}|x-y|^{N-b}\nonumber\\ & \lesssim|x-z|^{N-b-\alpha}+|y-z|^{N-b-\alpha}+\min\{d_\Sigma(x),d_\Sigma(y)\}^{-\alpha}(|x-z|^{N-b}+|y-z|^{N-b})\nonumber\\ & \lesssim\frac{|x-z|^{N-b}}{\max\{|x-z|,d_\Sigma(x),d_\Sigma(z)\}^\alpha}+\frac{|z-y|^{N-b}}{\max\{|z-y|,d_\Sigma(z),d_\Sigma(y)\}^\alpha}. \end{align}$$

Since $d_{\partial \Omega }(x)\leq |x-y|+d_{\partial \Omega }(y),$ we can easily show that $\max \{|x-y|,d_{\partial \Omega }(x), d_{\partial \Omega }(y)\}\leq |x-y|+\min \{d_{\partial \Omega }(x),d_{\partial \Omega }(y)\}$. Hence,

$$\begin{align*} \frac{1}{{\mathcal{N}}_\alpha(x,y)}& =\frac{\max\{|x-y|,d_{\partial \Omega}(x),d_{\partial \Omega}(y)\}^2|x-y|^{N-2}}{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha} \\ & \leq \frac{2|x-y|^{N}}{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha} +\frac{2\min\{d_{\partial \Omega}(x),d_{\partial \Omega}(y)\}^2|x-y|^{N-2}}{\max\{|x-y|,d_\Sigma(x),d_\Sigma(y)\}^\alpha}. \end{align*}$$

The desired result follows by the above inequality and (6.13).