2.1 Evaluation of explanations and their application in NMT

The XAI literature distinguishes several dimensions of what explanations can provide. A central distinction is between plausibility and faithfulness/fidelity: plausibility refers to how well an explanation aligns with human intuition, whereas faithfulness describes how accurately an explanation reflects the model’s actual decision-making process (Arrieta et al. Reference Barredo Arrieta, Díaz-Rodríguez, Del Ser, Bennetot, Tabik, Barbado, Garcia, Gil-Lopez, Molina, Benjamins, Chatila and Herrera2020; Jacovi and Goldberg Reference Jacovi and Goldberg2020). Doshi-Velez and Kim (Doshi-Velez and Kim Reference Doshi-Velez and Kim2017, Reference Doshi-Velez and Kim2018) have proposed three approaches to evaluate XAI methods, one of which is a functionally grounded evaluation protocol. In this protocol, explanations are assessed by automatic task performance metrics rather than human judgments. This paradigm has been adopted in NLP for evaluating the faithfulness of saliencyFootnote ^b methods. For example, Arras et al. (Reference Arras, Horn, Montavon, Müller and Wojciech2016); Nguyen (Reference Nguyen2018); DeYoung et al. (Reference De Young, Jain, Rajani, Lehman, Xiong, Socher and Wallace2020); Atanasova et al. (Reference Atanasova, Simonsen, Lioma and Augenstein2020); Nauta et al. (Reference Nauta, Trienes, Pathak, Nguyen, Peters, Schmitt, Schlötterer, van Keulen and Seifert2023) proposed automatic, task-based metrics for classification models that quantify how well feature attributions capture model behavior.

Our focus is on the NMT task, in particular, transformer-based (Vaswani Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017) seq2seq models (Sutskever et al. Reference Sutskever, Vinyals and Le2014). Compared to standard classification, NMT introduces additional challenges for explanation because it decomposes prediction into a sequence of conditional next-token decisions, one per decoding timestep, and the importance of each source token depends not only on the current target token but also on the previous target prefix (Stahlberg Reference Stahlberg2020; Shakil, Farooq, and Kalita Reference Shakil, Farooq and Kalita2024). This temporal and source–target coupling complicates both the design and the evaluation of token-level attribution methods.

A large body of work has studied the attention mechanism as an interpretable component of NMT, using encoder–decoder attention weights to estimate word importance and to approximate word alignments (Ghader and Monz Reference Ghader and Monz2017; Raganato and Tiedemann Reference Raganato and Tiedemann2018; Kobayashi et al. Reference Kobayashi, Kuribayashi, Yokoi and Inui2020; Ferrando and Costa-jussà Reference Ferrando and Costa-Jussà2021). These studies show that attention patterns correlate with, but do not faithfully reproduce, traditional word alignments, and that attention weights also reflect how the model balances source information against the evolving target prefix. On the other hand, other work has questioned the plausibility and faithfulness of attention as an explanation, arguing that attention weights should be interpreted cautiously based on the task and model, and in some cases, augmented with more explicit attribution mechanisms (Jain and Wallace Reference Jain and Wallace2019; Meister et al. Reference Meister, Lazov, Augenstein and Cotterell2021; Madsen et al. Reference Madsen, Meade, Adlakha and Reddy2022).

Beyond attention, other explanation methods exist to compute per-token importance. The most common metrics to evaluate token importance are comprehensiveness (does removing the highlighted tokens reduce the model’s confidence or translation quality?) and sufficiency (are the highlighted tokens alone sufficient to preserve model performance?) (DeYoung et al. Reference De Young, Jain, Rajani, Lehman, Xiong, Socher and Wallace2020; Nauta et al. Reference Nauta, Trienes, Pathak, Nguyen, Peters, Schmitt, Schlötterer, van Keulen and Seifert2023). In NMT, such approaches have been explored to measure the drop in log-probability of the chosen next token after input perturbations or at the sequence level, where changes in BLEU scores after manipulating important source tokens, according to XAI methods, were used as a test bed for assessing whether attribution maps identify tokens that are necessary and/or sufficient for the model’s predictions (He et al. Reference He, Tu, Wang, Wang, Lyu and Shi2019; Moradi, Kambhatla, and Sarkar Reference Moradi, Kambhatla and Sarkar2021).

Word alignment (Brown et al. Reference Brown, Pietra, Pietra and Mercer1993) has also been widely used as a proxy for the plausibility of model attributions in NMT. X. Li et al. (Reference Li, Li, Liu, Meng and Shi2019) showed that attention-based alignments have clear limitations, motivating alternative alignment models and prediction-difference techniques to improve alignment quality. In parallel, Zenkel, Wuebker, and De Nero (Reference Zenkel, Wuebker and DeNero2019) proposed augmenting NMT models with dedicated alignment layers, treating alignment as an auxiliary prediction task rather than a by-product of standard attention. Their approach improves alignment accuracy compared to classical tools such as GIZA++ (Och and Ney Reference Och and Ney2003) and FastAlign (Dyer, Chahuneau, and Smith Reference Dyer, Chahuneau and Smith2013). Ding, Xu, and Koehn (Reference Ding, Xu and Koehn2019) introduced saliency-driven, gradient-based methods that yield more interpretable alignment signals without modifying the underlying NMT architecture. Building on these ideas, Ferrando and Costa-jussà (Reference Ferrando and Costa-Jussà2021) analyzed encoder–decoder attention in detail, highlighting systematic alignment errors and proposing techniques that explicitly quantify the relative contributions of source and target contexts. Ferrando et al. (Reference Ferrando, Gállego, Alastruey, Escolano and Costa-Jussà2022) further developed ALTI+, an attention-rollout-based framework that traces contributions from both source and target contexts across layers in multilingual Transformer models. ALTI + has been used as an internal explanation metric for downstream diagnostic tasks; for instance, Dale et al. (Reference Dale, Voita, Barrault and Costa-Jussà2023) employ ALTI+-based scores to detect and mitigate hallucinations in NMT outputs. Related work by Voita, Sennrich, and Titov (Reference Voita, Sennrich and Titov2021) used layer-wise propagation (LPR) to analyze the intrinsic contributions of source and target contexts under different training regimes and dataset conditions, while Kobayashi et al. (Reference Kobayashi, Kuribayashi, Yokoi and Inui2020) performed a norm-based analysis of attention and transformed representations to study internal alignment mechanisms within Transformers. Closer to the current work, Li et al. (Reference Li, Liu, Li, Li, Huang and Shi2020) evaluated XAI methods in NMT by training surrogate models on the important words identified by the XAI methods and measuring the prediction success of each token $i$ based on the top-k tokens identified as the most contributing tokens to the generation of that token. Other work has explored and compared XAI methods, such as gradient and perturbation methods, to detect word-level translation errors in NMT (Eksi et al. Reference Eksi, Gelbing, Stieber and Vu2021; Fomicheva et al. Reference Fomicheva, Specia and Aletras2022).

2.2 Simulatability of XAI methods

A complementary line of work evaluates explanations through simulatability. Simulatability is how well an explanation helps a user replicate a model’s behavior. Doshi-Velez and Kim (Reference Doshi-Velez and Kim2017); Hase et al. (Reference Hase and Bansal2020) propose human-grounded protocols in which participants are asked to predict a model’s output on a given input, first without and then with access to explanations. The underlying idea is that an explanation method is better if it improves human prediction accuracy of the model’s decisions. These studies implement such user-in-the-loop evaluations on tabular and text classification tasks, using common XAI techniques to generate explanations that are shown to human subjects.

Closer to our setting, Pruthi et al. (Reference Pruthi, Bansal, Dhingra, Soares, Collins, Lipton, Neubig and Cohen2022) worked on a related idea without relying on human annotators. Building on the notion of simulatability, they transfer important information learned from one model to another. Token-level importance scores are used to guide a new model on several classification tasks, and the resulting change in performance is taken as evidence for the usefulness of the original attributions. In other words, an explanation is considered higher quality if it can be leveraged to train another model that better simulates or reconstructs the original model’s behavior. Our work adopts a similar spirit of model-based simulatability, but in the more complex seq2seq NMT setting.

2.3 Injection of knowledge into the attention mechanism

There is also some work on injecting external or structured linguistic knowledge directly into the attention mechanism (Jiao et al. Reference Jiao, Yichun, Lifeng, Xin, Xiao, Linlin, Wang and Liu2020; Bai et al. Reference Bai, Wang, Sun, Wu, Yang, Tang, Cao, Zhang, Tong, Yang, Bai, Zhang, Sun and Shen2022; Zhao and Shan Reference Zhao and Shan2024). In NMT Bugliarello and Okazaki (Reference Bugliarello and Okazaki2020) incorporate syntactic information by encoding, for each token, the distance to the syntactic ‘parent’ in a dependency tree, and using this signal to bias attention patterns. In their approach, knowledge injection is applied on the encoder side for the source sentence, enabling attention heads to exploit syntactic structure. Slobodkin, Choshen, and Abend (Reference Slobodkin, Choshen and Abend2022) augment encoder attention with syntactic and semantic information in the form of alignment-like constraints over the input. Their architecture modifies the attention computation so that heads are explicitly informed by these external signals, rather than learning them implicitly. They report that enriching attention with semantic information benefits translation quality. Similar to the current work Nourbakhsh, Lamsiyah, and Schommer (Reference Nourbakhsh, Lamsiyah and Schommer2025) inject attributions in the form of hard alignments and compare them to linguistic information, such as POS and Dependency information. In this work, we deal with soft attributions and more diverse XAI methods. These works suggest that attention can serve as a locus for the injection of task-relevant prior knowledge.

Bringing together these strands of work, we propose an evaluation framework for XAI attribution methods to inject attribution matrix scores into the attention mechanism and compare the resulting effects on the translation task itself. Our design is inspired by Remove and Retrain (ROAR) retraining paradigms (Hooker et al. Reference Hooker, Erhan, Kindermans and Kim2019), the simulatability of XAI methods (Hase et al. Reference Hase and Bansal2020), and prior work on knowledge injection into attention, and it contributes to this line of research by providing a comprehensive, functionally grounded comparison of several attribution methods in the seq2seq NMT model.

2.4 NMT with sequence-to-sequence models

Seq2seq models (Sutskever et al. Reference Sutskever, Vinyals and Le2014; Bahdanau, Cho, and Bengio Reference Bahdanau, Cho and Bengio2015), originally introduced for machine translation (Cho et al. Reference Cho, van Merriënboer, Gulcehre, Bahdanau, Bougares, Schwenk and Bengio2014), are conditional language models that learn to generate the target sentence token by token, conditioned on the source sentence and the previously generated target tokens. The original Transformer model, a precursor to encoder-based classifiers and decoder-based large language models, was an encoder–decoder (Vaswani Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017):

An NMT encoder–decoder model operates on two sequences:

\begin{equation*} \mathbf{x} = (x_1, x_2, \ldots , x_{T_x}), \quad \mathbf{y} = (y_1, y_2, \ldots , y_{T_y}). \end{equation*}

Where $\mathbf{x}$ and $\mathbf{y}$ are the source and target sequences, an NMT model with a seq2seq architecture defines a conditional probability distribution over the target sequence given the source sequence and previous target tokens:

\begin{equation*} p(\mathbf{y} \mid \mathbf{x};\, \theta ) = \prod _{t=1}^{T_y} p (y_t \mid y_{\lt t}, \mathbf{x};\, \theta ), \end{equation*}

where $y_{\lt t} = (y_1, \ldots , y_{t-1})$ and $\theta$ are all model parameters.

The encoder maps the source sequence to a sequence of continuous contextual representations based on stacked self-attention and feed-forward layers.

\begin{equation*} \mathbf{H} = ( \mathbf{h}_1, \ldots , \mathbf{h}_{T_x} ), \end{equation*}

The decoder is another neural network that, at each time step $t$ , takes as input the previously generated target tokens (through masked self-attention over $y_{\lt t}$ ) and attends to the encoder representations $\mathbf{H}$ (via cross-attention), producing a decoder representation $\mathbf{s}_t$ from which the next-token distribution $p(y_t \mid y_{\lt t}, \mathbf{x};\, \theta )$ is computed.

Given a training corpus

\begin{equation*} \mathcal{D} = \big\{\big(\mathbf{x}^{(n)}, \mathbf{y}^{(n)}\big)\big\}_{n=1}^N \end{equation*}

of source–target pairs, model parameters $\theta$ are typically learned by maximizing the conditional log-likelihood:

\begin{equation*} \mathcal{L}(\theta ) = - \sum _{n=1}^N \sum _{t=1}^{T_y^{(n)}} \log p\Bigl (y_t^{(n)} \mid y_{\lt t}^{(n)}, \mathbf{x}^{(n)};\, \theta \Bigr ). \end{equation*}

During training, teacher forcing is commonly used, meaning that the decoder receives the ground-truth previous token $y_{t-1}^{(n)}$ as input when predicting $y_t^{(n)}$ .

Figure 1.

An example of attribution maps derived from different XAI methods. For the source sentence ‘Dann gibt es noch Anbieter, die kaum Fahrraderfahrung, jedoch gute Fernostkontakte haben und so an günstige E-Bikes kommen.’ and the target ‘Then there are suppliers with little or no experience in the bicycle industry but good contacts in the Far East, thus giving them access to low-cost e-bikes.’. In the heatmaps, the rows correspond to source tokens, and the columns to target tokens. The heatmaps are generated from the normalized columns using the MinMax normalizer.

Figure 1 Long description

The image displays a comparative heat map analysis of various Explainable AI (XAI) techniques applied to Marian-MT and mBART models. Each heat map represents the attribution of input features to the output of neural network models, with rows corresponding to source tokens and columns to target tokens. The heat maps are generated from normalized columns using the MinMax normalizer. The methods compared include IxG, Saliency, LGxA, IG, GSHAP, DeepLIFT, Attention, and ValueZeroing. The heat maps show distinct patterns of attribution, with some methods like Attention and ValueZeroing yielding larger gains in BLEU scores, indicating better alignment between source and target representations. The heat maps for Marian-MT and mBART models reveal different levels of intensity and distribution of attribution scores, highlighting the varying effectiveness of different XAI techniques in capturing relevant signals.

2.5 AI explainability methods

XAI attribution methods can be broadly categorized into three main types: gradient-based, internal model-based, and perturbation-based approaches. Below, we briefly describe the attribution methods used in this work to extract attribution maps. To extract these attribution maps, we used the Inseq Python library (Sarti et al. Reference Sarti, Feldhus, Sickert and van der Wal2023).

Saliency: The saliency method was originally introduced for image classification tasks using convolutional neural networks (CNNs) and is one of the earliest gradient-based explanation techniques (Simonyan et al. Reference Simonyan, Vedaldi and Zisserman2014). It treats a trained neural network as a locally linear function of its input around a given example. The primary objective is to identify which pixels in an input image are most influential for a particular class prediction. The ‘saliency map’ is defined as the gradient of the class score with respect to each input dimension, often visualized as the element-wise magnitude of this gradient. Intuitively, if a small change in a particular input dimension (e.g., a pixel intensity) leads to a large change in the class score, that dimension is considered important for the model’s decision for class $c$ .

Let $\mathbf{x} \in \mathbb{R}^d$ be an input and $S_c(\mathbf{x})$ the score for class $c$ . The saliency map for class $c$ at $\mathbf{x}_0$ is defined as:

(1) \begin{equation} M_c^{(i)}(\mathbf{x}_0) = \frac {\partial S_c(\mathbf{x}_0)}{\partial x_i}, \quad i=1,\ldots ,d. \end{equation}

In NLP, $\mathbf{x}$ corresponds to token embeddings; we aggregate per-dimension gradients to obtain a scalar attribution per token (see Section 3).

Input $\times$ Gradient (I $\times$ G): It is a simple extension of saliency that combines information about how sensitive a prediction is to a feature with how strongly that feature is present in the input (Denil, Demiraj, and De Freitas Reference Denil, Demiraj and de Freitas2014). As in the saliency method, it considers the gradient of the class score with respect to the input, but instead of using the gradient alone, each input dimension is weighted by its own value. Intuitively, a feature should only be considered important if (i) small changes in that feature have a large effect on the score, and (ii) the feature is actually active in the current example. Raw gradients ignore the importance of the feature itself. I $\times$ G takes this information into account by scaling the gradient by the input. In image models, this corresponds to weighting pixel-wise gradients by the pixel intensities. Let $\mathbf{x} \in \mathbb{R}^d$ be an input and $S_c(\mathbf{x})$ the score for class $c$ . The Input $\times$ Gradient attribution for class $c$ at $\mathbf{x}_0$ is defined as:

(2) \begin{equation} M_c^{(i)}(\mathbf{x}_0) = x_{0,i}\,\frac {\partial S_c(\mathbf{x}_0)}{\partial x_i}, \quad i=1,\ldots ,d. \end{equation}

Layer Gradient $\times$ Activation (LG $\times$ A): LG $\times$ A applies I $\times$ G to a chosen hidden layer. Let $\mathbf{h}(\mathbf{x}_0)\in \mathbb{R}^{m}$ be its activation vector; the attribution for unit $j$ is:

(3) \begin{equation} M_{c}^{(j)}(\mathbf{x}_0) = h_j(\mathbf{x}_0)\,\frac {\partial S_c(\mathbf{x}_0)}{\partial h_j}, \quad j=1,\ldots ,m. \end{equation}

Integrated Gradients (IG): IG is motivated by two axioms: (i) Sensitivity, which requires that features responsible for a change in the model output relative to a baseline receive non-zero attribution, and (ii) Implementation invariance, which requires that attributions depend only on the input–output function $S(\mathbf{x})$ , not on a particular network parameterization (Sundararajan et al. Reference Sundararajan, Taly and Yan2017). The authors define IG as:

(4) \begin{equation} \mathrm{IG}_i(\mathbf{x}_0;\,\mathbf{x}') = (x_{0,i}-x'_i)\int _{0}^{1} \frac {\partial S_c \left (\mathbf{x}' + \alpha (\mathbf{x}_0-\mathbf{x}')\right )}{\partial x_i}\,d\alpha . \end{equation}

Given an input $\mathbf{x}$ and a baseline $\mathbf{x}'$ representing the absence of information (e.g., the zero vector), IG attributes feature $i$ by integrating gradients along the straight-line path from $\mathbf{x}'$ to $\mathbf{x}$ , capturing how the prediction changes as the input moves from the baseline to the actual example.

Gradient SHAP (GSHAP): GSHAP mixes IG with SHAP and estimates SHAP-style (Lundberg et al. Reference Lundberg and Lee2017) attributions by averaging gradients over randomized reference points. For each of $n$ samples, it adds small noise to the input, draws a random baseline from a set of baselines, and picks a random interpolation point along the straight line to compute the gradientFootnote ^c .

Deep learning important features (DeepLIFT): DeepLIFT explains a prediction by comparing the network’s response on an input $\mathbf{x}_0$ to a reference input $\mathbf{x}'$ (Shrikumar, Greenside, and Kundaje Reference Shrikumar, Greenside and Kundaje2017). Instead of raw gradients, it propagates differences from reference layer by layer and assigns contribution scores to input features that sum to the output difference $S_c(\mathbf{x}_0)-S_c(\mathbf{x}')$ , which helps mitigate gradient saturation and captures both positive and negative influences more reliably than standard gradient-based saliency methods.

Attention: In the Transformer model (Vaswani Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017), each token representation $x_t\in \mathbb{R}^{d_{\text{model}}}$ is linearly projected into a query, key, and value using learned parameter matrices $W_Q,W_K,W_V\in \mathbb{R}^{d_{\text{model}}\times d_k}$ . Scaled dot-product attention computes similarities between queries and keys, normalizes them with a softmax to obtain attention weights, and then uses these weights to form weighted sums of the values. In multi-head attention, $H$ such projections $\{W_Q^{(h)},W_K^{(h)},W_V^{(h)}\}_{h=1}^{H}$ are used in parallel, and the concatenated head outputs are linearly projected back to the model dimension with a learned matrix $W_O\in \mathbb{R}^{(H d_k)\times d_{\text{model}}}$ .

Given queries $Q\in \mathbb{R}^{T_q\times d_k}$ , keys $K\in \mathbb{R}^{T_k\times d_k}$ , and values $V\in \mathbb{R}^{T_k\times d_v}$ , scaled dot-product attention is

(5) \begin{equation} \mathrm{Attention}(Q,K,V) = \mathrm{softmax} \left (\frac {QK^\top }{\sqrt {d_k}}\right )\!V. \end{equation}

In (5), the score matrix $QK^\top$ can be interpreted as pairwise similarity scores between queries and keys; after softmax normalization, these become attention weights that determine how strongly each query attends to each key.

Value Zeroing (ValueZeroing): In the attention mechanism, the value vector $V_j$ for token $j$ carries contextual content that is mixed into the representation of other tokens via attention weights derived from the $QK^\top$ scores. ValueZeroing is an ablation technique that quantifies how much a context token $j$ contributes to the representation of an output token at position $i$ by recomputing the model’s hidden representation at $i$ after zeroing out the value vector of token $j$ , while keeping all keys and queries fixed (Mohebbi et al. Reference Mohebbi, Zuidema, Chrupała and Alishahi2023).

Let $\tilde {\mathbf{x}}_i$ denote the original representation of the output token at position $i$ , and let $\tilde {\mathbf{x}}_i^{\neg j}$ denote the representation obtained when the value vector of token $j$ is replaced by the zero vector. The context-mixing score between output position $i$ and token $j$ is defined as the cosine distance between these two representations:

(6) \begin{equation} C_{i,j} = 1 - \cos \bigl (\tilde {\mathbf{x}}_i^{\neg j},\,\tilde {\mathbf{x}}_i\bigr ). \end{equation}

Higher values of $C_{i,j}$ indicate that token $j$ induces a larger change in the output representation at position $i$ , and therefore has a stronger influence on that output.

In this subsection, we briefly summarize the XAI attribution methods used in our comparison. Our goal was to include representatives from all three major families of attribution methods while also respecting practical constraints on computation. In particular, many perturbation-based methods are computationally expensive, and generating their attribution maps at scale for our datasets would be infeasible within a reasonable runtime.

6.1 Attributor implementation

Our proposed Attributor network is a lightweight encoder–decoder transformer. The source sentence is tokenized and mapped to learned token embeddings, which are combined with learned positional embeddings and then processed by a 3-layer, 8-head self-attention encoder. The target sentence is embedded separately with its own learned positional embeddings and passed through 3 causal decoder layers. A standard triangular mask enforces autoregressive ordering to ensure that each target position attends only to previous target tokens. The resulting contextualized source and target embeddings are then fed into a modified cross-attention layer. For each target position of each attention head it returns a row attention score vector over source positions, which are essentially vectors of the per-head attention score matrices.

Figure 3.

Column-wise entropy based on Marian-MT attributions.

Figure 3 Long description

Three violin plots compare mean entropy per sentence across different attribution methods for three language pairs. Each plot represents a different language pair: de-en, fr-en, and ar-en. The attribution methods listed vertically include IG, Saliency, LXA, IG, Attribution Method, GSHP, DeepLIFT, Attention, and ValueZering. The x-axis measures mean entropy per sentence in bits, ranging from 0 to 6. Each violin plot shows the distribution of entropy values for each attribution method, with summary statistics provided in the legend. The plots reveal variations in entropy distributions across different methods and language pairs, highlighting differences in how each method attributes importance to input features.

Figure 4.

Column-wise entropy based on mBART attributions.

Figure 4 Long description

Three violin plots compare mean entropy per sentence in bits across different attribution methods for three language pairs using mBART. Each plot represents a different language pair: de-en, fr-en, and ar-en. The attribution methods include IG, Saliency, LXA, IG Attribution Method, GSHAP, DeepLIFT, Attention, and Valueazing. The x-axis represents mean entropy per sentence in bits, ranging from 1 to 7. The y-axis lists the attribution methods. Each violin plot shows the distribution of entropy values for each method, with summary statistics including median, quartiles, and density. The plots reveal variations in entropy distributions across different methods and language pairs, highlighting differences in how each method attributes importance to input features in shaping model outputs.

Figure 5.

Column-wise entropy based on Marian-MT attributions (Marian-MT-generated targets).

Figure 5 Long description

Three violin plots compare mean entropy per sentence using different attribution methods for Marian-MT-generated targets in three language pairs. Each plot represents a different language pair: de-en, fr-en, and ar-en. The attribution methods include IG, Saliency, LXA, IG, GSHAP, DeepLIFT, Attention, and ValueZeroing. The x-axis represents mean entropy per sentence in bits, ranging from 0 to 6. Each violin plot shows the distribution of entropy values for each attribution method. The summary statistics box within each plot provides the mean, standard deviation, and other statistical measures for each method. The plots reveal variations in entropy distributions across different attribution methods and language pairs, highlighting differences in how these methods attribute importance to input features in Marian-MT-generated sequences.

More formally, let $H$ denote the number of heads and $S$ the number of source positions. For each target token $t$ , the cross-attention block produces per-head attention vectors $\vec {s}_{t,h} \in \mathbb{R}^{S}$ :

(13) \begin{equation} \vec {s}_{t,h} = \frac {\vec {q}_{t,h} K_{h}^\top }{\sqrt {d_k}} + \text{mask}, \quad h = 1,\ldots ,H, \end{equation}

where $\vec {q}_{t,h}, K_{h} \in \mathbb{R}^{d_k}$ are the query and key projections.

To aggregate information across heads, we use a weighted mean of these score vectors across the head dimension. The weights are produced by a small MLP which maps each contextualized target token embedding $h_t \in \mathbb{R}^{d}$ to a probability vector over heads $p_t$ :

(14) \begin{equation} \vec {p}_t = \operatorname {softmax} (W_2 \,\mathrm{GELU}(W_1 \vec {h}_t) ) \in \mathbb{R}^{H}, \end{equation}

Finally, the combined logit vector over source positions for each target token is obtained as a weighted mean of the per-head attention scores vectors for that target token, after which it is passed to softmax to obtain a distribution $\hat {a}_t$ over source tokens:

(15) \begin{equation} \hat {\vec {a}}_t = \operatorname {softmax}\bigg(\sum _{h=1}^{H} {p_{t}}_h\, \vec {s}_{t,h}\bigg) \in \mathbb{R}^{S}. \end{equation}

The attribution matrix $\hat {A}$ is essentially stacked $\hat {\vec {a}}_t$ for all $t$ .

The objective function during the training is to minimize the average row-wise Kullback–Leibler divergence between the predicted attribution matrix $\hat {A}$ and the gold attribution matrix $A$ , summed over non-padded target positions:

(16) \begin{equation} \mathcal{L}_{\text{KL}} = \frac {1}{T_{\text{valid}}} \sum _{t \in \mathcal{T}_{\text{valid}}} \mathrm{KL}(A_t \ \vert \vert \ \hat {A}_t), \end{equation}

where $A_t$ and $\hat {A}_t$ denote the gold and predicted distributions over source tokens for target position  $t$ .

6.2 Attributor evaluation

We trained the Attributor on three datasets derived from our earlier experiments, covering all three language pairs. (a) A dataset constructed from Marian-MT attribution maps paired with gold (human) target translations. (b) A dataset constructed from mBART attribution maps paired with gold (human) target translations. (c) A faithfulness-oriented dataset in which Marian-MT attribution maps are paired with targets generated by the Marian-MT teacher model.

As a part of preprocessing, source and target sentences were tokenized according to the respective model’s tokenizers, and attribution maps were normalized in the source dimension to represent a valid probability distribution (to accommodate the KL-divergence loss function). To assess the accuracy of the approximation of their attribution maps by the Attributor, we selected the following metrics:

• • KL-divergence. Being the minimization target, it is the first choice to quantify the difference between the approximated and the gold attributions

• • Overlap@3 The overlap between sets of top-3 valued source token indices per target token for the approximated and gold attribution maps. As highlighted by the Nourbakhsh et al. (Reference Nourbakhsh, Lamsiyah and Schommer2025) we hypothesized that most of the useful signal attributions per target token comes from a small number of the highest-scoring source tokens. The overlap@3 is calculated as

  (17) \begin{equation} \text{overlap@3}(t) = \frac {| \text{Top}_3(a_t) \cap \text{Top}_3(\hat a_t) |}{3} \end{equation}

• • Tau@3 Kendall’s $\tau$ calculated on top-3 values over the source dimension of the gold attribution and values with the same indices but from the approximated attributions. This metric accompanies overlap@3 by quantifying the rank correlation.

6.3 Attributor results

Obtained values for de-en, fr-en, and ar-en pairs are presented in the three settings mentioned above and can be seen in Figures 6–8.

To quantify the link between approximability and downstream gains, we correlate the BLEU scores obtained by Marian-MT when augmented with each attribution method with our three approximability metrics (mean KL divergence, Overlap@3 and Kendall’s $\tau @3$ between gold and predicted target–source maps). We compute BLEU separately for each injection operator (add, average, multiply, replace) and also consider the best BLEU per method across operators.

Across all three language pairs (ar-en, de-en, fr-en) and all operators, BLEU shows a very strong positive correlation with the ‘top-3’ alignment metrics. Pearson’s (r) between BLEU and Overlap@3 lies in the range (r $\approx$ 0.88–0.97), and Kendall’s ( $\tau @3$ ) achieves (r $\approx$ 0.74–0.95). In other words, between roughly 75% and 90% of the variance in BLEU across attribution methods can be explained by how well their target–source patterns can be reconstructed by the Attributor. Rank correlations show identical results: Spearman’s ( $\rho$ ) between BLEU and Overlap@3/( $\tau @3$ ) is consistently high (typically $\rho$ $\approx$ 0.65–0.85), and for the fr-en pair with multiplication injection the ranking of methods by BLEU is exactly the same as the ranking by $\tau @3$ ( $\rho$ = 1.0).

In contrast, KL divergence exhibits only weak and unstable association with BLEU. Pearson correlations between BLEU and mean KL are low–moderate and positive (r $\approx$ 0.27–0.56), while Spearman’s ( $\rho$ ) tends to be low negative and even changes sign for some operators ( $\rho$ $\approx$ −0.26–0.11). This behavior, especially for Pearson, is largely driven by outliers such as ValueZeroing, which combines very high KL divergence with strong BLEU gains. These results suggest that global distribution similarity is a poor predictor of downstream effectiveness, whereas agreement on the locations and ordering of a few top-k source positions is highly predictive.

Taken together, these findings support our central claim: attribution methods whose target-source maps are closer to what an encoder-decoder transformer (Attributor) can reproduce are exactly the ones that yield the largest BLEU gains when injected into the student model. We first measured this closeness with full-distribution KL divergence, but the correlation analysis shows that the best predictors of BLEU are Overlap@3 and Kendall’s $\tau @3$ between the Attributor’s predictions and the gold attribution maps, which correlate very strongly with BLEU, whereas KL shows only weak and unstable correlations. These results suggest that an attribution method tends to be useful precisely when a transformer can reliably recover the same few most-salient source tokens per target token as the teacher. In contrast, matching the full attention distribution beyond the top positions appears much less informative for predicting BLEU gains.

For the other two settings of generated targets, the manifested trend is the same. For mBART, the highest correlation is observed with the multiplication operator. It should be noted that the approximation accuracy is the highest for Attention from mBART: the last source token consistently gets the highest attribution scores and their reconstruction is easier. However, there are still residual scores for other tokens whose presence can guide the translation. The regression plot of the above approximation metrics for the three language pairs and the corresponding outcomes is provided in the figures 9–11 (see Appendix).

In short, the student benefits from what it can imitate: attribution methods for which target tokens a transformer can reliably reconstruct the same top-3 source tokens are precisely those that give student models the largest gains.

Figure 6.

KL-divergence, Overlap@3, and $\tau @3$ for the prediction of attribution on Marian-MT attribution and gold (human) data.

Figure 6 Long description

The image contains six bar graphs arranged in two rows of three. The top row shows mean KL divergence for three language pairs: ar-en, de-en, and fr-en. Each graph compares different XAI methods: Attention, ValueZeroing, LGxA, Saliency, DeepLIFT, IXG, IG, and GSHAP. The y-axis represents mean KL, with lower values being better. The bottom row shows top-k=3 agreement for the same language pairs and XAI methods. The y-axis represents the score from zero to one. Each graph includes two bars: one for Overlap k=3 and one for Kendall τ k=3. The graphs illustrate the performance of various XAI methods in predicting attribution for Marian-MT models compared to human gold data. All values are approximated.

Figure 7.

KL-divergence, Overlap@3, and $\tau @3$ for the prediction of attribution on mBART attribution and gold (human) data.

Figure 7 Long description

The image contains six bar graphs arranged in two rows of three. The top row shows KL divergence for three language pairs: ar-en, de-en, and fr-en. Each graph compares multiple attribution methods, including Attention, ValueZeroing, LGXA, DeepLIFT, Saliency, IG, IXG, and GSHAP. The y-axis represents mean KL values, with lower values indicating better performance. The bottom row displays top-k agreement for the same language pairs, with two metrics: Overlap k equals 3 and Kendall tau k equals 3. The y-axis represents scores ranging from 0 to 1. Each graph compares the same attribution methods, with blue bars for Overlap and orange bars for Kendall tau. The graphs illustrate the performance of different attribution methods in predicting human annotations for mBART models. All values are approximated.

Figure 8.

KL-divergence, Overlap@3, and $\tau @3$ for the prediction of attribution on Marian-MT attribution generated target.

Figure 8 Long description

The image contains six bar graphs arranged in two rows of three. The top row shows mean KL divergence for language pairs ar-en, de-en, and fr-en. Each graph compares different methods such as Attention, ValueZeroing, LGxA, IG, Saliency, DeepLIFT, ixG, and GSHAP. The y-axis represents mean KL, with lower values being better. The bottom row shows top-k agreement for the same language pairs, with two metrics: Overlap k equals 3 and Kendall tau k equals 3. The y-axis represents the score from 0 to 1. Each graph compares the same methods as the top row. The bars indicate the performance of each method, with error bars showing variability. The graphs illustrate the effectiveness of different attribution methods for various language pairs.