2.1 Fact-checking

Fact-checking is based on the confirmation or rejection of claims made in a text piece with the use of explicit references to knowledge sources, ideally credible ones such as Wikipedia (Thorne et al. Reference Thorne, Vlachos, Christodoulopoulos and Mittal2018). Automated fact-checking is done in a two-step procedure: in the first step relevant, articles are retrieved from the knowledge base, and in the second step, inference (refuting or supporting the input claim) is performed by a neural network.

The approaches to fact-checking go beyond Wikipedia: Wadden et al. (Reference Wadden, Lin, Lo, Wang, van Zuylen, Cohan and Hajishirzi2020) introduce scientific claim verification and demonstrate that domain adaptation techniques improve performance compared to models trained on Wikipedia or political news. In Augenstein et al. (Reference Augenstein, Lioma, Wang, Chaves Lima, Hansen, Hansen and Simonsen2019), the authors use a data set from fact checking websites and apply evidence ranking to improve veracity prediction. The research by Leippold and Diggelmann (Reference Leippold and Diggelmann2020) introduces a data set for verification of climate change-related claims and adapts the methodology of Thorne et al. (Reference Thorne, Vlachos, Christodoulopoulos and Mittal2018). Another study (Rashkin et al. Reference Rashkin, Choi, Jang, Volkova and Choi2017) probes the feasibility of automatic political fact checking and concludes that stylistic cues can help determine the truthfulness of text.

Fact-checking and fake news detection have been the main topics of CLEF competitions since 2018. In the 2018 edition, the second task “Assessing the veracity of claims” asked to assess whether a given check-worthy claim made by a politician in the context of a debate/speech is factually true, half-true, or false (Nakov et al. Reference Nakov, Barrón-Cedeño, Elsayed, Suwaileh, Màrquez, Zaghouani, Atanasova, Kyuchukov and Da San Martino2018). The data set consists of less than 90 verified statements from a presidential debate. In 2019 (Elsayed et al. Reference Elsayed, Nakov, Barrón-Cedeño, Hasanain, Suwaileh, Da San Martino and Atanasova2019), ‘Task 2 Evidence and Factuality” asked to (A) rank a given set of web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) classify these same web pages according to their degree of usefulness for fact-checking the target claim, (C) identify useful passages from these pages, and (D) use the useful pages to predict the claim’s factuality. In CLEF 2020 (Barrón-Cedeño et al. Reference Barrón-Cedeño, Elsayed, Nakov, Da San Martino, Hasanain, Suwaileh, Haouari, Babulkov, Hamdan, Nikolov, Shaar and Ali2020), “Task 4: Claim Verification” asked to predict the veracity of a target tweet’s claim by using a set of web pages and potentially useful snippets contained within. The data sets for the 2019 and 2020 tasks are in Arabic. The CLEF 2021 “Task 3a: Multi-Class Fake News Detection of News Articles” (Nakov et al. Reference Nakov, Da San Martino, Elsayed, Barrón-Cedeño, Mguez, Shaar, Alam, Haouari, Hasanain, Babulkov, Nikolov, Shahi, Struß and Mandl2021b) focused on multi-class fake news detection and topical domain detection of news articles in several languages, including English. The data are not publicly available. The CLEF 2021 “Task 2 Detecting previously fact-checked claims from tweets” used data from Snopes and ClaimsKG (Tchechmedjiev et al. Reference Tchechmedjiev, Fafalios, Boland, Gasquet, Zloch, Zapilko, Dietze and Todorov2019) to rank previously fact-checked claims in order to measure their usefulness.

The fully automated fact-checking pipeline could include steps such as identifying claims worthy of fact-checking, followed by detecting relevant previously checked claims and then only focus on verification of selected, check-worthy claims (Nakov et al. Reference Nakov, Da San Martino, Elsayed, Barrón-Cedeño, Mguez, Shaar, Alam, Haouari, Hasanain, Babulkov, Nikolov, Shahi, Struß and Mandl2021b). Systems like this can facilitate the work of fact-checking professionals (Nakov et al. Reference Nakov, Corney, Hasanain, Alam, Elsayed, Barrón-Cedeño, Papotti, Shaar and Da San Martino2021a).

2.2 Knowledge lean

The second type of methods infer veracity only from the input text and do not use any external sources of information. Such approaches are based on various linguistic and stylometric features aided by machine learning.

Features such as n-grams, LIWC psycholinguistic lexicon (Pennebaker et al. Reference Pennebaker, Boyd, Jordan and Blackburn2015), readability measures, and syntax were all used in Pérez-Rosas et al. (Reference Pérez-Rosas, Kleinberg, Lefevre and Mihalcea2018). Then, an SVM classifier was applied to predict veracity. A similar set of features including n-grams, parts of speech, readability scores, and the General Inquirer lexicon features (Stone et al. Reference Stone, Dunphy, Smith and Ogilvie1966) was used in Potthast et al. (Reference Potthast, Kiesel, Reinartz, Bevendorff and Stein2018). Interestingly, they argue that style-based fake news classification does not generalize well between news of both partisan orientations of the USA. A combination of features at four levels—lexicon, syntax, semantic, and discourse—was used in Zhou et al. (Reference Zhou, Jain, Phoha and Zafarani2020). They applied classification methods such as SVM (with linear kernel), random forest (RF), and XGBoost. In yet another study, (Przybyla Reference Przybyla2020) designed two classifiers: a neural network and a model based on stylometric features to capture the typical fake news style. Style analysis allowed to extract sensational and affective vocabulary that is typical of fake news.

Performance drops in different domains were carefully studied by Silva et al. (Reference Silva, Luo, Karunasekera and Leckie2021) and concluded with a new framework that jointly preserves domain-specific and cross-domain knowledge. This allowed for a new cross-domain classifier to be trained on data selected by an unsupervised technique (and manually labelled).

The problem of excessive dependence on training sets and the low robustness of text-based fake news detection was observed by Janicka et al. (Reference Janicka, Pszona and Wawer2019). The authors used four different fake news corpora to train models in both in-domain and cross-domain settings. They concluded that the results achieved by models trained and tested on the same corpus (in-domain) are unrealistically optimistic with regard to the true performance of the models on real-world data. The performance in the cross-domain setting was on average 20% worse than in the case of the in-domain setting. Compared to Janicka et al. (Reference Janicka, Pszona and Wawer2019), our work is a further step forward: our goal is to find the means to improve the robustness and cross-domain performance of fake news detection models.

4.1 Feature space

1. (1) Bag-of-words (BOW) – in the first approach, for each data set we calculated the term frequency–inverse document frequency (TF-IDF) of uni-, bi-, and tri-grams.

2. (2) Linguistic features – using various sources of information, we created a set of 271 features containing semantic, syntactic, and psycholinguistic information. More than 180 features stem from the General Inquirer (Stone et al. Reference Stone, Dunphy, Smith and Ogilvie1966), a tool providing a wide range of psycholinguistic categories for analyzing text content in terms of sentiment, emotions, as well as multiple other social and cognitive categories. We enriched the data with hand-crafted dictionaries of linguistic hedges and exclusion terms, which are presented in Table 4. Each feature was normalized by the number of all words in the text. Similarly, using the spaCy library,Footnote i we added part-of-speech information in each text. We also added the sentence-level subjectivity measure from an LSTM-based modelFootnote j that was trained on a data set released by Pang and Lee (Reference Pang and Lee2004). Finally, various readability indicators were used: Flesh-Kincaid (Flesch Reference Flesch1948; Kincaid et al. Reference Kincaid, Fishburne, Rogers and Chissom1975), ARI (Smith and Senter Reference Smith and Senter1967), Coleman-Liau (Coleman and Liau Reference Coleman and Liau1975), etc.

3. (3) GloVe (Pennington, Socher, and Manning Reference Pennington, Socher and Manning2014) – we used 100-dimensional word embeddings that were pre-trained using aggregated global word co-occurrence statistics found on Wikipedia and the Gigaword corpora.

4. (4) Universal Sentence Encoder (Cer et al. Reference Cer, Yang, Kong, Hua, Limtiaco, St. John, Constant, Guajardo-Cespedes, Yuan, Tar, Strope and Kurzweil2018) – a versatile sentence embedding model based on the Transformer architecture trained in a transfer learning manner. It converts text into 512-dimensional vector representations, which can capture rich semantic information. They can be used in various downstream tasks and have proven to yield good results in many text classification problems.

5. (5) ELMo embeddings (Peters et al. Reference Peters, Neumann, Iyyer, Gardner, Clark, Lee and Zettlemoyer2018) – we used this model to represent each text by a 1024-dimensional vector. These features were used exclusively for visualization, to compare whether they convey more information than the embeddings obtained from the Universal Sentence Encoder.

4.2 Models

As the baseline approach, we used a support vector machine classifier with linear kernel trained on bag-of-words features. The default parameters were used: C=1 and squared hinge loss as the loss function.

Table 4. Dictionaries of hedges and exclusions

To create a classifier based on linguistic features, we tested various machine learning algorithms from the scikit-learn (Pedregosa et al. Reference Pedregosa, Varoquaux, Gramfort, Michel, Thirion, Grisel, Blondel, Prettenhofer, Weiss, Dubourg, Vanderplas, Passos, Cournapeau, Brucher, Perrot and Duchesnay2011) library, such as support vector machines, stochastic gradient descent, extra trees, and gradient-boosted trees (XGBoost). The default hyperparameter values were used in each case. The best test set performance for all five data sets was obtained by XGBoost, which was used in further experiments.

Next, we tested a bidirectional-LSTM architecture initialized with 100-dimensional GloVe embeddings. The scheme is shown in Figure 2 on the right. According to a benchmark study on fake news detection methods by Khan et al. (Reference Khan, Khondaker, Afroz, Uddin and Iqbal2021), this classifier achieved the best results among the tested approaches. Both the output dimension of the LSTM and the number of time steps were set to 100. The outputs of the last LSTM units (one for each direction) were concatenated and followed by dropout and a dense classification layer with the sigmoid activation function. We used the ADAM optimizer to minimize the binary cross-entropy loss. The model was implemented using the PyTorch library.

Figure 2. The architecture of two models: based on USE embeddings (left) and bi-directional LSTM using GloVe embeddings (right).

As a basic model utilizing USE embeddings, we used the LinearSVC classifier which takes as input a vector representation of whole texts. This approach proved to be very successful in some text classification tasks with limited data thanks to SVM’s robustness and regularization (Xu, Caramanis, and Mannor Reference Xu, Caramanis and Mannor2009). Since one of our goals was to compare different deep domain adaptation methods, we also investigated the performance of a simple neural model utilizing USE embeddings as a baseline. The architecture is presented in Figure 2 (left).

The embeddings were used as the input and were followed by three blocks, each constituted of a dense layer with residual connection and a normalization layer. Finally, the obtained representation was projected to two classes by two dense layers. Hyperparameters such as dropout and dense layer dimensions were fine-tuned using the Kaggle data set, while the epoch count was set to 200 based on the mean scores obtained for all data sets.

Moreover, we tested the BERT language model (Devlin et al. Reference Devlin, Chang, Lee and Toutanova2019), which is based on the Transformer architecture and has achieved state-of-the-art results in many NLP tasks. We used the TensorFlow implementation and tested sequence lengths of 32, 128, and 512. The number of training epochs was limited to 5, as there was no further improvement in the classification performance. Typically, one or two epochs provided maximum accuracy. Therefore, we finally used a sequence length of 128 and trained the model for 2 epochs.

4.3 Evaluation of classifiers

Comparison of classifier performance is a difficult task and cannot be limited to the analysis of the main score, often calculated using k-fold cross validation. Although the model with the best mean performance is expected to be better, this is not always true. Such a simple approach can be misleading, as the difference in performance might be caused by chance. Therefore, to be sure that one model provides significantly higher accuracy than the others, statistical tests should be performed. Fortunately, there is a variety of statistical methods enabling the selection of best performing machine learning model.

As the first step we decided to use a Cochran’s Q test (Cochran Reference Cochran1950), which is used to determine if there are differences on a dichotomous dependent variable (which can take only one of two possible values) between more than two related groups. It tests the null hypothesis that the proportion of “successes” is the same in all groups. We applied this test to the predictions of the investigated models. If the null hypothesis was rejected, multiple pairwise comparisons between groups were performed as the second step.

We applied the Wilcoxon signed-ranked test (Wilcoxon Reference Wilcoxon1945; Demšar Reference Demšar2006) to verify if two samples differ significantly from each other, testing the null hypothesis that the related paired samples are drawn from the same distribution. It is a non-parametric version of the paired Student’s t-test. Contrary to Student’s t-test, the Wilcoxon signed-rank pair test does not require normally distributed values.

Figure 3 presents how the pair-wise evaluation and comparison of models under different settings were performed. Instead of using raw models predictions like in Cochran’s Q test, we compared the accuracies obtained during k-fold cross validation. In our experiments, we used 10-fold cross validation and set the significance level to 0.05. In the results tables, we marked the groups of models for which the Cochran’s Q test revealed no differences with a gray background. Moreover, within the remaining groups, the pairs of models for which the Wilcoxon signed-ranked test indicated a p-value $> 0.05$ (non-significant differences) are marked with the same letters in the superscript.

Figure 3. The scheme of validation and comparing machine learning models under different conditions: in-domain and cross-domain setting.

4.4 In-domain results

We compared the performance of the above classifiers in data set-specific or in-domain settings using 10-fold cross validation. The obtained results are presented in Table 5. In general, all classifiers performed significantly better on the Kaggle and ISOT data sets when compared with the others. This fact does not seem surprising considering that those collections are several times larger than the others.

Table 5. In-domain accuracy of different fake news detection models on five data sets

^a We could not replicate the results of Perez-Rosas et al., who reported an accuracy of 0.62 using similar settings: a linear SVM classifier based on TF-IDF values for unigrams and bigrams with 5-fold cross-validation; with these settings we achieved an accuracy of 0.43. The accuracy is significantly higher for char-level settings - 0.66. The value reported here is for 10-fold cross-validation of an SVM classifier based on TF-IDF values for unigrams, bigrams, and trigrams.

Taking into account the results on all data sets, the best-performing classifier is XGBoost trained on linguistic features. It achieved the highest accuracy on two out of five data sets: Kaggle and AMT. Moreover, its accuracy on the three remaining data sets—ISOT, LIAR, and FNN—was lower than the accuracy of the best-performing classifiers by only 0.01, 0.05, and 0.05, respectively. Finally, its advantage over other models on the AMT data set is enormous: the obtained accuracy is higher by 0.13 when compared to the runner-up. It seems that this data set is the most challenging for the other methods. The accuracies of models using USE embedding are very low: 0.57 obtained by LinearSVC and 0.51 achieved by a neural network based on dense layers. The poor performance of classifiers based on embeddings in the case of the AMT data set is discussed further. Even worse results were provided by LinearSVC using bag-of-words features (0.40). The reason behind such poor results is strong overfitting to the training data. In 10-fold cross validation settings, the accuracy on the training sets regularly reached a value of almost 1.0, while on the test parts it fluctuated around 0.40. In the case of char-level settings (using character-level n-grams), the values are respectively 0.82 and 0.66, but we decided to focus on word-level analysis. The comparison of our results with the best existing ones (as in Table 3) revealed that we managed to outperform the accuracy on the ISOT data set (0.92 vs. 0.99) reported in Ahmed et al. (Reference Ahmed, Traore and Saad2017). Our results are similar in the case of Kaggle; however, we did not obtain a higher accuracy than Zhou et al. (Reference Zhou, Jain, Phoha and Zafarani2020) for the FNN data set. As for the LIAR data set, we obtained significantly higher accuracy than the best results reported by Wang (Reference Wang2017), Popat et al. (Reference Popat, Mukherjee, Yates and Weikum2018). However, these values cannot be directly compared. Six classes were used in the original analysis, while we narrowed it down to two. It should be noted that we used the same method for all data sets (without hyperparameter tuning for a specific data set). Hence, in our opinion, the results are satisfactory.

We also investigated which linguistic features are the most important for classifying fake news. The results are shown in Table 6 and indicate that among the linguistic features, part-of-speech tags are the most informative. The addition of other linguistic features (the first row of Table 6) did not significantly improve the accuracy—only by 0.03 for AMT and 0.04 for FNN. In the case of LIAR data set, it even led to a slight decrease in performance.

Table 6. In-domain accuracy of different fake news detection models on five data sets for various linguistic features

The obtained results indicate that designing a high-accuracy classifier is more challenging for some data sets than for others. To get insight into the arrangement of data in both high-dimensional feature spaces—USE embeddings (512) and linguistic features (271)—we applied t-Distributed Stochastic Neighbor Embedding (t-SNE). The vectors containing linguistic features were standardized before dimensionality reduction. Figure 4 presents the obtained results for three selected data sets.

This visualization can explain why the classification based on USE embeddings failed in the case of AMT. For a large number of points related to true texts, points related to fake texts occurred in a very close distance. This might be caused by the way in which this data set was created. Initially, legitimate news pieces were collected from a variety of websites (their veracity was confirmed by manual fact-checking). However, the fake news collection did not originate in any existing source, but was written specifically for this data set. Amazon Mechanical Turk workers were asked to write a fake version for every legitimate news story. At the same time, they were requested to imitate the journalistic style and preserve the names mentioned in the original news. As a result, both the originals and their fake counterparts were closely related. Therefore, their USE embeddings were often similar (USE embeddings proved to be successful in finding semantic similarity).

For a comparison, we also visualized more expressive embeddings—ELMo (Figure 4b). The t-SNE plots for both USE and ELMo embeddings appear quite similar. Still, the ELMo representations for paired true and fake texts were often located close to each other. However, in the case of linguistic representation the samples in AMT data sets were much more scattered, that is vectors for related original and fabricated texts were more separated. It indicates that USE and ELMo embeddings are more related to the content of the text (its semantics), while the linguistic features are not strongly related to its meaning but contain information about part-of-speech tags, readability indices, etc. Even if both texts cover the same topic they are written differently, which results in a larger change in linguistic features than their embeddings. Therefore, the linguistic feature vectors of the fake and true texts are better separated, which allows for more accurate classification.

In the case of LIAR, linguistic representations of the data tended to form many well-separated clusters, while USE embeddings seemed more dispersed. This might be due to the length of the texts in the LIAR data set. As they contain single short sentences, their linguistic representations contain many zeros. Unfortunately, true and fake texts do not form separate clusters, which makes classification very difficult. The visualization of the data arrangement suggests that USE embeddings might work better for this data set.

Figure 4. t-SNE visualization of USE embeddings, ELMo embeddings, and linguistic features.

The distributions of USE embeddings and linguistic features obtained for the Kaggle data differed significantly. Most strikingly, the linguistic representations of fake news formed one large cluster and a few smaller ones. As for USE embeddings, they formed smaller clusters. The points related to both classes rarely appeared in the same clusters. The classifiers utilizing USE embeddings reached an accuracy of 0.93. An even higher accuracy of 0.99 was achieved when linguistic representation was used.

To conclude, the use of linguistic and psycholinguistic features proved to be the most versatile approach for the detection of fake news. However, it only works for longer texts and not for very short ones, like in the LIAR data set.

As the t-SNE plots revealed large differences between the studied data sets, we decided to see how the classification accuracy relates to the number of training samples.

The goal of this analysis was to show that variability in classification accuracy was not solely a result of different data set sizes. We also hoped to observe how quickly the classification models learn and to approximate how accuracy increases when adding training samples. Lastly, the behavior of classifiers with limited ground truth access was examined.

The classifiers were trained on randomly selected samples that maintained class representations. The remaining samples were used as the test set. Due to large variance in the results, especially for a small number of training samples, the whole procedure was repeated 10 times and the obtained accuracy scores were averaged.

Figure 5 shows the results obtained for two classifiers: LinearSVC and XGBoost. We compared two representations: USE embeddings and linguistic features. Interestingly, the obtained plots reveal significant differences. The most expected are the plots for Kaggle and ISOT: the larger the training set, the higher the accuracy accompanied by lower variance. As it turned out, LinearSVC performed slightly better than XGBoost when trained on USE embeddings. The behavior of models trained on FakeNewsNet was very similar to the classifiers trained on Kaggle and ISOT. Unfortunately, due to the much smaller size of FakeNewsNet, it could not be compared in larger ranges. Nevertheless, the observed similarity suggests that the addition of new labeled samples would improve classification accuracy.

Figure 5. The dependence of accuracy of fake news detection on the size of training set.

The plots obtained for the AMT data set look strikingly different. The most surprising is the behavior of the XGBoost classifier trained on USE embeddings. Its accuracy did not exceed 0.5 and decreases with the increasing size of the training set. Such unusual behavior occurs exclusively for the AMT data set, which indicates that it is related to the nature of USE embeddings of texts contained in that data set (as previously discussed). t-SNE visualization of these embeddings (Figure 4) shows that they often occur in highly similar pairs: the embeddings of the original (true) text and the fake one. As a reminder, the USE embeddings for original and fabricated texts are often close to each other, as they cover the same topic. Interestingly, LinearSVC deals significantly better with such specific data.Footnote k The situation was different in the case of classifiers trained on linguistic features: the accuracy for both LinearSVC and XGBoost increased upon expansion of the training set for the AMT data set. Just as for ISOT and Kaggle, LinearSVC also performed slightly better than XGBoost on a small training set. When its size exceeded 50 samples, XGBoost gained a large advantage.

Models trained on LIAR exhibited a still different behavior. The increase of the training set size had little impact on classification accuracy. Moreover, for both types of representations, the XGBoost model provided better performance.

The obtained results clearly show that the investigated fake news corpora differ significantly. Only Kaggle and ISOT are similar to each other. It also seems that FakeNewsNet has a lot in common with these data sets, but its size is two orders of magnitude lower. Most likely this is the reason why models trained on this corpus did not achieve high accuracy.

Moreover, for really small data sets, ranging from 5 to 50 samples, the representation by linguistic and psycholinguistic features provided better performance. It indicates that such representations are more universal. The advantage of linguistic representation over USE embeddings on small data sets might result from differences in their dimensionalities: 271 and 512, respectively. Furthermore, in the case of AMT, models trained on USE embeddings proved ultimately unsuccessful.

4.5 Cross-domain results

We also tested cross-domain performance: the models were trained on one data set and tested on other data sets. The results of 10-fold cross validation are presented in Table 7, where we can observe a substantial performance decrease of the classifiers within this test setting.

Table 7. Cross-domain accuracy. Averages are calculated from cross-data set results (not in-domain test sets). Nonsignificant pairwise comparisons are marked with superscript letters (Wilcoxon test), nonsignificant group comparisons are greyed out (Cochran Q test)

The data set-specific accuracy of classifiers trained on Kaggle and ISOT exceeded 0.9. However, when the same classifiers were tested on data sets different from the one used for training, the accuracy decreased to even slightly below 0.5 in some cases. This shows that those models are good in recognizing fake news within one data set, but not fake news in general, that is independently from training data. The most universal method which achieved the highest accuracy averaged over all train–test pairs turned out to be LinearSVC on BOW. In general, the highest scores in cross-domain setting were obtained for the ISOT-Kaggle pair due to the similarity of both data sets. The lowest accuracy of models trained on Kaggle and ISOT was achieved by XGBoost with linguistic features tested on the AMT data set. While Kaggle and ISOT data sets are similar in terms of size and structure, they significantly differ form the AMT data set. However, the best method was LinearSVC on BOW. It does not mean that the topics were the same. Most likely, similar stylistic means were used in fake news across various data sets, which is recognized by this classifier.

Models trained on AMT showed the most unpredictable results. LinearSVC with BOW features, which obtained very low accuracy in data set-specific settings, turned out to perform very well on ISOT data. This result is surprising, with 0.4 accuracy on its test data and 0.78 on the ISOT test set. We examined these results, looking for the words that have the greatest impact on the predicted label. The AMT’s most significant n-gram features were neutral words such as “their”, “be”, “this”, “they”, and “are”. This was due to the way the data set was created. For each true text, there is a corresponding false one on the same topic, as we explained in Section 3. These influential words are not as abundant in the AMT test set, but they are consistent with the fake news style in other data sets, which explains why the results vary so much.

As far as the linguistic features are concerned, the classifier trained and tested on AMT obtained good results (0.80). At the same time, when tested on Kaggle and ISOT, their accuracy was 0.42 and 0.38, respectively, which were the lowest cross-domain results. Also, we can observe that some classifiers trained on different data sets managed to perform better on AMT than classifiers trained on AMT. For instance, LinearSVC using bag-of-words achieved an accuracy of 0.57 on AMT when trained on FakeNewsNet, and 0.61 when trained on Kaggle, which was higher by 0.21 than its results for data set-specific settings. This shows that the small size of the AMT data set hinders its effective training.

On the LIAR data set, the highest cross-domain accuracy was 0.63, achieved by several classifiers.

In general, the decline in the quality of the model predictions in this scenario compared to the in-domain setting turned out to be very large. This phenomenon is at least partly due to the heterogeneous content of the data sets, containing texts of different nature, topics, and structure. Different ways of defining and creating false and true texts, as well as the time of their writing, may also explain this decline in quality.

7.1 Intersection of top features

During the search for features that are important for fake news detection across all data sets, we investigated the intersection of top n (n = 20, 50, 100, 150, 200) linguistic features from each data set. As Table 10 shows, the Mutual Information method—compared with MRMR—resulted in a considerably lower number of important common features across all five data sets. The intersection of the top 20 features turned out to be empty and there was only one feature in the top-50 intersection: TIME. This category comes from the General Inquirer dictionary and describes words indicating time consciousness. Selected features are shown in Table 11.

Table 10. The number of top n features common for all data sets (5)

Table 11. Intersection of top n linguistic features

When the MRMR method was used, two POS features turned out to be present in the intersection of the top 20 features: the percentage of nouns and adjectives in the text. The third feature found (complex_words) was the ratio of complex words to all words in the document. This is one of the indicators used to measure text readability.

Figure 6 shows the frequencies of three relevant parts of speech: nouns, adjectives, and numerals. The importance of the percentage of nouns in fake news classification is in line with the research by Horne and Adali (Reference Horne and Adali2017). Their study revealed that the number of nouns is lower in fake news than in real news, and close to that found in satire. Figure 6a illustrates that all data sets except LIAR showed this trait. The LIAR data set consists of claims collected from transcripts, speeches, news stories, press releases, and campaign brochuresFootnote l by PolitiFact journalists, which makes the LIAR texts different in length and style from the other data sets.

Figure 6. Part-of-speech statistics.

When it comes to the average percentage of adjectives in a sentence (Figure 6b), we can observe differences in feature distribution between classes for Kaggle and ISOT data sets. However, in the Kaggle data set fake news had more adjectives than in real news, while in the ISOT data set it was the opposite—real news had on average more adjectives than fake news. It is therefore hard to draw any conclusions about the role of adjectives in distinguishing fake and real news based on these facts alone. One possible continuation of this research may be to perform expert analysis on journalistic style traits present in different data sets, which may explain some discrepancies in feature distribution. However, our aim is to explore the possibility of a universal classification model in the area of fake news detection, so we put aside differences in news writing styles to be examined in more depth in future research.

Aside from the number of complex words, other readability indicators also turned out to be important when it comes to fake news classification. The intersection of the top 50 features from each data set (30 features) contained 23% readability-related features, whereas readability indicators made up only 7% of all features. The well-known readability measures are shown in Table 11 and consist of the Gunning–Fog index (Gunning Reference Gunning1969), the number of long words, and the type-token ratio (lexical diversity), among others.

We can hypothesize that to fulfill their role of misinformation, fake messages need to be easily understood, whereas real news seeks to express the complexity of the reported events and thus may require more sophisticated language. The importance of readability in fake news detection is also confirmed by Pérez-Rosas et al. (Reference Pérez-Rosas, Kleinberg, Lefevre and Mihalcea2018) and Horne and Adali (Reference Horne and Adali2017). What is more, it also proved to be true for different languages (Santos et al. Reference Santos, Pedro, Leal, Vale, Pardo, Bontcheva and Scarton2020). However, the hypothesis about the source of differences in text readability between fake and real news should be further investigated via sociolinguistic studies.

Analogically, we investigated the intersection of bag-of-words features. We checked how many of the top n features (n=20, 50, 100, 150, 200, 300, 500) were common for all data sets. The results are presented in Table 10. As we can see, the two feature selection methods yielded distinctly different results. Mutual Information provided intersections ranging between 39 and 57% of the top features. For example, considering the top 20 features selected for each data set, half of them (10) were common for all data sets. This may be surprising when we consider that the BOW feature space is very vast, containing thousands of n-grams. For MRMR, we decided to take into consideration only the 5000 most frequent features in a given corpus to prevent the algorithm from selecting the rare ones. Nevertheless, the set of common features selected with the MRMR algorithm was empty for every n value.

In Figure 6c, we can see the general tendency of fake news to contain fewer cardinal numerals, which holds for every data set. In Horne and Adali (Reference Horne and Adali2017), the authors observed that fake news contains fewer technical and analytical words. Our findings are in line with their study showing that fake news is less likely to present concrete numerical data, which are rather common in technical or analytical writing.

The results of the MI feature selection on BOW data are consistent with the observation made by investigating linguistic features (see Table 11), where the percentage of pronouns, prepositions, and conjunctions per text was selected as an important linguistic feature that was common for every data set.

Furthermore, we checked the distribution of these features within each class and observed the tendency for fake news to have on average more pronouns than real news, which was especially visible in the AMT, ISOT, and FNN data sets. A more in-depth analysis of pronoun distributions in fake news versus real news conducted by Yang et al. (Reference Yang, Zheng, Zhang, Cui, Li and Yu2018) showed fewer first- and second-person pronouns in fake news, but a higher number of third-person pronouns. Considering the average number of prepositions, we may observe that they were dominant within the real news class in LIAR, Kaggle, and ISOT data sets, and only AMT showed the opposite pattern. According to Santos et al. (Reference Santos, Pedro, Leal, Vale, Pardo, Bontcheva and Scarton2020), prepositions are indicators of text cohesion, which is further studied in the paper for the Brazilian Portuguese language. Our result can therefore be a premise to conduct such research for the English language as well.

The purpose of this experiment was to investigate if there is a way to create a classifier that will show similar accuracy independent of the origin of the test set based on five different data sources. We used feature selection methods to find out which features are good indicators of fake or real news across all five data sets. Further, we compared selected feature distributions between classes within the studied data sets. In some cases (nouns, pronouns, cardinal numbers, prepositions), we found traits common for most of the data sets, whereas in other cases (e.g. adjectives) features played an important role for most data sets but the class they indicated differed between data sets. One of the plausible continuations of this research is to investigate the differences between journalism styles and compare them to the style of fake news or possibly cluster fake news based on linguistic features and try to uncover some differences in the style of fake news itself.

7.2 Intersection-based classifier

In this subsection, we present the results of classifiers based on the intersection of the top features. We trained a model on one data set using selected features and tested it on another. The procedure of feature selection was similar to the one presented in the previous subsection; however, only three data sets were used to create the top feature intersection instead of five. The test set was not supposed to take part in the creation of the model, so it could not be a part of the process of feature selection. Also, the training set was excluded since selecting top features common for several data sets is meant to reduce the bias towards training data and provide more universal results.

Moreover, a lower number of data sets in the process of feature selection resulted in a larger number of selected features. For example, the five-data set intersection of the top 50 linguistic features selected with the MI algorithm had only one feature (see Table 10), whereas for three data sets the number of features varied between 1 and 7 (depending on the selected data sets). Similarly, when the same settings were considered for bag-of-words features, the intersection of all five data sets contained 30 items (see Table 10), while the number of common features for three data sets varied between 31 and 42.

For the intersection-based classifiers, we chose the same algorithms as those described in Subsection 4.2: XGBoost on linguistic features and LinearSVC on BOW. We tested them in a cross-domain manner, as in Subsection 4.5, but using only selected features. For each train-test pair, a set of features was selected based on the intersection of the top features of the three remaining data sets. For linguistic classifiers, we considered two methods of selecting top features: Mutual Information and Minimum Redundancy Maximum Relevance. With the bag-of-words feature space the MRMR method selected very different features for each data set, so the intersection was empty (even for only three data sets). Therefore, we could not create an intersection-based classifier for BOW using MRMR and we used only the Mutual Information algorithm.

We adopted average gain as the measure of the overall performance of a given feature selection setting. To calculate the overall performance, we first subtracted full-feature model cross-domain results (Table 7) from the results of the intersection-based classifiers for each train–test pair. Then, we added up the individual gains and divided the result by the total number of pairs (20) to obtain the average gain.

Figure 7 shows average gains for varying number of top features (n). We can observe that for the MRMR method, none of the investigated values of the n parameter provided any improvement in the results. As for the Mutual Information method, n=100 provided the best results for both linguistic and BOW features. On average, the accuracy improved by 4% and 1%, respectively.

Figure 7. Average cross-domain accuracy gain for various numbers of top features (n).

Table 12 presents the cross-domain results of classifiers trained on the intersection of the top 100 features selected with the Mutual Information method. In the last column, we present the average gain for all pairs with a given data set used for training, while the last row presents the average gain for all pairs when the data set was used for testing.

Table 12. Accuracy of classifiers based on Mutual Information, intersection of the top 100 linguistic features

Linguistic classifiers showed an overall greater improvement (4% averaged over all train-test pairs) than the ones based on BOW (average gain 1%). The accuracy improved for almost every training set, with the most significant gain for Kaggle (6%) and AMT (11%). This proves that a carefully designed feature space, which takes into account feature importance computed for several different data sets, can improve generalization of a model trained on a particular data set. Although the BOW classifier had a lower average gain than the linguistic one, it showed higher accuracy, especially when tested on Kaggle, ISOT, and AMT. As the BOW classifier performed better from the beginning, there was less room for improvement. This may explain why this method was less effective in shifting the cross-domain performance.

8.1 Results of domain adaptation methods

The results of the above described domain adaptation machine learning techniques are presented in Tables 13 and 14. They were applied to two feature spaces: linguistic features and embeddings obtained from the Universal Sentence Encoder. Figure 8 visualizes the change in classification accuracy for each train–test pair upon application of domain adaptation methods. Positive values are marked in red, while negative values are marked in blue. As a reference, XGBoost results were used for classifiers trained on linguistic features. LinearSVC was used for models utilizing USE embeddings.

Unfortunately, there seems to be no universal method that would guarantee an improvement for all configurations of the train–test pairs. While there was a significant increase in accuracy for some pairs, others showed a decrease in performance. In general, the domain adaptation methods provided a higher improvement for linguistic classifiers than for the ones based on USE embeddings. The highest overall improvement (averaged over all train–test pairs), equal to 3%, was achieved by subspace alignment applied to a classifier based on linguistic features.

9.1 Results of deep domain adaptation methods

The selected deep domain adaptation methods were applied to two models whose architectures are depicted in Figure 2. One of them utilized USE text embeddings, while the other was based on bi-directional LSTM using GloVe embeddings. In the context of domain adaptation, these models (except the part devoted to the final classification) are considered as features extractors. The performance of deep domain adaptation methods was evaluated for each source–target (train–test) pair of data sets. The entirety of one data set was used as the source, while the other was used as the target.

Table 17. Ranking of methods for linguistic features

Table 18. Ranking of methods for GloVe features

Table 19. Ranking of methods for USE features

Table 20. Ranking of feature types

The adaptation methods were implemented in such a way that one batch of the source data and one batch of the target data were processed simultaneously. After each pair of batches, the model’s loss was updated. Therefore, the batch sizes were adjusted in order to make the number of batches for both collections equal. The batch sizes were calculated according to the formula: $\left\lceil {0.9\,dataset\_size/num\_of\_batches} \right\rceil$ for source and $\left\lceil {dataset\_size/num\_of\_batches} \right\rceil$ for target. The factor 0.9 originates from using a 10-fold cross validation procedure. In our experiments, we set the parameter $num\_of\_batches$ for 30. In the case of domain adaptation methods that were applied to the neural network based on USE embeddings, the number of epochs was set for 30. For bi-LSTM utilizing GloVe embeddings, we used 10 epochs due to the long training time.

The results of the deep domain adaptation methods are presented in Tables 15 and 16. Figure 9 illustrates the improvement achieved by applying different deep domain adaptation methods for each source–target pair of data sets. As a reference, we used the results that were obtained by the same classifiers, but without implementing any deep domain adaptation techniques.

On the whole, the domain adaptation methods did not provide any consistent improvement in classification accuracy for all pairs of data sets. In some cases, the results turned out to be lower than before domain adaptation. Such instances are marked in blue.

In the case of the model built on bidirectional LSTM architecture that takes GloVe embeddings as the input, the domain adaptation techniques provided some improvement for less than half of the source–target pairs. The overall gain for all the tested methods (ADR, CORAL, VADA) was close to zero.

The maps visualizing the gain in accuracy vary depending on the adaptation method and the model that was used; still, some common features can be observed.

First, all the investigated deep domain-adaptation methods improved classification accuracy for the two smallest data sets—AMT and FakeNewsNet—when Kaggle was used as a source. At the same time, the highest decrease in accuracy is observed for the ISOT-Kaggle pair, the two largest data sets. Secondly, when LIAR was used as a source, hardly any change was observed.

In the case of a model utilizing USE embeddings, the results were slightly better. They could be improved by selecting the number of epochs depending on the source–target pair and methods that are used.

Both VADA and adversarial dropout regularization contributed to improved accuracy when large data sets such as Kaggle and ISOT were used as a source.

To conclude, domain adaptation methods achieved higher improvement when applied to models utilizing USE embeddings.