WikiCheck: An End-to-End Open Source Automatic Fact-Checking API Based on Wikipedia

Abstract

With the growth of fake news and disinformation, the NLP community has been working to assist humans in fact-checking. However, most academic research has focused on model accuracy without paying attention to resource efficiency, which is crucial in real-life scenarios. In this work, we review the State-of-the-Art datasets and solutions for Automatic Fact-checking and test their applicability in production environments. We discover overfitting issues in those models, and we propose a data filtering method that improves the model’s performance and generalization. Then, we design an unsupervised fine-tuning of the Masked Language models to improve its accuracy working with Wikipedia. We also propose a novel query enhancing method to improve evidence discovery using the Wikipedia Search API. Finally, we present a new fact-checking system, the WikiCheck API that automatically performs a facts validation process based on the Wikipedia knowledge base. It is comparable to SOTA solutions in terms of accuracy and can be used on low-memory CPU instances.

Citation

@inproceedings{10.1145/3459637.3481961,
  author = {Trokhymovych, Mykola and Saez-Trumper, Diego},
  title = {WikiCheck: An End-to-End Open Source Automatic Fact-Checking API Based on Wikipedia},
  year = {2021},
  isbn = {9781450384469},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3459637.3481961},
  doi = {10.1145/3459637.3481961},
  booktitle = {Proceedings of the 30th ACM International Conference on Information \& Knowledge Management},
  pages = {4155--4164},
  numpages = {10},
  location = {Virtual Event, Queensland, Australia},
  series = {CIKM '21},
}

Other publications

• Adversarial Creation and Detection of AI-Generated Social Bot Content (CIKM’26)
• Multilingual Reference Need Assessment System for Wikipedia (WWW’26)
• An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation (UNLP’26)
• Graph-Linguistic Fusion: Using Language Models for Wikidata Vandalism Detection (ACL’25)
• Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine (UNLP’25)
• Characterizing Knowledge Manipulation in a Russian Wikipedia Fork (ICWSM’25)
• An Open Multilingual System for Scoring Readability of Wikipedia (ACL’24)
• Wikidata Vandalism Detection with Graph-Linguistic Fusion (2024)
• Fair Multilingual Vandalism Detection System for Wikipedia (KDD’23)
• GeoDD: End-to-End Spatial Data De-duplication System (2023)
• WikiFactFind: Semi-automated fact-checking based on Wikipedia (2022)

← All publications