An Open Multilingual System for Scoring Readability of Wikipedia

Abstract

With over 60M articles, Wikipedia has become the largest platform for open and freely accessible knowledge. While it has more than 15B monthly visits, its content is believed to be inaccessible to many readers due to the lack of readability of its text. However, previous investigations of the readability of Wikipedia have been restricted to English only, and there are currently no systems supporting the automatic readability assessment of the 300+ languages in Wikipedia. To bridge this gap, we develop a multilingual model to score the readability of Wikipedia articles. To train and evaluate this model, we create a novel multilingual dataset spanning 14 languages, by matching articles from Wikipedia to simplified Wikipedia and online children encyclopedias. We show that our model performs well in a zero-shot scenario, yielding a ranking accuracy of more than 80% across 14 languages and improving upon previous benchmarks. These results demonstrate the applicability of the model at scale for languages in which there is no ground-truth data available for model fine-tuning. Furthermore, we provide the first overview on the state of readability in Wikipedia beyond English.

Citation

@inproceedings{trokhymovych-etal-2024-open,
  title = {An Open Multilingual System for Scoring Readability of {W}ikipedia},
  author = {Trokhymovych, Mykola and Sen, Indira and Gerlach, Martin},
  editor = {Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  month = aug,
  year = {2024},
  address = {Bangkok, Thailand},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2024.acl-long.342},
  doi = {10.18653/v1/2024.acl-long.342},
  pages = {6296--6311},
}

Other publications

• Adversarial Creation and Detection of AI-Generated Social Bot Content (CIKM’26)
• Multilingual Reference Need Assessment System for Wikipedia (WWW’26)
• An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation (UNLP’26)
• Graph-Linguistic Fusion: Using Language Models for Wikidata Vandalism Detection (ACL’25)
• Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine (UNLP’25)
• Characterizing Knowledge Manipulation in a Russian Wikipedia Fork (ICWSM’25)
• Wikidata Vandalism Detection with Graph-Linguistic Fusion (2024)
• Fair Multilingual Vandalism Detection System for Wikipedia (KDD’23)
• GeoDD: End-to-End Spatial Data De-duplication System (2023)
• WikiFactFind: Semi-automated fact-checking based on Wikipedia (2022)
• WikiCheck: An End-to-End Open Source Automatic Fact-Checking API Based on Wikipedia (CIKM’21)

← All publications