Loading...
Loading...
Contributors:
Dataset URI:
b6p3jgdx83HistoriKorpus is the first dataset of OCR output and its corresponding ground truth for historical Indonesian newspapers published during the post-independence era (1946–1965). The dataset was constructed from 101 pages of newspapers sourced from the National Library of Indonesia (Perpusnas) via the Khastara digital repository. The corpus consists of two primary components. First, 1,508 articles that were manually transcribed by Indonesian university students, including history majors, to ensure accurate interpretation of archaic orthographic conventions. Second, a parallel corpus of 5,383 excerpts, each pairing a raw OCR output with its verified manual transcription (ground truth), totaling approximately 285,366 tokens and 1,944,940 characters. The dataset captures two historical Indonesian orthographic systems — the Van Ophuijsen spelling (pre-1947) and the Soewandi/Ejaan Republik spelling (1947–1972) — which differ substantially from contemporary Indonesian orthography. OCR was performed using a dedicated pipeline consisting of DocLayout-YOLO for text block detection and Tesseract 4 for text extraction. Baseline evaluation yielded a Character Error Rate (CER) of 8.25% and a Word Error Rate (WER) of 34.22%. The dataset directly serves as a benchmark for developing and evaluating OCR, post-OCR correction, layout segmentation, and analyzing historical orthographic and lexical shifts. Furthermore, the corpus provides a foundation for downstream NLP tasks such as named entity recognition, text summarization, and historical fact-checking. For practical usage, researchers are recommended to perform data splitting (train/validation/test) at the excerpt level for low-level visual tasks like post-OCR correction, and at the article or document level for contextual downstream applications to prevent data leakage. This dataset operates under a dual-licensing scheme: - Transcriptions & OCR Data: Licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). - Source Newspaper PDFs and Images in zip format: Marked with Public Domain Mark 1.0 (PDM 1.0)
Keyword(s):
Data Science; History.
Grants and Funding:
Associated Publication(s):
No Associated Publication Data Available
Supplementary Data:
No Supplementary Data Available
Related Dataset(s):
No Related Dataset Data Available
The National Library of Medicine (NLM) provides information about biomedical datasets. Inclusion on an NLM Web page does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: NLM Web Policies (nih.gov)