Authors: Youngmin Baek, Michael Hentschel, Yu Nakagome, Shuta Ichimura, Jeong Tae Lee, Chankyu Choi

Affiliation: NAVER/LINE WORKS

Description: We used CREPE method, an end-to-end KIE model, to perform text detection, recognition, and linking tasks without any post-processing. While the original paper regresses four quadrilateral coordinates, we modified the approach to predict eight control points based on Bezier curves. For pretraining, we use ArT dataset to learn effective representations of curved text, and during fine-tuning, we train exclusively on the Rumsey dataset.

Authors: Mengjie Zou, Tianhao Dai, Remi Petitpierre, Beatrice Vaienti, Frederic Kaplan, Isabella di Lenardo

Affiliation: EPFL, Swiss Federal Institute of Technology in Lausanne

Email: remi.petitpierre@epfl.ch

Description: For word detection and recognition, our approach relies on DeepSolo, whose architecture is derived from Detection Transformers (DETR). In short, DeepSolo extracts hierarchical visual features from map images and processes them through an encoder-decoder architecture to detect words as segments bounded by Bézier curves. The model specifically returns four control points of central Bézier curves per word and then uniformly samples query points along these curves to segment, classify, and delineate each text instance precisely. To resolve duplicate word detections, we implement a postprocessing step inspired by Non-Maximum Suppression. It involves calculating the Fréchet distance between the Bézier curves of potential duplicate word pairs, or "directional synonyms", and merging those below a defined threshold.

Our text linking methodology consists of four steps: (1) neighbor sampling, (2) self-sequencing, (3) graph assignment, and (4) ordering. In the first step, a word segment is designated as the query, and neighbor segments are used as candidates. For the second step, we introduce Self-Sequencer, a trainable Transformer-based model that iteratively returns ordered local sequences based on the input query segment, and candidate neighbors. Each input text segment is represented only by the control points of its bounding Bézier curves. A Transformer Encoder generates a deep representation from spatial-directional features. Then, an Attentive Pointer module predicts local word sequences based on the concatenated hidden states of candidate word pairs. The aim of the third step, which we call graph assignement, is to aggregate the local link predictions. In this perspective, the links predicted by the Self-Sequencer are used to create a global directional graph. Each strongly connected component of the global graph is considered a distinct linked word set. In the fourth and last step, the order of the words in the sequence is retrieved by applying a consensus ranking algorithm, based on the local sequence order predicted by the Self-Sequencer. More details on the model, algorithms, and specific implementation are provided in our separate article [1].

The training leverages several real and synthetical datasets: ICDAR MapText [2], MapKuratorHuman [3], SynthMap [3], and Paris and Jerusalem Maps Text Dataset [4].

References:

[1] Zou, M., Dai, T., Petitpierre, R., Vaienti, B., Kaplan, F., & di Lenardo I. (2025). Recognizing and Sequencing Multi-word Texts in Maps Using an Attentive Pointer.

[2] Lin, Y., Li, Z., Chiang Y.Y., & Weinman J. (2024). Rumsey Train and Validation Data for ICDAR'24 MapText Competition (Version 1.3). Zenodo. https://doi.org/10.5281/zenodo.11516933

[3] Kim, J., Li, Z., Lin Y., Namgung, M., Jang, L., & Chiang Y.Y. (2023) The mapKurator System: A Complete Pipeline for Extracting and Linking Text from Historical Maps. In: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems. https://arxiv.org/abs/2306.17059

[4] Dai, T., Johnson, K., Petitpierre, R., Vaienti, B., & di Lenardo, I. (2025). Paris and Jerusalem Maps Text Dataset (Version 1.0.0). Zenodo. https://doi.org/10.5281/zenodo.14982662