Authors: Applica.ai Research Team

Affiliation: Applica.ai

Email: rafal.powalski@applica.ai, dawid.jurkiewicz@applica.ai

Description: TILT neural network architecture which simultaneously learns layout information, visual features, and textual semantics. Contrary to previous approaches, we rely on a encoder-decoder architecture. Results were obtained from single TILT-Large model pre-trained as described in a paper. Model was finetuned on challenge train set.

Authors: Ryota Tanaka, Kyosuke Nishida

Affiliation: NTT Media Intelligence Laboratories, NTT Corporation

Email: ryouta.tanaka.rg@hco.ntt.co.jp

Description: IG-BERT is a V+L model pre-trained on large-scale infographic-text pairs. The model was initialized from BERT-large and trained on training and validation data. We extracted icon visual features using faster-rcnn trained on Visually29K. In the preprocessing stage, we used the google vision API to extract OCR.

Authors: Jungjun Kim, Teakgyu Hong, Hyungmin Lee, Junbum Cha, Sungrae Park

Affiliation: NAVER Corp.

Email: teakgyu.hong@navercorp.com

Description: We used CLOVA OCR to obtain OCR results for images, and then preprocessed the OCR results to solve the problem by extractive QA method. For preprocessing, we followed HyperDQA's approach. To train the extractive QA model, we first pre-trained BROS[1] model (with slight modification - sharing parameters between projection matrices in self-attention) on the IIT-CDIP dataset. Then, additional pre-training was performed on the SQuAD and WikitableQa datasets. After that, answers were obtained through fine-tuning on the DocVQA dataset.

[1] https://openreview.net/pdf?id=punMXQEsPr0