method: ssbaseline2020-09-09

Authors: Qi Zhu, Chenyu Gao, Peng Wang, Qi Wu

Affiliation: Northwestern Polytechnical University

Email: zephyrzhuqi@gmail.com

Description: We wish this work to set the new baseline for these two OCR text related applications and to inspire new thinking of multi-modality encoder design.

https://arxiv.org/abs/2012.05153

Source code

method: SMA2020-03-06

Authors: Anonymous

Affiliation: Anonymous

Description: Structured Multimodal Attentions

method: VTA2019-04-30

Authors: Fengren Wang, iFLYTEK, frwang@iflytek.com; Jinshui Hu, iFLYTEK, jshu@iflytek.com; Jun Du, USTC, jundu@ustc.edu.cn; Lirong Dai, USTC, lrdai@ustc.edu.cn; Jiajia Wu, iFLYTEK, jjwu@iflytek.com

Description: An ED model for ST-VQA
1. We use OCR and object detection models to extract text and objects from images.
2. Then We use Bert to encode the extracted text and QA pairs.
3. Finally We use a similar model of Bottom-Up and Top-Down[1] to handle the image and question input and give the answer output.

[1] P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR, 2018. 1, 2, 7

Ranking Table

Description Paper Source Code
DateMethodScore
2020-09-09ssbaseline0.5490
2020-03-06SMA0.5081
2019-04-30VTA0.5063
2021-08-15ss1.00.5045
2020-05-22RUArt0.4817
2019-04-29USTB-TQA0.4553
2019-04-29Focus: A bottom-up approach for Scene Text VQA0.2959
2019-04-30Visual Question Answering via deep multimodal learning0.1411
2019-04-29USTB-TVQA0.1243
2019-04-29TRAN MINH TRIEU0.0545

Ranking Graphic