method: Mybank-DocRetrieval2021-11-18

Authors: Members

Affiliation: Mybank

Description: We use Bert-large to train a classification task and use the logits as the ranking score to rerank the documents. Then they are filtered by the extracted information from questions. We use a QA model trained on Task1 to extract the answer.

method: Infrrd-RADAR2021-04-01

Authors: JiangLong He, Aditya Kumar Sarda, Deepak Kumar, Cesar Duran

Affiliation: Infrrd.ai

Description: The Infrrd-RADAR (Retrieval of Answers by Document Analysis and Re-ranking) performs OCR on the set of images present in the dataset. The OCR data is utilized with the image to extract the information such as pdc filed date, candidate name, office, party and other key information from each forms. The extracted information is stored in a csv format file. Totally 28 fields are extracted from the forms. The natural language questions are parsed using spaCy. The chunks are categorized into subject, object, and dependency object. The entities are categorized into person, geo-political entity, organization. Using the categorized information, each question is converted into a set of SQL queries. The SQL queries are used with fuzzy-search algorithm to retrieve set of relevant documents. BERT-Large based model is then used to rerank the set of relevant documents. The reranked document ids are used to filter the extracted information. Based on the parsed questions, a particular field is collected and posted as an answer.

method: (Baseline) Database2021-04-12

Authors: DocVQA Organizers: R. Tito, D. Karatzas, E. Valveny

Affiliation: CVC-UAB

Description: Two-step method that uses extracted key-value pairs from Amazon Textract to build a database-like data structure to rank and answer the questions that are manually parsed to SQL language.

Please check cited paper for detailed information.

Ranking Table

Description Paper Source Code
DateMethodANLSLRetrieval MAP
2021-11-18Mybank-DocRetrieval0.793080.90%
2021-04-01Infrrd-RADAR0.774374.66%
2021-04-12(Baseline) Database0.706871.06%
2021-04-12(Baseline) Text spotting - BERT0.451372.84%
2020-05-16PingAn-OneConnect-Gammalab-DQA0.000080.90%
2020-05-05iFLYTEK-DOCR0.000079.15%

Ranking Graphic

Ranking Graphic