method: GRAM2024-01-16
Authors: Tsachi Blau, Sharon Fogel, Roi Ronen, Alona Golts, Roy Ganz, Shahar Tsiper, Elad Ben Avraham, Aviad Aberdam, Ron Litman
Affiliation: AWS AI Labs and Technion Israel
Description: GRAM model based on Docformerv2 trained on DUDE and Multi-Page DocVQA dataset.
method: GRAM C-Former2024-01-16
Authors: Tsachi Blau, Sharon Fogel, Roi Ronen, Alona Golts, Roy Ganz, Shahar Tsiper, Elad Ben Avraham, Aviad Aberdam, Ron Litman
Affiliation: AWS AI Labs and Technion Israel
Description: GRAM model with C-Former based on Docformerv2 trained on DUDE and Multi-Page DocVQA dataset.
method: Hi-VT5-beamsearch2023-04-18
Authors: JiangLong He, Mamatha N, Shiv Vignesh, Deepak Kumar
Description: Hi-VT5 model pretrained with private custom document collection using span masking objective. Pretrained model is then trained with DUDE dataset and Multi-Page DocVQA dataset.
| Answer | Calibration | OOD Detection | ANLS per Answer type | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Date | Method | ANLS | ECE | AURC | AUROC | Extractive | Abstractive | List of answers | Unanswerable | |||
| 2024-01-16 | GRAM | 0.5336 | 0.4404 | 0.4404 | 0.5000 | 0.5683 | 0.5232 | 0.1996 | 0.6543 | |||
| 2024-01-16 | GRAM C-Former | 0.5097 | 0.4613 | 0.4613 | 0.5000 | 0.5515 | 0.5046 | 0.1726 | 0.6104 | |||
| 2023-04-18 | Hi-VT5-beamsearch | 0.3574 | 0.6104 | 0.6104 | 0.5000 | 0.2831 | 0.3298 | 0.1060 | 0.6290 | |||
| 2023-04-21 | Hi-VT5-beamsearch with token type embeddings | 0.3559 | 0.2803 | 0.4603 | 0.4876 | 0.3095 | 0.3515 | 0.1176 | 0.5250 | |||