Authors: LayoutLM Team

Affiliation: LayoutLM Team

Description: We noticed that there are some OCR mismatches in the testing set, which will never be correct even if the model output matches perfectly with corresponding images. However, excluding these errors when submitting could lead to higher precision while keeping recall unchanged. Obviously, the F1 will be higher but it would lead to *UNFAIR* results. So, we submit this result not for getting a higher F1 score but for showing the potential vulnerability. We hope the organizer of SROIE can fix this evaluation process either by checking the OCR mismatches, or removing the whitespaces within each field during the evaluation, so that the model could do better on recall and the F1 could be "truely" improved.