News - Document Information Localization and Extraction

02/28/2023

DUDE - Updated submission schedule

Dear participants, 

Following inquiries regarding the submission schedule, we have opted to better align the submissions for evaluation phase 1 (test set for Task 1) and for evaluation phase 2 (domain shift and selective question-answering).

We hope that this renewed schedule gives participants more clarity and time to work on their submissions (3 weeks, 1 week for method description) for either evaluation phase. We would urge participants to submit method descriptions (just a couple of paragraphs to understand the overall idea) together with their submissions as the competition report deadline is strict. We would not want to disqualify any team for not providing their method description in time. wink 

When giving participants access to questions from the test set, we will provide a mixture of the questions from evaluation phase 1 and phase 2.
Submissions for phase 1 would need to only provide the answers key and preferably answers_confidence for calibration evaluation.
Submissions for phase 2 would need to as well provide both the keys answers_confidence and answers_abstain for all question-answer pairs. 

Cheers, 

The DUDEs

Challenge News

Important Dates

Test set published and submissions open: 24 Apr 2023

Register by: 28 Apr 2023 (at http://clef2023-labs-registration.dei.unipd.it/)

Benchmark submission deadline: 24 May 2023

Working note submission deadline: 5 June 2023

Notification of working note acceptance: 23 June 2023

 

Note: DocILE 2023 runs simultaneously as a CLEF 2023 Lab and follows CLEF 2023 schedule: https://clef2023.clef-initiative.eu/index.php?page=Pages/schedule.html

Working note papers: On top of providing method description for the submission on RRC website, working note must be submitted to organizers as explained in the competition rules: https://docile.rossum.ai/static/docile_rules_and_prize_eligibility.pd