The CLEF-HIPE-2026 workshop will take place during the CLEF 2026 conference in Jena, Germany.

On This Page

Key Dates

For the complete shared-task schedule, see the Timeline & Author Instructions page.

Milestone Date
Short system description due 28 May 2026
Submission of participant CEUR working notes papers 28 May 2026
End of review process 25 June 2026
Notification of acceptance 30 June 2026
Camera-ready submission 6 July 2026
Working Notes preview (author checking) 21–25 July 2026

All deadlines are Anywhere on Earth (AoE).

Working Notes

Participant working notes will be published in the CEUR-WS Working Notes of CLEF 2026. All submissions, reviews, and camera-ready versions are handled through EasyChair: https://easychair.org/conferences/?conf=clef2026

Short System Description

After result publication, participants who have chosen to publish their results and proceed with a working notes paper are kindly asked to send us a short system description by Thursday 28 May 2026.

The description should be a single paragraph of about 12–15 lines formatted using the CEUR-WS one-column template (see Formatting section below), and submitted by email to the organizers at the address used for your run submission.

We will use these descriptions when writing the HIPE overview paper to summarise participating approaches.

Paper Content

Working notes are technical reports written in English describing your system and experiments. Papers should include at minimum:

  • Title, authors, affiliations, and email addresses of all authors
  • Body of text covering:
    • Tasks performed
    • Main objectives of the experiments
    • Approach(es) used and progress beyond baselines or state of the art
    • Resources employed
    • Results obtained — please report on test sets, and on dev sets where appropriate. Note that the organizers’ overview paper will focus on the best run per team; your paper may report on individual runs in more detail.
    • Analysis and discussion of results
    • Perspectives for future work

In the interest of reproducibility, we encourage you — whenever possible — to publish:

  • your code on a public repository with an associated licence
  • any additional resources you may have used

Formatting

Papers must be formatted using the CEUR-WS one-column template available here.

Maximum length: No strict upper limit, as papers are published electronically only. We recommend 6–10 pages excluding references. Concise, focused papers are more effective.

Submission and Review

Submit your paper electronically via EasyChair: https://easychair.org/conferences/?conf=clef2026

After logging in, select the HIPE track and follow the instructions.

Important: The metadata entered in EasyChair will be used for the final working notes. Please ensure that the following information is correct and matches the PDF of your paper:

  • Complete and correctly spelled author names
  • Authors listed in the correct order
  • Correct paper title

Review Process

Reviews will be conducted by the organizing team, with the objective of ensuring paper quality. Unless there are significant shortcomings, all submitted papers are expected to be accepted and published in the working notes.

Publication and Cross-References

Post-Publication Changes

Once the CEUR-WS Working Notes are published, they cannot be modified — including PDFs, titles, or author names. A short pre-publication checking phase (21–25 July 2026) will allow authors and organizers to catch any errors in the table of contents before final publication. Please make sure all submitted materials are complete and correct by the camera-ready deadline.

Cross-references between the HIPE Overview Paper and Participant Working Notes Papers

Shared task organisers will publish two overview papers: 1) a condensed overview in the CLEF 2026 LNCS proceedings, and 2) an extended overview in the CEUR-WS Working Notes proceedings.

Participant teams will publish working notes papers in the CEUR-WS Working Notes proceedings.

Shared task organisers will cite participant papers in both overviews, and participants are encouraged to cite the HIPE extended overview in their working notes papers. The overview papers will present the task, include a brief summary of each participant’s approach, present the results and the main findings.

References

[1] Juri Opitz, Maud Ehrmann, Corina Raclé, Andrianos Michail, Matteo Romanello, Emanuela Boros, Simon Gabay, and Simon Clematide. 2026. Extended Overview of HIPE-2026: Evaluating Accurate and Efficient Person–Place Relation Extraction from Multilingual Historical Texts. In CLEF 2026 working notes, CEUR workshop proceedings, 2026. CEUR-WS. https://doi.org/10.5281/zenodo.20344461

@inproceedings{opitz_extended_2026,
  title = {Extended {{Overview}} of {{HIPE-2026}}: {{Evaluating Accurate}} and {{Efficient Person}}--{{Place Relation Extraction}} from {{Multilingual Historical Texts}}},
  booktitle = {{{CLEF}} 2026 Working Notes, {{CEUR}} Workshop Proceedings},
  author = {Opitz, Juri and Ehrmann, Maud and Racl{\'e}, Corina and Michail, Andrianos and Romanello, Matteo and Boros, Emanuela and Gabay, Simon and Clematide, Simon},
  editor = {S{\'a}nchez Salido, Eva and {Barr{\'o}n-Cede{\~n}o}, Alberto and {Garc{\'i}a Seco de Herrera}, Alba and MacAvaney, Sean and Stru{\ss}, Julia Maria},
  year = 2026,
  publisher = {CEUR-WS},
  doi = {10.5281/zenodo.20344461}
}

[2] Juri Opitz, Maud Ehrmann, Corina Raclé, Andrianos Michail, Matteo Romanello, and Simon Clematide. 2026. Overview of HIPE-2026: Person–Place Relation Extraction from Multilingual Historical Texts. In Experimental IR meets multilinguality, multimodality, and interaction. Proceedings of the seventeenth international conference of the CLEF association (CLEF 2026) (Lecture notes in computer science (LNCS)), 2026. Springer.

@inproceedings{opitz_overview_2026,
  title = {Overview of {{HIPE-2026}}: {{Person}}--{{Place Relation Extraction}} from {{Multilingual Historical Texts}}},
  booktitle = {Experimental {{IR}} Meets Multilinguality, Multimodality, and Interaction. {{Proceedings}} of the Seventeenth International Conference of the {{CLEF}} Association ({{CLEF}} 2026)},
  author = {Opitz, Juri and Ehrmann, Maud and Racl{\'e}, Corina and Michail, Andrianos and Romanello, Matteo and Clematide, Simon},
  editor = {Hagen, Matthias and Potthast, Martin and Stein, Benno and Schaer, Philipp and Zangerle, Eva and MacAvaney, Sean and Stru{\ss}, Julia Maria and S{\'a}nchez Salido, Eva and {Barr{\'o}n-Cede{\~n}o}, Alberto and {Garc{\'i}a Seco de Herrera}, Alba},
  year = 2026,
  series = {Lecture Notes in Computer Science ({{LNCS}})},
  publisher = {Springer}
}

Preparation of Camera-Ready Working Notes: General Guidelines

Team and Paper Identification

  • Team name: Please make sure your team name is mentioned in the paper, including in the abstract, so that it can be related to the result tables in the overview papers.
  • Paper title: Please do not change your paper title, as it is already referenced in the overview papers and this would break the cross-referencing to your paper.

Article Writing

  • Declaration of use of generative AI: CEUR-WS requires a declaration of use (or non-use) of generative AI tools in the preparation of your paper. Please refer to the CEUR-WS policy and add the appropriate statement: https://ceur-ws.org/GenAI/Policy.html
  • Bibliography: Please make sure your bibliography is complete and correct, and that all references are cited in the text. Pay attention to correct casing of titles.

Cross-References

  • References to the overview papers: Referencing both overview papers is mandatory, as required by the CLEF chairs. Doing so also supports the visibility of CLEF, shared tasks and their long-term sustainability. Please note that the BibTeX entries have been updated — please replace any existing entries with the correct versions available in the References section above.

    The condensed overview is also available as an arXiv preprint (https://arxiv.org/abs/2606.25935) and can serve as background reading; please cite the Springer version rather than the preprint.

  • References to other participants’ papers: You may reference other participants’ papers where relevant. We are shaing BibTeX entries for all participant papers here. Summaries of all systems are also available in the condensed overview paper linked above.

Task Definition, Setup, and Results

  • Task presentation: Participant papers should be self-contained and include a presentation of the task and the evaluation profiles. This does not need to be lengthy and can point to the overview paper, but it should be present. When introducing the task, consider adding the task website URL in a footnote: https://hipe-eval.github.io/HIPE-2026/

  • System rankings: If you mention the rank of your system(s):
    • Specify the run number alongside the rank.
    • State the total number of participating teams (17) and submitted runs by participants (45), so that readers can situate your system in context.
    • Point to the results tables of the overview papers for completeness.
  • Metric and score names: Please use metric and score names carefully and consistently.
    • The official metric is macro recall (also referred to as balanced accuracy).
    • The per-relation scores are MacroRecall_at and MacroRecall_isAt.
    • The language-specific profile score is their arithmetic mean, and the overall Test A ranking score averages this across the three languages.
    • The three evaluation dimensions should be referred to as profiles (Accuracy profile, Efficiency profile, Generalization profile), not as tracks, tasks, or subtasks.
    • The efficiency score is the arithmetic mean of three ranks — accuracy rank, model parameter count rank, and deployed model size rank — and is therefore not a recall or accuracy figure: a lower efficiency score is better.
  • Dataset and data split names: To facilitate cross-paper comparison, we encourage consistent naming of datasets and splits:
    • For test data, please use one of the following:Test A or Newspaper test set, and Test B or French literary surprise test set.
    • For training data, the following distinctions apply.
      • Gold Train A refers to the manually annotated training data (available in the /newspapers/v1.0 folder of the data repository).
      • Silver Train A and Silver Dev A refer to the LLM-annotated data produced by majority voting (available in the /sandbox folder); these may also be referred to as sandbox train and sandbox dev, but we recommend using Silver Train A and Silver Dev A to align with the terminology of the overview papers.
  • Consistency of reported results: Please ensure that scores mentioned in the abstract and conclusion match those in your result tables. When reporting results, make clear in every table caption which test set the scores correspond to — Test A (newspaper test set), Test B (French literary surprise test set), or both.

Reproducibility and Code Sharing

  • Competition data: When mentioning the competition data, please indicate the version used in a footnote: https://github.com/hipe-eval/HIPE-2026-data/releases/tag/v1.0
  • Official scorer: If you used the official scorer, please cite and reference it in the same footnote: https://github.com/hipe-eval/HIPE-2026-data/releases/tag/v1.0
  • Submission files and evaluation orchestrator: If you wish to point to your system submission files and the shared task evaluation orchestrator, link to: https://github.com/hipe-eval/HIPE-2026-eval
  • System code and model weights: Consider providing a link to your system code and/or model weights, if available.
  • When presenting your models, please make clear on which data they were trained and validated.
  • Prompt descriptions: If your system uses prompting, we encourage you to include the full prompt(s) — either as an appendix or via a link to your code repository. Specifying the prompt language (especially when it differs from the document language) and any prompt variants evaluated is important for reproducibility.