Overview of the TREC 2023 Tip-of-the-Tongue track

Tip-of-the-tongue (ToT) known-item retrieval involves supporting searchers interested in refinding a previously encountered item for which they are unable to reliably recall an identifier. ToT requests tend to be verbose and include several complex phenomena, making them especially difficult for ex- isting information retrieval systems. The TREC 2023 ToT track focused on a single ad-hoc retrieval task in the movie domain. Requests were sampled from an existing ToT dataset and the document corpus consisted of a subset of Wikipedia pages associated with the “audiovisual works” category. This year 11 groups submitted a total of 33 runs. Consistent with earlier findings, there is a negative correlation between query length and retrieval performance. We found that successful teams were able to leverage large external datasets to substantially improve performance. While a closed large language model managed to beat 26 participant runs, it did so with much lower recall.