SIGIR 2009 WorkshopJuly 23, 2009

Redundancy, Diversity, and
Interdependent Document Relevance (IDR '09)


Call for Papers

This workshop will explore how ranking, performance assessment, and learning to rank can move beyond the assumption that the relevance of documents in a ranking is independent.

The focus will be on three key themes:

  • The effect of redundancy on information retrieval utility. Example topics of interest include:
    • Techniques to measure redundancy and avoid it in evaluation.
    • Redundancy as evidence and a means for increasing trust.
    • User interface approaches to redundancy and diversity (e.g. site collapse and deep links) and methods for measuring their utility.
    • Inferring redundancy and set-level judgments from implicit feedback.
  • The role of diversity. Example topics of interest include:
    • Methods for result diversification.
    • Assessing ambiguity in real user information needs.
    • Diversity as an approach for catering to a distribution of users.
    • Characterizing the trade-offs between different levels of result diversification.
    • Merging ranking outputs from rankers trained to optimize different criteria (e.g. relevance vs. diversity).
  • Approaches to set-level evaluation and optimization. Example topics of interest include:
    • Evaluating IR systems using novel types of set-level judgments.
    • Efficient means for collecting set-level and other non-independent types of judgments that provide reliable evaluation.
    • Characterizing the trade-offs in collecting and using novel types of judgments (e.g. assessor time per judgment vs. agreement vs. growth rate vs. ability to capture document interdependence).
    • Information retrieval or collaborative filtering applications that use set-level responses rather than ratings on individual items.