Publications

Preprint

RankList -- A Listwise Preference Learning Framework for Predicting Subjective Preferences
A. R. Naini, F. Diaz, C. Busso
arxiv, 2025
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
A. Olteanu, S.-L. Blodgett, A. Balayn, A. Wang, F. Diaz, F. du Pin Calmon, M. Mitchell, M. D. Ekstrand, R. Binns, S. Barocas
arxiv, 2025
Retrieval-Enhanced Machine Learning: Synthesis and Opportunities
T.-E. Kim, A. Salemi, A. Drozdov, F. Diaz, H. Zamani
arxiv, 2024

Conference

MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers
J. Kalra, X. Zhao, T.-E. Kim, F. Cai, F. Diaz, T. Wu
EMNLP, 2025
Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
T.-E. Kim and F. Diaz
ICTIR, 2025
Tip of the Tongue Query Elicitation for Simulated Evaluation
Y. He, T.-E. Kim, F. Diaz, J. Arguello, B. Mitra
SIGIR 2025
Contextual Metric Meta-Evaluation by Measuring Local Metric Accuracy
A. Deviyani, F. Diaz
NAACL 2025
Offline Evaluation of Set-Based Text-to-Image Generation
N. Arabzadeh, F. Diaz, J. He
SIGIR-AP 2024
Pessimistic Evaluation
F. Diaz
SIGIR-AP 2024
Density-based User Representation using Gaussian Process Regression for Multi-interest Personalized Retrieval
H. Wu, O. Meshi, M. Zoghi, F. Diaz, X. Liu, C. Boutilier, M. Karimzadehgan
NeurIPS 2024
Overview of the TREC 2024 Tip-of-the-Tongue track
J. Arguello, S. Bhargav, F. Diaz, T.-E. Kim, Y. He, E. Kanoulas, B. Mitra
TREC 2024
Extrinsic Evaluation of Cultural Competence in Large Language Models
S. Bhatt and F. Diaz
EMNLP Findings 2024
Scaling Laws Do Not Scale
F. Diaz and M. Madaio
AIES, 2024
The Impact of Group Membership Bias on the Quality and Fairness of Exposure in Ranking
A. Vardasbi, M. de Rijke, F. Diaz, M. Dehghani
SIGIR 2024
Fairness Through Domain Awareness: Mitigating Popularity Bias For Music Discovery
R. Salganik, F. Diaz, G. Farnadi
ECIR 2024
Overview of the TREC 2023 Tip-of-the-Tongue track
J. Arguello, S. Bhargav, F. Diaz, E. Kanoulas, B. Mitra
TREC 2023
AI Consent Futures: A Case Study on Voice Data Collection with Clinicians
L. Wilcox, R. Brewer, F. Diaz
CSCW 2023
Measuring Commonality in Recommendation of Cultural Content: Recommender Systems to Enhance Cultural Citizenship
A. Ferraro, G. Ferreira, F. Diaz, G. Born
RecSys 2022 (Late Breaking Results)
On Natural Language User Profiles for Transparent and Scrutable Recommendation
F. Radlinski, K. Balog, F. Diaz, L. Dixon, B. Wedin
SIGIR 2022 (Perspectives)
Joint Multisided Exposure Fairness for Recommendation
H. Wu, B. Mitra, C. Ma, F. Diaz, X. Liu
SIGIR 2022
Retrieval-Enhanced Machine Learning
H. Zamani, F. Diaz, M. Dehghani, D. Metzler, M. Bendersky
SIGIR 2022 (Perspectives)
Offline Retrieval Evaluation Without Evaluation Metrics
F. Diaz, A. Ferraro
SIGIR 2022
Learning to Limit Data Collection via Scaling Laws: A Computational Interpretation for the Legal Principle of Data Minimization
D. Shanmugam, F. Diaz, S. Shabanian, M. Finck, A. J. Biega
FAccT 2022
Exposing Query Identification for Search Transparency
R. Li, J. Li, B. Mitra, F. Diaz, A. J. Biega
WWW 2022
Artsheets for Art Datasets
R. Srinivasan, E. Denton, J. Famularo, N. Rostamzadeh, F. Diaz, B. Coleman
NeurIPS (Datasets and Benchmarks track), 2021
"I Can’t Reply with That": Characterizing Problematic Email Reply Suggestions
R. Robertson, A. Olteanu, F. Diaz, M. Shokouhi, P. Bailey
CHI 2021
Estimation of Fair Ranking Metrics with Incomplete Judgments
O. Kirnap, F. Diaz, A. J. Biega, M. Ekstrand, B. Carterette, E. Yilmaz
WWW 2021
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
J. Arguello, A. Ferguson, E. Fine, B. Mitra, H. Zamani, F. Diaz
CHIIR 2021
Overview of the TREC 2020 Fair Ranking track
A. Biega, F. Diaz, M. D. Ekstrand, S. Kohlmeier
TREC 2020
Evaluating Stochastic Rankings with Expected Exposure
F. Diaz, B. Mitra, M. D. Ekstrand, A. J. Biega, B. Carterette
CIKM 2020
When Are Search Completion Suggestions Problematic?
A. Olteanu, F. Diaz, G. Kazai
CSCW 2020
Operationalizing the Legal Principle of Data Minimization for Personalization
A. J. Biega, P. Potash, H. Daumé III, F. Diaz, M. Finck
SIGIR 2020
Analyzing and Learning from User Interactions for Search Clarification
H. Zamani, B. Mitra, E. Chen, G. Lueck, F. Diaz, P. N. Bennett, N. Craswell, S. T. Dumais
SIGIR 2020
Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems
R. Mehrotra, J. McInerney, H. Bouchard, M. Lalmas, F. Diaz
CIKM 2018
Understanding and Evaluating User Satisfaction with Music Discovery
J. Garcia-Gathright, B. St. Thomas, C. Hosey, Z. Nazari, F. Diaz
SIGIR 2018
Using Query Performance Predictors to Reduce Spoken Queries
J. Arguello, S. Avula, F. Diaz
ECIR 2017
Learning to Match Using Local and Distributed Representations of Text for Web Search
B. Mitra, F. Diaz, N. Craswell
WWW 2017
Auditing Search Engines for Differential Performance Across Demographics
R. Mehrotra, A. Anderson, F. Diaz, A. Sharma, H. Wallach, E. Yilmaz
WWW 2017
Overview of the TREC 2016 Real-Time Summarization track
J. Lin, A. Roegiest, L. Tan, R. McCreadie, E. Voorhees, F. Diaz
TREC 2016
A Study of Realtime Summarization Metrics
M. Ekstrand, R. McCreadie, V. Pavlu, F. Diaz
CIKM 2016
The Social Dynamics of Language Change in Online Networks
R. Goel, S. Soni, N. Goyal, J. Paparrizos, H. Wallach, F. Diaz, J. Eisenstein
SocInfo 2016
Learning to Rank with Labeled Features
F. Diaz
ICTIR 2016
Query Expansion with Locally-Trained Word Embeddings
F. Diaz, B. Mitra, N. Craswell
ACL 2016
Search Result Prefetching Using Cursor Movement
F. Diaz, Q. Guo, R. White
SIGIR 2016
Real-Time Web Scale Event Summarization Using Sequential Decision Making
C. Kedzie, F. Diaz, K. McKeown
IJCAI 2016
Pseudo-Query Reformulation
F. Diaz
ECIR 2016
Using Query Performance Predictors to Improve Spoken Queries
J. Arguello, S. Avula, F. Diaz
ECIR 2016
Overview of the TREC 2015 Temporal Summarization track
J Aslam, F Diaz, M Ekstrand-Abueg, R McCreadie, V Pavlu, T Sakai
TREC 2015
Condensed List Relevance Models
F. Diaz
ICTIR 2015
Predicting Salient Updates for Disaster Summarization
C. Kedzie, K. McKeown, and F. Diaz
ACL 2015
Overview of the TREC 2014 Temporal Summarization track
J Aslam, F Diaz, M Ekstrand-Abueg, R McCreadie, V Pavlu, T Sakai
TREC 2014
Overview of the TREC 2014 Web track
K Collins-Thompson, C Macdonald, P Bennett, F Diaz, E Voorhees
TREC 2014
Mobile Query Reformulations
M. Shokouhi, R. Jones, U. Ozertem, K. Raghunathan, F. Diaz
SIGIR 2014
Whole Page Optimization: How Page Elements Interact with the Position Auction
P. Metrikov, F. Diaz, S. Lahaie, J. Rao
EC 2014.
CrisisLex: A Lexicon for Collecting and Filtering Microblogged Communications in Crises
A. Olteanu, C. Castillo, F. Diaz, S. Vieweg
ICWSM 2014
Contextual and Dimensional Relevance Judgments for Reusable SERP-level Evaluation
P. Golbus, I. Zitouni, J. Kim, A. Hassan, F. Diaz
WWW 2014
Overview of the TREC 2013 Temporal Summarization track
J Aslam, F Diaz, M Ekstrand-Abueg, V Pavlu, T Sakai
TREC 2013
Overview of the TREC 2013 Web track
K Collins-Thompson, P Bennett, F Diaz, C Clarke, E Voorhees
TREC 2013
Robust Models of Mouse Movement on Dynamic Web Search Results Pages
F. Diaz, R. White, D. Liebling, G. Buscher
CIKM 2013
Extracting information nuggets from disaster-related messages in social media
M. Imran, S. Elbassuoni, C. Castillo, F. Diaz, P. Meier
ISCRAM 2013.
Updating users about time critical events
Q. Guo, F. Diaz, E. Yom-Tov
ECIR 2013
Learning to Aggregate Vertical Results into Web Search Results
J. Arguello, F. Diaz, J. Callan
CIKM 2011
Out of Sight, Not Out of Mind: On the Effect of Social and Physical Detachment on Information Need
E. Yom-Tov and F. Diaz
SIGIR 2011
Generalized Link Suggestions via Web Site Clustering
J. Seo, F. Diaz, E. Gabrilovich, V. Josifovski, B. Pang
WWW 2011
A Methodology for Evaluating Aggregated Search Results
J. Arguello, F. Diaz, J. Callan, B. Carterette
ECIR 2011
Cross-Market Model Adaptation with Pairwise Preference Data for Web Search Ranking
J. Bai, F. Diaz, Y. Chang, Z. Zheng
COLING 2010
Vertical Selection in the Presence of Unlabeled Verticals
J. Arguello, F. Diaz, J-F. Paiement
SIGIR 2010
Relevance and Ranking in Online Dating Systems
F. Diaz, D. Metzler, S. Amer-Yahia
SIGIR 2010
Time is of the Essence: Improving Recency Ranking Using Twitter Data
A. Dong, R. Zhang, P. Kolari, J. Bai, F. Diaz, Y. Chang, Z. Zheng, H. Zha
WWW 2010
Towards recency ranking in web search
A. Dong, Y. Chang, Z. Zheng, G. Mishne, J. Bai, R. Zhang, K. Buchner, C. Liao, F. Diaz
WSDM 2010
Classification-based Resource Selection
J. Arguello, J. Callan, F. Diaz
CIKM 2009
Sources of Evidence for Vertical Selection
J. Arguello, F. Diaz, J. Callan, J-F. Crespo
SIGIR 2009
Adaptation of Offline Vertical Selection Predictions in the Presence of User Feedback
F. Diaz and J. Arguello
SIGIR 2009
Integration of News Content Into Web Results
F. Diaz
WSDM 2009
A Method for Transferring Retrieval Scores Between Collections with Non-Overlapping Vocabularies
F. Diaz
SIGIR 2008
Improving Relevance Feedback in Language Modeling Retrieval with Score Regularization
F. Diaz
SIGIR 2008
Theoretical Bounds On and Empirical Robustness of Score Regularization to Different Similarity Measures
F. Diaz
SIGIR 2008
Performance prediction using spatial autocorrelation
F. Diaz
SIGIR 2007
Pseudo-aligned multilingual corpora
F. Diaz and D. Metzler
IJCAI 2007
Improving the estimation of relevance models using large external corpora
F. Diaz and D. Metzler
SIGIR 2006
Regularizing ad hoc retrieval scores
F. Diaz
CIKM 2005
Using temporal profiles of queries for precision prediction
F. Diaz and R. Jones
SIGIR 2004
A user-centered approach to evaluating topic models
D. Kelly, F. Diaz, N. J. Belkin, J. Allan
ECIR 2004
Using wearable computers to construct semantic representations of physical spaces
F. Diaz
ISWC 2002

Journal

Recall, Robustness, and Lexicographic Evaluation
F. Diaz, M. D. Ekstrand, and B. Mitra
ACM Transactions on Recommender Systems, February 2025
A Public Service Media Perspective on the Algorithmic Amplification of Cultural Content
G. Born and F. Diaz
Knight First Amendment Institute, July 2024
Commonality in Recommender Systems: Evaluating Recommender Systems to Enhance Cultural Citizenship
A. Ferraro, G. Ferreira, F. Diaz, G. Born
ACM Transactions on Recommender Systems, March 2024
Distributionally-Informed Recommender System Evaluation
M. D. Ekstrand, B. Carterette, F. Diaz
ACM Transactions on Recommender Systems, March 2024
A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
H. Wu, C. Ma, B. Mitra, F. Diaz, X. Liu
ACM Transactions on Information Systems, September 2022
Fairness and discrimination in information access systems
M. Ekstrand, A. Das, R. Burke, F. Diaz
Foundations and Trends in Information Retrieval, 2022
Making Sense of Metrics in the Music Industries
N. Baym, R. Bergmann, R. Bhargava, F. Diaz, T. Gillespie, D. Hesmondhalgh, E. Maris, C. Persaud
International Journal of Communication, 2021
Artificial Intelligence, Music Recommendation, and the Curation of Culture
G. Born, J. Morris, F. Diaz, A. Anderson
Schwartz Reisman Institute White Paper, June 2021
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
A. Olteanu, C. Castillo, E. Kiciman, F. Diaz
Frontiers in Big Data, July 2019
Research Frontiers in Information Retrieval: Report from the Third Strategic Workshop on Information Retrieval in Lorne (SWIRL 2018)
J. S. Culpepper, F. Diaz, M. D. Smucker
SIGIR Forum, June 2018
Search Result Prefetching on Desktop and Mobile
R. White, F. Diaz, Q. Guo
ACM Transactions on Information Systems, June 2017
Worst Practices for Designing Production Information Access Systems
F. Diaz
SIGIR Forum, June 2016
Online and social media data as an imperfect continuous panel survey
F. Diaz, M. Gamon, J. Hofman, E. Kiciman, D. Rothschild
PLoS ONE, January 2016
Processing Social Media Messages in Mass Emergency
M. Imran, C. Castillo, F. Diaz, S. Vieweg
ACM Comput. Surv. 47, 4, Article 67 (June 2015), 38 pages
The Economic and Cognitive Costs of Annoying Display Advertisements
D. G. Goldstein, S. Suri, R. P. McAfee, M. Ekstrand-Abueg, F. Diaz
Journal of Marketing Research, December 2014
Experimentation Standards for Crisis Informatics
F. Diaz
SIGIR Forum, December 2014
Emergency-Relief Coordination on Social Media: Automatically Matching Resource Requests and Offers
H. Purohit, C. Castillo, F. Diaz, A. Sheth, P. Meier
First Monday, January 2014
Improving Recency Ranking Using Twitter Data
Y. Chang, A. Dong, P. Kolari, R. Zhang, Y. Inagaki, F. Diaz, H. Zha, Y. Liu
ACM Transactions Intelligent Systems Technology, February 2013
The Effect of Social and Physical Detachment on Information Need
E. Yom-Tov and F. Diaz
ACM Transactions on Information Systems, January 2013
Regularizing Query-Based Retrieval Scores
F. Diaz
Information Retrieval, December 2007
Temporal profiles of queries
R. Jones and F. Diaz
ACM Transactions on Information Systems, July 2007

Workshop

LTRR: Learning To Rank Retrievers for LLMs
T.-E. Kim and F. Diaz
SIGIR LiveRAG Workshop, 2025
Best-Case Retrieval Evaluation: Improving the Sensitivity of Reciprocal Rank with Lexicographic Precision
F. Diaz
EVIA, 2023
Striving for data-model efficiency: Identifying data externalities on group performance
E. Rolf, B. Packer, A. Beutel, F. Diaz
NeurIPS 2022 Workshop on Trustworthy and Socially Responsible Machine Learning (TSRML 2022)
On Evaluating Session-Based Recommendation with Implicit Feedback
F. Diaz
Perspectives on the Evaluation of Recommender Systems Workshop, RecSys 2021
The Ethics of Autonomous Experimentation
S. Bird, S. Barocas, K. Crawford, F. Diaz, H. Wallach
FATML 2016
Exploratory Gradient Boosting for Reinforcement Learning in Complex Domains
D. Abel, A. Agarwal, F. Diaz, A. Krishnamurthy, R. Schapire
ICML Workshop on Abstraction in Reinforcement Learning, 2016
Experimentation Standards for Crisis Informatics
F. Diaz
KDD Workshop on Data Science for Social Good 2014
Geographic Features in Web Search Retrieval
A. Hassan, R. Jones, F. Diaz
GIR 2008
UMass at Robust 2005: Using mixtures of relevance models for query expansion
D. Metzler, F. Diaz, T. Strohman, W. B. Croft
TREC 2005
When less is more: Relevance feedback falls short and term expansion succeeds at HARD 2005
F. Diaz and J. Allan
TREC 2005
UMass at TREC 2004: Novelty and HARD
N. Abdul-Jaleel, J. Allan, W. B. Croft, F. Diaz, L. Larkey, X. Li, M. D. Smucker, C. Wade
TREC 2004
Browsing-based user language models for information retrieval
F. Diaz and J. Allan
University of Massachusetts Amherst, 2003

Dissertation

Autocorrelation and Regularization of Query-Based Retrieval Scores
F. Diaz
University of Massachusetts, 2008

Tutorial

Tutorial on Retrieval-Enhanced Machine Learning: Synthesis and Opportunities
F. Diaz, A. Drozdov, T.-E. Kim, A. Salemi, H. Zamani
SIGIR, 2025
Mixed Method Development of Evaluation Metrics
P. Chandar, F. Diaz, C. Hosey, B. St. Thomas
KDD 2021
Beyond Accuracy: Grounding Evaluation Metrics for Human-Machine Learning Systems
P. Chandar, F. Diaz, B. St. Thomas
NeurIPS 2020
Fairness and discrimination in retrieval and recommendation
M. Ekstrand, R. Burke, F. Diaz
RecSys 2019
Fairness and discrimination in retrieval and recommendation
M. Ekstrand, R. Burke, F. Diaz
SIGIR 2019
Mixed methods for evaluating user satisfaction
J. Garcia-Gathright, C. Hosey, B. St. Thomas, B. Carterette, F. Diaz
RecSys 2018
Leveraging Social Media and Web of Data to Assist Crisis Response Coordination
C. Castillo, F. Diaz, H. Purohit
SDM 2014
Temporal web dynamics and its application to information retrieval
K. Radinsky, F. Diaz, S. Dumais, M. Shokouhi, A. Dong, Y. Chang
WSDM 2013
Integrating and ranking aggregated content on the web
F. Diaz, J. Arguello, M. Shokouhi
WWW 2012
From Federated to Aggregated Search
F. Diaz, M. Lalmas, M. Shokouhi
SIGIR 2010