Recently, conversational models (Galactica, YOU.com, perplexity.ai) started to be able to access the web or backup their claims with sources (a.k.a. attribution). These chatbots are thus arguably information retrieval machines, competing against or even substituing traditional search engines. We would like to dedicate a space to these models but also to the more general field of generative information retrieval. We tentatively devide the field in two main topics: Grounded Answer Generation and Generative Document Retrieval. We also include generative recommendation, generative grounded summarization etc.
Pull-requests welcome!
- Live Generative Retrieval
- Blog Posts
- Datasets
- Tools
- Workshops
- Epistemology Papers
- Grounded Answer Generation
- Generative Document Retrieval
- Summarization
- Generative Recommendation
- Knowledge Graph Generation
Although some of these are not accompanied by a paper, they might be useful to other Generative IR researchers for empirical studies or interface design considerations.
⚡️ devmarizer Mar 2023 [live] ⚡️ TaxGenius Mar 2023 [live] ⚡️ doc-gpt Mar 2023 [live] ⚡️ book-gpt Feb 2023 [live] ⚡️ Neeva Feb 2023 [live] ⚡️ Golden Retriever Feb 2023 [live] ⚡️ Bing – Prometheus Feb 2023 [waitlist] ⚡️ Google – Bard Feb 2023 [under dev.] ⚡️ Paper QA Feb 2023 [code] [demo] ⚡️ DocsGPT Feb 2023 [live] [code] ⚡️ DocAsker Jan 2023 [live] ⚡️ Lexii.ai Jan 2023 [live] ⚡️ YOU.com Dec 2022 [live] ⚡️ arXivGPT Dec 2022 [Chrome extension] ⚡️ GPT Index Nov 2022 [API] ⚡️ BlenderBot Aug 2022 [live (USA)] [model weights] [code] [paper1] [paper2] ⚡️ PHIND date? [live] ⚡️ Perplexity date? [live] ⚡️ Galactica date? [demo] [API] [paper] ⚡️ Elicit date? [live] ⚡️ ZetaAlpha date? [live] uses OpenAI API
Under development: ⚡️ Open-Assistant [open source] [code] ⚡️ Transformer Reinforcement Learning [open source] [code] ⚡️ Sparrow [paper]
Honorable mentions of Claude [private beta] [paper] and ChatGPT [live] that are borderline retrieval models (hallucinate a lot and don't cite their sources for now)
See PowerdByGPT for more.
What Makes a Dialog Agent Useful?
Nazneen Rajani, Nathan Lambert, Victor Sanh, Thomas Wolf
Hugging Face Blog – Jan 2023 [link]
Forecasting potential misuses of language models for disinformation campaigns and how to reduce risk
Josh A. Goldstein, Girish Sastry, Micah Musser, Renée DiResta, Matthew Gentzel, Katerina Sedova
OpenAI Blog – Jan 2023 [link]
ChatGPT-RetrievalQA
Arian Askari, Mohammad Aliannejadi, Evangelos Kanoulas, Suzan Verberne
Github – Feb 2023 [code]
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Stephanie Lin, Jacob Hilton, Owain Evans
arXiv – Sep 2021 [paper] [code]
Complex Answer Retrieval
Laura Dietz, Manisha Verma, Filip Radlinski, Nick Craswell, Ben Gamari, Jeff Dalton, John Foley
TREC – 2017-2019 [link]
PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development
Avirup Sil, Jaydeep Sen, Bhavani Iyer, Martin Franz, Kshitij Fadnis, Mihaela Bornea, Sara Rosenthal, Scott McCarley, Rong Zhang, Vishwajeet Kumar, Yulong Li, Md Arafat Sultan, Riyaz Bhat, Radu Florian, Salim Roukos
arXiv – Jan 2023 [paper] [code]
First Workshop on Generative Information Retrieval
Gabriel Bénédict, Ruqing Zhang, Donald Metzler
SIGIR 23 – Jul 2023 [link]
Generative Recommendation: Towards Next-generation Recommender Paradigm
Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, Weizhu Chen
arXiv – April 2023 [paper]
Augmented Language Models: a Survey
Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, Thomas Scialom
arXiv – Feb 2023 [paper]
Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations
Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, Katerina Sedova
arXiv – Jan 2023 [paper]
Conversational Information Seeking. An Introduction to Conversational Search, Recommendation, and Question Answering
Hamed Zamani, Johanne R. Trippas, Jeff Dalton and Filip Radlinski
arXiv – Jan 2023 [paper]
Facts
Kevin Mulligan and Fabrice Correia
The Stanford Encyclopedia of Philosophy – Winter 2021 [url]
Truthful AI: Developing and governing AI that does not lie
Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, William Saunders
arXiv – Oct 2021 [paper]
Rethinking Search: Making Domain Experts out of Dilettantes
Donald Metzler, Yi Tay, Dara Bahri, Marc Najork
SIGIR Forum 2021 – May 2021 [paper]
Could be subdivided into something like retrieval-enhanced-LLMs and attribution-for-conversations.
Evaluating Verifiability in Generative Search Engines
Nelson F. Liu, Tianyi Zhang, Percy Liang
arXiv – April 2023 [paper] [code]
Active Retrieval Augmented Generation
Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig
arXiv – May 2023 [paper] [code]
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro
arXiv – Apr 2023 [paper] [code]
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, Jianfeng Gao
arXiv – Feb 2023 [paper] [code]
Toolformer: Language Models Can Teach Themselves to Use Tools
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom
arXiv – Feb 2023 [paper]
REPLUG: Retrieval-Augmented Black-Box Language Models
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
arXiv – Jan 2023 [paper]
In-Context Retrieval-Augmented Language Models
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
AI21 Labs – Jan 2023 [paper] [code]
AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Hamed Zamani, Johanne R. Trippas, Jeff Dalton and Filip Radlinski
arXiv – Jan 2023 [paper]
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, Matei Zaharia
arXiv – Dec 2022 [paper]
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster
arXiv – Dec 2022 [paper]
Galactica: A Large Language Model for Science
Ross Taylor, Marcin Kardas, Guillem Cucurull,
Thomas Scialom, Anthony Hartshorn, Elvis Saravia,
Andrew Poulton, Viktor Kerkez, Robert Stojnic
Galactica.org – 2022 [paper]
Recitation-Augmented Language Models
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, Denny Zhou
ICLR 2023 – Sep 2022 [paper]
Generate rather than Retrieve: Large Language Models are Strong Context Generators
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
ICLR 2023 – Sep 2022 [paper]
Improving alignment of dialogue agents via targeted human judgements
Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, Geoffrey Irving
arXiv – Sep 2022 [paper]
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kambadur, Jason Weston
arXiv – Aug 2022 [paper]
LaMDA: Language Models for Dialog Applications
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, Quoc Le
arXiv – Jan 2022 [paper]
WebGPT: Browser-assisted question-answering with human feedback
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, John Schulman
arXiv – Dec 2021 [paper]
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, Laurent Sifre
arXiv – Dec 2021 [paper]
Language Models As or For Knowledge Bases
Simon Razniewski, Andrew Yates, Nora Kassner, Gerhard Weikum
DL4KG 2021 – Oct 2021 [paper]
Recipes for Building an Open-Domain Chatbot
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, Jason Weston
EACL 2021 – Apr 2021 [paper]
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora Kassner, Hinrich Schütze
EMNLP 2020 – Nov 2020 [paper]
Generalization through Memorization: Nearest Neighbor Language Models
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis
ICLR 2020 – Sep 2019 [paper] [code]
A Hybrid Retrieval-Generation Neural Conversation Model
Liu Yang, Junjie Hu, Minghui Qiu, Chen Qu, Jianfeng Gao, W. Bruce Croft, Xiaodong Liu, Yelong Shen, Jingjing Liu
arXiv – Apr 2019 [paper]
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion,
Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon,
Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain,
Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller,
Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosiute, Liane Lovitt,
Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma,
Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec,
Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly,
Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann,
Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Jared Kaplan
Anthropic.com – Dec 2022 [paper]
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback
Jing Xu, Megan Ung, Mojtaba Komeili, Kushal Arora, Y-Lan Boureau, Jason Weston
arXiv – Aug 2022 [paper]
Retrieval-Augmented Multimodal Language Modeling
Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
arXiv – Nov 2022 [paper]
RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training
Zheng Yuan, Qiao Jin, Chuanqi Tan, Zhengyun Zhao, Hongyi Yuan, Fei Huang, Songfang Huang
arXiv – Mar 2023 [paper]
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao
arXiv – Oct 2022 [paper]
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, Weizhu Chen
arXiv – Mar 2023 [[paper](https://arxiv.org/pdf/2303.12570.pdf
DocPrompting: Generating Code by Retrieving the Docs
Shuyan Zhou, Uri Alon, Frank F. Xu, Zhiruo Wang, Zhengbao Jiang, Graham Neubig
ICLR 23 – Jul 2022 [paper] [code] [data]
Discovering Latent Knowledge in Language Models Without Supervision
Collin Burns, Haotian Ye, Dan Klein, Jacob Steinhardt
ICLR 23 – Feb 2023 [paper] [code]
We jump-started this section by reusing the content of awesome-generative-retrieval-models and give full credit to Chriskuei for that! We now have added some content on top.
TOME: A Two-stage Approach for Model-based Retrieval
Ruiyang Ren, Wayne Xin Zhao, Jing Liu, Hua Wu, Ji-Rong Wen, Haifeng Wang
ACL - 2023 [paper]
Understanding Differential Search Index for Text Retrieval
Xiaoyang Chen, Yanjiang Liu, Ben He, Le Sun, Yingfei Sun
ACL Findings - 2023 [paper]
Learning to Tokenize for Generative Retrieval
Weiwei Sun, Lingyong Yan, Zheng Chen, Shuaiqiang Wang, Haichao Zhu, Pengjie Ren, Zhumin Chen, Dawei Yin, Maarten de Rijke, Zhaochun Ren
arXiv – Apr 2023 [paper]
DynamicRetriever: A Pre-trained Model-based IR System Without an Explicit Index
Yu-Jia Zhou, Jing Yao, Zhi-Cheng Dou, Ledell Wu, Ji-Rong Wen
Machine Intelligence Research – Jan 2023 [paper]
DSI++: Updating Transformer Memory with New Documents
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Jinfeng Rao, Marc Najork, Emma Strubell, Donald Metzler
arXiv – Dec 2022 [paper]
CodeDSI: Differentiable Code Search
Usama Nadeem, Noah Ziems, Shaoen Wu
arXiv – Oct 2022 [paper]
Contextualized Generative Retrieval
Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vlad Karpukhin, Yi Lu, Minjoon Seo
arXiv – Oct 2022 [paper]
Transformer Memory as a Differentiable Search Index
Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, Donald Metzler
Neurips 2022 – Oct 2022 [paper] [Video] [third-party code]
A Neural Corpus Indexer for Document Retrieval
Wang et al.
Arxiv 2022 [paper]
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation
Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, and Daxin Jiang
Arxiv 2022 [paper] [Code]
DynamicRetriever: A Pre-training Model-based IR System with Neither Sparse nor Dense Index
Zhou et al
Arxiv 2022 [paper]
Ultron: An Ultimate Retriever on Corpus with a Model-based Indexer
Zhou et al
Arxiv 2022 [paper]
A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yiqun Liu, Yixing Fan, Xueqi Cheng
SIGIR 2023 – Apr 2023 [paper] [Code]
CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks
Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Yiqun Liu, Yixing Fan, Xueqi Cheng
CIKM 2022 – Aug 2022 [paper] [Code]
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen-tau Yih, Sebastian Riedel, Fabio Petroni
arXiv – Apr 2022 [paper] [Code]
Autoregressive Entity Retrieval
Nicola De Cao, Gautier Izacard, Sebastian Riedel, Fabio Petroni
ICLR 2021 – Oct 2020 [paper] [Code]
Data-Efficient Autoregressive Document Retrieval for Fact Verification
James Thorne
SustaiNLP@EMNLP 2022 – Nov 2022 [paper]
GERE: Generative Evidence Retrieval for Fact Verification
Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Yixing Fan, Xueqi Cheng
SIGIR 2022 [paper] [Code]
Generative Multi-hop Retrieval
Hyunji Lee, Sohee Yang, Hanseok Oh, Minjoon Seo
Arxiv – Apr 2022 [paper]
Learning to summarize with human feedback
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, Paul Christiano
NeurIPS 2020 – Sep 2020 [paper]
On Faithfulness and Factuality in Abstractive Summarization
Joshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan McDonald
ACL 2020 – May 2020 [paper]
DiffuRec: A Diffusion Model for Sequential Recommendation
Zihao Li, Aixin Sun, Chenliang Li
arXiv – Apr 2023 [paper]
Diffusion Recommender Model
Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, Tat-Seng Chua
SIGIR 2023 – Apr 2023 [paper]
Blurring-Sharpening Process Models for Collaborative Filtering
Jeongwhan Choi, Seoyoung Hong, Noseong Park, Sung-Bae Cho
SIGIR 2023 – Apr 2023 [paper] [code]
Recommender Systems with Generative Retrieval
Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, Maheswaran Sathiamoorthy
non-archival – Mar 2023 [paper]
Generative Slate Recommendation with Reinforcement Learning
Romain Deffayet, Thibaut Thonet, Jean-Michel Renders, and Maarten de Rijke
WSDM 2023 – Feb 2023 [paper]
Recommendation via Collaborative Diffusion Generative Model
Joojo Walker, Ting Zhong, Fengli Zhang, Qiang Gao, Fan Zhou
KSEM 2022 – Aug 2022 [paper]
From Retrieval to Generation: Efficient and Effective Entity Set Expansion
Shulin Huang, Shirong Ma, Yangning Li, Yinghui Li, Hai-Tao Zheng, Yong Jiang
arXiv – Apr 2023 [paper]
Crawling the Internal Knowledge-Base of Language Models
Roi Cohen, Mor Geva, Jonathan Berant, Amir Globerson
arXiv – Jan 2023 [paper]
To get just the paper titles do grep '\*\*' README.md | sed 's/\*\*//g'