Selected and/or recent papers by William W. Cohen

Recent papers: 2024

1. Tal Schuster, Adam D. Lelkes, Haitian Sun, Jai Gupta, Jonathan Berant, William W. Cohen, Donald Metzler (2024): SEMQA: Semi-Extractive Multi-Source Question Answering in NAACL-2024.
2. Yury Zemlyanskiy, Michiel de Jong, Luke Vilnis, Santiago Ontañón, William W. Cohen, Sumit Sanghai, Joshua Ainslie (2024): MEMORY-VQ: Compression for Tractable Internet-Scale Memory in NAACL-2024.

Recent papers: 2023

1. Chung-Ching Chang, William W. Cohen, Yun-Hsuan Sung (2023): Characterizing Tradeoffs in Language Model Decoding with Informational Interpretations in progress.
2. Haitian Sun, William W. Cohen, Ruslan Salakhutdinov (2023): Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions in progress.
   
   • Following up the 'QA is the new KR' paper, we present a new collection of question-answer pairs automatically generated from Wikipedia which are more specific and ambiiguous than generated questions used in prior work, and show that this can be used to answer ambiguous questions. On the challenging ASQA benchmark, which requires generating long-form answers that summarize the multiple answers to an ambiguous question, our method improves performance by 10-15%. The new queston DB can also be used to improve diverse passage retrieval.
3. Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Sumit Sanghai, William W. Cohen, Joshua Ainslie (2023): GLIMMER: generalized late-interaction memory reranker in progress.
4. Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, William W. Cohen (2023): Subject-driven Text-to-Image Generation via Apprenticeship Learning in NeurIPS-2023.
5. Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William W. Cohen (2023): Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute in ICML-2023.
6. Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, William W. Cohen (2023): MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text in EACL-2023.
7. Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, William Cohen (2023): FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference in ACL-2023 (Findings).
8. Wenhu Chen, Hexiang Hu, Chitwan Saharia, William W. Cohen (2023): Re-Imagen: Retrieval-Augmented Text-to-Image Generator in ICLR-2023.
9. Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, William W. Cohen (2023): WinoDict: Probing language models for in-context word acquisition in EACL-2023.
   
   • One of two winners of an Outstanding Paper Award at EACL.
10. John Wieting, Jonathan H. Clark, William W. Cohen, Graham Neubig, Taylor Berg-Kirkpatrick (2023): Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval in ACL-2023.
11. Wenhu Chen, William W. Cohen, Michiel De Jong, Nitish Gupta, Alessandro Presta, Pat Verga, John Wieting (2023): QA Is the New KR: Question-Answer Pairs as Knowledge Bases in AAAI-2023.
    
    • Proposes that symbolic KBs can be replaced with a collection of question-answer pairs automatically generated from a corpus, augmented with entity-linking annotations. Like a symbolic KB, this representation is well-suited to structured queries involving joins and aggregation, and can support 'multi-hop' reasoning. However, it has the advantage that the information in it is closely aligned to likely user information needs, as modeled by the question generation process.
12. Haitian Sun, William W. Cohen, Ruslan Salakhutdinov (2023): Scenario-based Question Answering with Interacting Contextual Properties in ICLR-2023.

Recent papers: 2022

1. Haitian Sun, William W. Cohen, Ruslan Salakhutdinov (2022): Reasoning over Logically Interacted Conditions for Question Answering in progress.
2. Wenhu Chen, Xueguang Ma, Xinyi Wang, William W. Cohen (2022): Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks in progress.
3. Vidhisha Balachandran, Hannaneh Hajishirzi, William Cohen, Yulia Tsvetkov (2022): Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling in EMNLP-2022.
4. Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, and Kellie Webster (2022): Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models in progress.
5. Wenhu Chen, Pat Verga, Michiel de Jong, John Wieting, William W. Cohen (2022): Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering in EACL-2022.
   
   • Extends the techniques of Mention Memory in several important ways. (1) The memory is a memory of generated question-answer pairs, which is more interpretable than neural entity-mention encodings; (2) it is based on pre-trained T5, not a custom Transformer; and (3) it allows use of the token-level encoding of retrieved QA pairs as well as neural encodings of them for reasoning. Using QA pairs instead of passages allows a clever pre-training trick for learning to retrieve, and the model greatly outperfoms a prior similar model (i.e., RePAQ) on smaller QA benchmarks.
6. Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen and Donald Metzler (2022): Transformer Memory as a Differentiable Search Index in NeurIPS 2022.
7. Siddhant Arora, Danish Pruthi, Norman Sadeh, William W. Cohen, Zachary C. Lipton, Graham Neubig (2022): Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations in AAAI 2022.

Selected other papers

1. Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Fei Sha, William Cohen (2021): Mention Memory: incorporating textual knowledge into Transformers through entity mention attention in ICLR 2021.
   
   • Similar to the Entities-as-Experts model, but uses a much larger memory of entity mentions, which allows the model to potentially provide meaningful provenance for information. The model, called TOME, outperforms Entities-as-Experts on several tasks, and required some non-trivial technical innovations relating to memory pre-training and efficient retrieval.
2. Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen (2021): Time-Aware Language Models as Temporal Knowledge Bases in preparation.
3. Pat Verga, Haitian Sun, Livio Baldini Soares, and William W. Cohen (2021): Adaptable and Interpretable Neural Memory Over Symbolic Knowledge in NAACL-2021.
   
   • Most recent paper on Fact-Injected Language Model (FILM), which includes an Entities-as-Experts style memory of neural entity encodings, plus a second "fact memory" of KG triples. FILM has good results on KBQA tasks, and allows one to use an edited KB with retraining.
4. Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang Ren, William W. Cohen (2020): Differentiable Open-Ended Commonsense Reasoning in NAACL-2021.
   
   • Extends DrKIT's virtual KB to a corpus of documents of common-sense statements ("facts"). In DrFact, entities are replaced by noisy and ambiguous concepts, and navigation is between documents with overlapping sets of mentions. Also introduces new "open" tasks for common-sense QA.
5. Haitian Sun, Andrew O. Arnold, Tania Bedrax-Weiss, Fernando Pereira, William W. Cohen (2020): Faithful Embeddings for Knowledge Base Queries in NeurIPS2020.
   
   • An extension to Neural Query Language (NQL) which extends the query language to work with a "centroid-sketch" representation of sets. The centroid encoders a geometric area, and the sketch is a randomized data structure that adds capacity to the sketch, allowing faithful differential logical reasoning to be combined with good generalization.
6. Pat Verga, Haitian Sun, Livio Baldini Soares, and William W. Cohen (2020): Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge in arxiv.
   
   • Earlier draft of the NAACL paper on FILM (Fact-Injected LM).
7. William W. Cohen, Fan Yang, and Kathryn Rivard Mazaitis (2020): TensorLog: A Probabilistic Database Implemented Using Deep-Learning Infrastructure in JAIR.
   
   • Most complete paper on TensorLog, a predecessor of NQL/EmQL that was a Prolog-like logic, not a dataflow query language.
8. William W. Cohen, Haitian Sun, R. Alex Hofer, Matthew Siegler (2020): Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base in ICLR-2020.
   
   • Paper on Neural Query Language (NQL) a differentiable dataflow query language. NQL is useful for building KBQA systems that can be trained from denotations, but relies heavily on sparse-matrix operations that are not implemented in all accelerators.
9. Bhuwan Dhingra, Manzil Zaheer, Vidhisha Balachandran, Graham Neubig, Ruslan Salakhutdinov, William W. Cohen (2020): Differentiable Reasoning over a Virtual Knowledge Base in ICLR-2020.
   
   • Describes DrKIT, which allows one to answer multihop chain queries on a "virtual KB"---a corpus of entity-linked documents. In DrKIT, entity mentions are indexed for neural retrieval with a rich representation of their context, and reasoning consists of navigating between co-occurring mentions.
10. Haitian Sun, Tania Bedrax-Weiss, William W. Cohen (2019): PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text in EMNLP-2019.
11. Zhilin Yang, Jake (Junbo) Zhao, Bhuwan Dhingra, Kaiming He, William W. Cohen, Ruslan Salakhutdinov, Yann LeCun (2018): GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations in NIPS-2018.
12. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, Christopher D. Manning (2018): HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering in EMNLP-2018.
13. Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William W. Cohen (2018): Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text in EMNLP-2018.
14. Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov (2018): Neural Models for Reasoning over Multiple Mentions using Coreference in NAACL-2018.
15. T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, B. Yang, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. La, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov,M. Greaves, J. Welling (2017): Never-Ending Learning in CACM.
16. Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen (2017): Breaking the Softmax Bottleneck: A High-Rank RNN Language Model in arxiv.org 1711.03953.
17. Fan Yang, Zhilin Yang, William W. Cohen (2017): Differentiable Learning of Logical Rules for Knowledge Base Reasoning in NIPS-2017.
18. William W. Cohen and Fan Yang (2017): TensorLog: Deep Learning Meets Probabilistic Databases in arxiv.org 1707.05390.
19. Bhuwan Dhingra, Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov (2017): Linguistic Knowledge as Memory for Recurrent Neural Networks in arxiv 1703.02620.
20. Rose Catherine, William W. Cohen (2017): TransNets: Learning to Transform for Recommendation in RecSys-2017.
21. Bhuwan Dhingra, Hanxiao Liu, William W. Cohen, and Ruslan Salakhutdinov (2017): Gated-Attention Readers for Text Comprehension in ACL-2017.
22. Rose Catherine and William W. Cohen (2016): Personalized Recommendations using Knowledge Graphs: A Probabilistic Logic Programming Approach in RecSys 2016.
23. Zhilin Yang, Ruslan Salakhutdinov, William Cohen (2016): Revisiting Semi-Supervised Learning with Graph Embeddings in ICML-2016.
24. Jay Pujara, Hui Miao, Lise Getoor, and William W. Cohen (2015): Using semantics and statistics to turn data into knowledge in AI Magazine 2015.
25. William Yang Wang, Kathryn Mazaitis, and William W. Cohen (2015): Joint Information Extraction and Reasoning: A Scalable Statistical Relational Learning Approach in ACL-2015.
26. Dana Movshovitz-Attias and William W. Cohen (2015): KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts in ACL-2015.
27. William Yang Wang, Kathryn Mazaitis, Ni Lao, Tom Mitchell, and William W. Cohen (2015): Efficient Inference and Learning in a Large Knowledge Base: Reasoning with Extracted Information using a Locally Groundable First-Order Probabilistic Logic in Machine Learning, 2015.
28. Bhavana Dalvi, Einat Minkov, Partha P. Talukdar, and William W. Cohen (2015): Automatic Gloss Finding for a Knowledge Base using Ontological Constraints in WSDM-2015.
29. T. Mitchell, W. Cohen, E. Hruscha, P. Talukdar, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner,B. Kisiel,J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohammad, N. Nakashole, E. Platanios,A. Ritter, M. Samadi, B. Settles, R.Wang, D.Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, J.Welling (2015): Never-Ending Learning in AAAI-2015.
30. Ramnath Balasubramanyan and William W. Cohen (2014): Block-LDA: Jointly Modeling Entity-Annotated Text and Entity-Entity Links in Handbook of Mixed Membership Models and Their Applications.
31. William Yang Wang, Kathryn Mazaitis, and William W. Cohen (2014): Structure Learning via Parameter Learning in CIKM-2014.
32. William Yang Wang, Kathryn Mazaitis, William W. Cohen (2013): Programming with Personalized PageRank: A Locally Groundable First-Order Probabilistic Logic in CIKM-2013.
    
    • Honorable Mention for Best Paper at CIKM-2013
33. Jay Pujara, Hui Miao, Lise Getoor, and William W. Cohen (2013): Knowledge Graph Identification in ISWC-2013.
    
    • Best Student Paper at ISWC-2013, and winner of SWSA Ten-Year Award.
34. Ramnath Balasubramanyan and William W. Cohen (2013): Regularization of Latent Variable Models to Obtain Sparsity in SDM-2013.
35. Nan Li, William W. Cohen, Kenneth R. Koedinger (2012): Learning to Perceive Two-Dimensional Displays Using Probabilistic Grammars in ECML-2012.
36. Ni Lao, Amar Subramanya, Fernando Pereira and William W. Cohen (2012): Reading The Web with Learned Syntactic-Semantic Inference Rules in EMNLP-CoNLL-2012.
37. Dana Movshovitz-Attias and William W. Cohen (2012): Bootstrapping Biomedical Ontologies for Scientific Text using NELL in BioNLP-2012.
38. Ramnath Balasubramanyan, William W. Cohen, Doug Pierce, and David P. Redlawsk (2012): Modeling Polarizing Topics: When Do Different Political Communities Respond Differently to the Same News? in ICWSM-2012.
39. Bhavana Dalvi, William W. Cohen, and Jamie Callan (2012): WebSets: Extracting Sets of Entities from the Web Using Unsupervised Information Extraction in WSDM-2012.
40. Ni Lao, Tom Mitchell, and William W. Cohen (2011): Random Walk Inference and Learning in A Large Scale Knowledge Base in EMNLP-2011.
41. Frank Lin and William W. Cohen (2011): Adaptation of Graph-Based Semi-Supervised Methods to Large-Scale Text Data in MLG-2011.
42. Ramnath Balasubramanyan, William W. Cohen, Doug Pierce, and David P. Redlawsk (2011): What pushes their buttons? Predicting comment polarity from the content of political blog posts in LSM-2011.
43. Ramnath Balasubramanyan, Frank Lin, and William W. Cohen (2010): Node Clustering in Graphs: An Empirical Study in NIPS-2010 Workshop on Networks Across Disciplines.
44. Einat Minkov and William W. Cohen (2010): Improving Graph-Walk Based Similarity with Reranking: Case Studies for Personal Information Management in TOIS-2010.
45. Ni Lao and William W. Cohen (2010): Relational Retrieval Using a Combination of Path-Constrained Random Walks in ECML-2010 and MLJ-2010 Special Issue.
46. Frank Lin and William W. Cohen (2010): Power Iteration Clustering in ICML-2010.
47. Frank Lin and William W. Cohen (2010): A Very Fast Method for Clustering Big Text Datasets in ECAI-2010.
48. A. Ahmed, A. Arnold, L. P. Coelho, J. Kangas, A.-S. Sheikh, E. Xing, W. Cohen, and R. F. Murphy (2010): Structured Literature Image Finder: Parsing Text and Figures in Biomedical Literature in Journal of Web Semantics.
49. Richard Wang and William W. Cohen (2009): Automatic Set Instance Extraction using the Web in ACL-IJNLP 2009.
50. Noboru Matsuda, Andrew Lee, William W. Cohen, and Ken Koedinger (2009): A Computational Model of How Learner Errors Arise from Weak Prior Knowledge in CogSci-2009.
51. Amr Ahmed, Andrew Arnold, Luis Pedro Coelho, Joshua Kangas, Abdul-Saboor Sheikk, Eric P. Xing, William W. Cohen, and Robert F. Murphy (2009): Structured Literature Image Finder in Biolink-2009.
52. Tae Yano, Noah A. Smith, and William W. Cohen (2009): Predicting Response to Political Blog Posts with Topic Models in NAACL-2009.
53. Andrew Arnold and William W. Cohen (2009): Information Extraction as Link Prediction: Using Curated Citation Networks to Improve Gene Detection in WASA-2009.
54. Ramesh Nallapati, Amr Ahmed, Eric Xing, and William W. Cohen (2008): Joint Latent Topic Models for Text and Citations in KDD-2008.
55. Richard Wang and William Cohen (2007): Language-Independent Set Expansion of Named Entities using the Web in ICDM-2007.
56. Ramesh Nallapati, William Cohen, Susan Ditmore, John Lafferty and Kin Ung (2007): Multiscale Topic Tomography in KDD-2007.
57. Vitor Carvalho and William W. Cohen (2007): Preventing Information Leaks in Email in SDM-2007.
58. Einat Minkov, Andrew Ng and William W. Cohen (2006): Contextual Search and Name Disambiguation in Email using Graphs in SIGIR-2006.
59. William W. Cohen & Einat Minkov (2006): A Graph-Search Framework for Associating Gene Identifiers with Documents in BMC Bioinformatics.
60. William W. Cohen & Vitor Carvalho (2005): Stacked Sequential Learning in IJCAI-2005.
61. Vitor Carvalho & William W. Cohen (2005): On the Collective Classification of Email Speech Acts in SIGIR 2005.
62. Zhenzhen Kou, William W. Cohen & Robert F. Murphy (2005): High-Recall Protein Entity Recognition Using a Dictionary in ISMB-2005.
63. Sunita Sarawagi & William W. Cohen (2004): Semi-Markov Conditional Random Fields for Information Extraction in NIPS 2004.
64. William W. Cohen, Vitor Carvalho & Tom Mitchell (2004): Learning to Classify Email into "Speech Acts" in EMNLP 2004.
65. Pradeep Ravikumar & William W. Cohen (2004): A Hierarchical Graphical Model for Record Linkage in UAI 2004.
66. William W. Cohen & Sunita Sarawagi (2004): Exploiting Dictionaries in Named Entity Extraction: Combining Semi-Markov Extraction Processes and Data Integration Methods in KDD 2004: 89-98.
67. William W. Cohen (2003): Learning and Discovering Structure in Web Pages in IEEE Data Eng. Bull. 26(3): 3-10 (2003).
68. Mikael Bilenko, Ray Mooney, William W. Cohen, Pradeep Ravikumar & Steve Fienberg (2003): Adaptive Name-Matching in Information Integration in IEEE Intelligent Systems 18(5): 16-23 (2003).
69. William W. Cohen (2003): Infrastructure Components for Large-Scale Information Extraction Systems in IAAI 2003: 71-78.
70. Cheng Zhai, William W. Cohen & John Lafferty (2003): Beyond Independent Topical Relevance: Methods and Evaluation Metrics for Subtopic Retrieval in SIGIR 2003: 10-17.
    
    • Won Ten-Year "Test of Time" Award at SIGIR 2014
71. William W. Cohen, Matthew Hurst & Lee S. Jensen (2003): A Flexible Learning System for Wrapping Tables and Lists in HTML Documents in Web Document Analysis: Challenges and Opportunities, ed. Antonacopoulos & Hu, Word Scientific Publishing.
72. Chumki Basu, Haym Hirsh, William W. Cohen & Craig Neville-Manning (2001): Technical Paper Recommendation: A Study in Combining Multiple Information Sources in J. Artif. Intell. Res. (JAIR) 14: 231-252 (2001).
73. William W. Cohen, David McAllester, and Henry Kautz (2000): Hardening Soft Information Sources in KDD 2000: 255-259.
74. William W. Cohen (2000): Automatically extracting features for concept learning from the Web in ICML 2000: 159-166.
75. William W. Cohen and Wei Fan (2000): Web-Collaborative Filtering: Recommending Music by Crawling The Web in Computer Networks 33(1-6): 685-698 (2000).
76. William W. Cohen and Wei Fan (2000): Web-Collaborative Filtering: Recommending Music by Crawling The Web in WWW 2000.
77. William W. Cohen (2000): Data Integration using Similarity Joins and a Word-based Information Representation Language in ACM Trans. Inf. Syst. 18(3): 288-321 (2000).
78. William W. Cohen and Yoram Singer (1999): Simple, Fast, and Effective Rule Learner in AAAI/IAAI 1999: 335-342.
79. William W. Cohen, Rob Schapire, Yoram Singer (1999): Learning to Order Things in J. Artif. Intell. Res. (JAIR) 10: 243-270 (1999).
80. William W. Cohen (1996): Learning Trees and Rules with Set-valued Features in AAAI/IAAI, Vol. 1 1996: 709-716.
81. William W. Cohen (1996): The Dual DFA Learning Problem: Hardness Results for Programming by Demonstration and Learning First-Order Representations (Extended Abstract) in COLT 1996: 29-40.
82. William W. Cohen (1996): Learning Rules that Classify E-Mail in AAAI Spring Symposium on ML and IR 1996.
83. William W. Cohen and Yoram Singer (1996): Context-sensitive learning methods for text categorization in SIGIR 1996: 307-315.
84. William W. Cohen (1995): Fast effective rule induction in ICML 1995: 115-123.
85. William W. Cohen and Haym Hirsh (1994): Learning the CLASSIC description logic: Theoretical and experimental results in KR 1994: 121-133.

[Selected papers| By topic: GNAT System| Retrieval Augmented LMs| Applications| Collaborative Filtering| Intelligent Tutoring| Explanation-Based Learning| Formal Results| Learning in Graphs| Inductive Logic Programming| Neural Knowledge Representation| Topic Modeling| Matching/Data Integration| Deep Learning| Rule Learning| Text Categorization| Info Extraction/Reading/QA| By year: All papers]