Human-AI Collaboration

Literature for Human-AI collaboration, Hybrid Intelligence, Human-AI interaction

Human-Centered Artificial Intelligence

• Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. (Shneiderman 2020)
• Human-Centered Artificial Intelligence: Three Fresh Ideas. (Shneiderman, 2020)
• Workshop: Human-Centered AI: Reliable, Safe & Trustworthy, University of Maryland, 2020.
• - Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems

Interactive Machine Learning

• Power to the People: The Role of Humans in Interactive Machine Learning (AI Magine 2014)

Hybrid Intelligence

• Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence (Karmar, IJCAI 2016)
• Hybrid Intelligence (Dominik et al., 2019)
• The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems (Dominik et al., 2019)
• On Facilitating Human-Computer Interaction via Hybrid Intelligence Systems (Lasecki, 2019)

Collective Intelligence / Crowdsourcing

• Human-Computer Interaction and Collective Intelligence (Bigham et al., 2014)
• Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research (2018)

Machine Teaching

• Machine Teaching A New Paradigm for Building Machine Learning Systems (2017)
• The Importance of UX for Machine Teaching (SSS18)
• Interactive Teaching Strategies for Agent Training (IJCAI 2016)

Agency vs Automation

• Man-Computer Symbiosis (Licklider, 1960)
• Direct Manipulation vs. Interface Agents (1997)
• Principles of Mixed-Initiative User Interfaces (Horvitz, 1999)
• Agency plus automation: Designing artificial intelligence into interactive systems (Heer, 2019)

Human-AI Interaction

• Guidelines for Human-AI Interaction (Amershi et al., CHI2019)
• Ask not what AI can do, but what AI should do: Towards a framework of task delegability (Lubars an Tan, 2019)

AI/ML as UX

• UX Design Innovation: Challenges for Working with Machine Learning as a Design Material (CHI 2017)
• Reimagining the Goals and Methods of UX for ML/AI
• The Role of Design in Creating Machine-Learning-Enhanced User Experience
• Investigating How Experienced UX Designers Effectively Work with Machine Learning (DIS 2018)
• Mapping Machine Learning Advances from HCI Research to Reveal Starting Places for Design Research (CHI 2018)
• Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design (Yang et al., CHI 2020)

Intelligibility, Explainability, Explainable AI

• How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy (CHI 2015)
• "Why Should I Trust You?": Explaining the Predictions of Any Classifier (KDD 2016)
• Towards A Rigorous Science of Interpretable Machine Learning (2017)
• Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda (CHI 2018)
• Designing Theory-Driven User-Centric Explainable AI (CHI 2019)
• Explaining Decision-Making Algorithms through UI (CHI 2019)
• The Challenge of Crafting Intelligible Intelligence (Weld and Bansal, CACM 2019)
• Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning (Kaur et al., CHI 2020)
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences (Liao et al., CHI 2020)
• No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML (Smith-Renner et al., CHI 2020)

FATE: Fairness, Accountability, Transparency, and Ethics in AI

• Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI (Madaio et al., CHI 2020)
• Principles for Accountable Algorithms and a Social Impact Statement for Algorithms

Human-AI Team

• Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff (AAAI 2019)
• Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance (HCOMP 2019)

Human-in-the-loop ML

• https://livebook.manning.com/book/human-in-the-loop-machine-learning/ (in-progress book, Robert Munro)

Applications for Human-AI Co-Learning

Writing

• Metaphoria: An Algorithmic Companion for Metaphor Creation (CHI 2019)
• [Creative writing with a machine in the loop: Case studies on slogans and stories](IUI 2018)
• LISA: lexically intelligent story assistant (AIIDE 2017)

Creativity (Drawing, Music, etc)

• In a Silent Way: Communication Between AI and Improvising Musicians Beyond Sound (CHI 2019)
• I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence (CHI 2018)

Design

• VisiBlends: An Interactive Pipeline for Creating Visual Blends (CHI 2019)
• Human Errors in Interpreting Visual Metaphor (C&C 2019)

Conversational Agents

• What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents (CHI 2019)
• Caring for Vincent: A Chatbot for Self-compassion(CHI 2019)

Others

• Calendar.help: Designing a workflow-based scheduling agent with humans in the loop
• AI-Mediated Communication: How the Perception that Profile Text was Written by AI Affects Trustworthiness (CHI 2019)

---

Courses

• Human-AI Interaction, by Chinmay Kulkarni an Mary Beth Kery, CMU (formo class by Jeff Bigham and Joseph Seering, Fall 2018)
• CS 889: Human-AI Interaction by Edith Law, U.Waterloo
• CS 294: Fairness in Machine Learning by ,UC Bekeley
• CS598RK: HCI for Machine Learning, by Ranjitha Kumar and Jinda Han,UIUC
• Human-centered Machine Learning (2018 Spring), by Chenhao Tan, University of Colorado Boulder
• CS6724: Advanced Topics in Human-Computer Interaction: Human-AI Interaction, by Kurt Lurther, Virginia Tech
• Designing AI to Cultivate Human Well-Being, by Jennifer Aaker and Fei-Fei Li, Stanford
• CS279R: Research Topics in HCI: Human-AI Interaction
• CS492F: Human-AI Interaction, by Jean Young Song and Juho Kim, KAIST
• CS279R Research Topics in HCI: Human-AI Interaction, by Elena L. Glassman, Harvard

Labs

• Polo Club of Data Science @ Georgia Tech

Workshops & Tutorials

• PAIR 2019 @Google
• HAI 2019 @Stanford
• Machine Teaching Group @MSR
• HAI-GEN 2020 @ IUI 2020 Workshop on Human-AI Co-Creation with Generative Models
• AAAI 2020 Tutorial – Guidelines for Human-AI Interaction
• Participatory Approaches to Machine Learning, ICML 2020 Workshop (July 17, 2020)

Others Links

• https://pair.withgoogle.com
• AI Guidelines in the Creative Process
• https://hcil.umd.edu/human-centered-ai-a-second-copernican-revolution/
• https://hcil.umd.edu/human-centered-ai-trusted-reliable-safe/

Other Awesome HAI list

• https://github.com/bwang514/awesome-HAI/blob/master/README.md

---

For artist

• https://ml4a.github.io/classes/itp-F18/