Egocentric Vision

• Surveys
• Papers
• Datasets
• Challenges
• Devices

Surveys

• Egocentric Vision-based Action Recognition: A survey - Adrián Núñez-Marcos, Gorka Azkune, Ignacio Arganda-Carreras, Neurocomputing 2021

• Predicting the future from first person (egocentric) vision: A survey - Ivan Rodin, Antonino Furnari, Dimitrios Mavroedis, Giovanni Maria Farinella, CVIU 2021

• Analysis of the hands in egocentric vision: A survey - Andrea Bandini, José Zariffa, TPAMI 2020

• Summarization of Egocentric Videos: A Comprehensive Survey - Ana Garcia del Molino, Cheston Tan, Joo-Hwee Lim, Ah-Hwee Tan, THMS 2017

• A survey of activity recognition in egocentric lifelogging datasets - El Asnaoui Khalid, Aksasse Hamid, Aksasse Brahim, Ouanan Mohammed, WITS 2017

• Recognition of Activities of Daily Living with Egocentric Vision: A Review - Thi-Hoa-Cuc Nguyen, Jean-Christophe Nebel, Francisco Florez-Revuelta, Sensors 2016

• The Evolution of First Person Vision Methods: A Survey - Alejandro Betancourt, Pietro Morerio, Carlo S. Regazzoni, Matthias Rauterberg, TCSVT 2015

Papers

Action/Activity Recognition

• Stacked Temporal Attention: Improving First-person Action Recognition by Emphasizing Discriminative Clips - Lijin Yang, Yifei Huang, Yusuke Sugano, Yoichi Sato, BMVC 2021

• With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition - Evangelos Kazakos, Jaesung Huh, Arsha Nagrani, Andrew Zisserman, Dima Damen, BMVC 2021

• Interactive Prototype Learning for Egocentric Action Recognition Xiaohan Wang, Linchao Zhu, Heng Wang, Yi Yang, ICCV 2021.

• Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionaries - Swathikiran Sudhakaran, Sergio Escalera, Oswald Lanz, T-PAMI 2021

• Slow-Fast Auditory Streams For Audio Recognition - Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen, ICASSP 2021

• Integrating Human Gaze Into Attention for Egocentric Activity Recognition - Kyle Min, Jason J. Corso, WACV 2021.

• Self-Supervised Joint Encoding of Motion and Appearance for First Person Action Recognition - Mirco Planamente, Andrea Bottino, Barbara Caputo, ICPR 2020

• Gate-Shift Networks for Video Action Recognition - Swathikiran Sudhakaran, Sergio Escalera, Oswald Lanz, CVPR 2020. [code]

• Trear: Transformer-based RGB-D Egocentric Action Recognition - Xiangyu Li, Yonghong Hou, Pichao Wang, Zhimin Gao, Mingliang Xu, Wanqing Li, TCDS 2020

• EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition - Kazakos, Evangelos and Nagrani, Arsha and Zisserman, Andrew and Damen, Dima, ICCV 2019. [code] [project page]

• Learning Spatiotemporal Attention for Egocentric Action Recognition - Minlong Lu, Danping Liao, Ze-Nian Li, WICCV 2019

• Multitask Learning to Improve Egocentric Action Recognition - Georgios Kapidis, Ronald Poppe, Elsbeth van Dam, Lucas Noldus, Remco Veltkamp, WICCV 2019

• Seeing and Hearing Egocentric Actions: How Much Can We Learn? - Alejandro Cartas, Jordi Luque, Petia Radeva, Carlos Segura, Mariella Dimiccoli, WICCV19

• Deep Attention Network for Egocentric Action Recognition - Minlong Lu, Simon Fraser, Ze-Nian Li, Yueming Wang, Gang Pan, TIP 2019

• LSTA: Long Short-Term Attention for Egocentric Action Recognition - Sudhakaran, Swathikiran and Escalera, Sergio and Lanz, Oswald, CVPR 2019. [code]

• Long-Term Feature Banks for Detailed Video Understanding - Chao-Yuan Wu, Christoph Feichtenhofer, Haoqi Fan, Kaiming He, Philipp Krähenbühl, Ross Girshick, CVPR 2019

• Attention is All We Need: Nailing Down Object-centric Attention for Egocentric Activity Recognition - Swathikiran Sudhakaran, Oswald Lanz, BMVC 2018

• Egocentric Activity Recognition on a Budget - Possas, Rafael and Caceres, Sheila Pinto and Ramos, Fabio, CVPR 2018. [demo]

• In the eye of beholder: Joint learning of gaze and actions in first person video - Li, Y., Liu, M., & Rehg, J. M., ECCV 2018.

• Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules - Cao, Congqi and Zhang, Yifan and Wu, Yi and Lu, Hanqing and Cheng, Jian, ICCV 2017.

• Action recognition in RGB-D egocentric videos - Yansong Tang, Yi Tian, Jiwen Lu, Jianjiang Feng, Jie Zhou, ICIP 2017

• Trajectory Aligned Features For First Person Action Recognition - S. Singh, C. Arora, and C.V. Jawahar, Pattern Recognition 2017.

• Modeling Sub-Event Dynamics in First-Person Action Recognition - Hasan F. M. Zaki, Faisal Shafait, Ajmal Mian, CVPR 2017

• First Person Action Recognition Using Deep Learned Descriptors - S. Singh, C. Arora, and C.V. Jawahar, CVPR 2016. [project page] [code]

• Delving into egocentric actions - Li, Y., Ye, Z., & Rehg, J. M., CVPR 2015.

• Pooled Motion Features for First-Person Videos - Michael S. Ryoo, Brandon Rothrock and Larry H. Matthies, CVPR 2015.

• Generating Notifications for Missing Actions: Don't forget to turn the lights off! - Soran, Bilge, Ali Farhadi, and Linda Shapiro, ICCV 2015.

• First-Person Activity Recognition: What Are They Doing to Me? - M. S. Ryoo and L. Matthies, CVPR 2013.

• Detecting activities of daily living in first-person camera views - Pirsiavash, H., & Ramanan, D., CVPR 2012.

• Learning to recognize daily actions using gaze - Fathi, A., Li, Y., & Rehg, J. M, ECCV 2012.

Hand-Object Interactions

• Egocentric Human-Object Interaction Detection Exploiting Synthetic Data - Rosario Leonardi, Francesco Ragusa, Antonino Furnari, Giovanni Maria Farinella ICIAP 2022

• Learning Visual Affordance Grounding from Demonstration Videos - Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, Dacheng Tao, 2021

• Domain and View-point Agnostic Hand Action Recognition - Alberto Sabater, Iñigo Alonso, Luis Montesano, Ana C. Murillo, 2021

• Understanding Egocentric Hand-Object Interactions from Hand Estimation - Yao Lu, Walterio W. Mayol-Cuevas, 2021

• Egocentric Hand-object Interaction Detection and Application - Yao Lu, Walterio W. Mayol-Cuevas, 2021

• The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain - Francesco Ragusa and Antonino Furnari and Salvatore Livatino and Giovanni Maria Farinella, WACV 2021. [project page]

• Is First Person Vision Challenging for Object Tracking? - Matteo Dunnhofer, Antonino Furnari, Giovanni Maria Farinella, Christian Micheloni, WICCV 2021

• Real Time Egocentric Object Segmentation: THU-READ Labeling and Benchmarking Results - E. Gonzalez-Sosa, G. Robledo, D. Gonzalez-Morin, P. Perez-Garcia, A. Villegas, WCVPR 2021

• Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video - Miao Liu, Siyu Tang, Yin Li, James M. Rehg, ECCV 2020. [project page]

• Understanding Human Hands in Contact at Internet Scale - Dandan Shan, Jiaqi Geng, Michelle Shu, David F. Fouhey, CVPR 2020

• Generalizing Hand Segmentation in Egocentric Videos with Uncertainty-Guided Model Adaptation - Minjie Cai and Feng Lu and Yoichi Sato, CVPR 2020. [code]

• Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild - Dominik Kulon, Riza Alp Güler, Iasonas Kokkinos, Michael Bronstein, Stefanos Zafeiriou, CVPR 2020

• Hand-Priming in Object Localization for Assistive Egocentric Vision - Lee, Kyungjun and Shrivastava, Abhinav and Kacorri, Hernisa, WACV 2020.

• Learning joint reconstruction of hands and manipulated objects - Yana Hasson, Gül Varol, Dimitrios Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, Cordelia Schmid, CVPR 2020

• H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions - Tekin, Bugra and Bogo, Federica and Pollefeys, Marc, CVPR 2019. [video]

• From Lifestyle VLOGs to Everyday Interaction - David F. Fouhey and Weicheng Kuo and Alexei A. Efros and Jitendra Malik, CVPR 2018. [project page]

• Analysis of Hand Segmentation in the Wild - Aisha Urooj, Ali Borj, CVPR 2018.

• First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations - Garcia-Hernando, Guillermo and Yuan, Shanxin and Baek, Seungryul and Kim, Tae-Kyun, CVPR 2018. [project page] [code]

• Jointly Recognizing Object Fluents and Tasks in Egocentric Videos - Liu, Yang and Wei, Ping and Zhu, Song-Chun, ICCV 2017.

• Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks with Spatiotemporal Transformer Modules - Cao, Congqi and Zhang, Yifan and Wu, Yi and Lu, Hanqing and Cheng, Jian, ICCV 2017.

• First Person Action-Object Detection with EgoNet - Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi, 2017

• Understanding Hand-Object Manipulation with Grasp Types and Object Attributes - Minjie Cai and Kris M. Kitani and Yoichi Sato, Robotics: Science and Systems 2016.

• Lending a hand: Detecting hands and recognizing activities in complex egocentric interactions - Bambach, S., Lee, S., Crandall, D. J., & Yu, C., ICCV 2015.

• Understanding Everyday Hands in Action From RGB-D Images - Gregory Rogez, James S. Supancic III, Deva Ramanan, ICCV 2015

• You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video - Dima Damen, Teesid Leelasawassuk, Osian Haines, Andrew Calway, and Walterio Mayol-Cuevas, BMVC 2014

• Detecting Snap Points in Egocentric Video with a Web Photo Prior - Bo Xiong and Kristen Grauman, ECCV 2014. [project page] [code]

• 3D Hand Pose Detection in Egocentric RGB-D Images - Grégory Rogez, Maryam Khademi, J. S. Supančič III, J. M. M. Montiel, Deva Ramanan, WECCV 2014

• Pixel-level hand detection in ego-centric videos - Li, Cheng, and Kris M. Kitani. CVPR 2013. [video] [code]

• Learning to recognize objects in egocentric activities - Fathi, A., Ren, X., & Rehg, J. M., CVPR 2011.

• Context-based vision system for place and object recognition - Torralba, A., Murphy, K. P., Freeman, W. T., & Rubin, M. A., ICCV 2003. [project page]

Usupervised Domain Adaptation

• Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition - Mirco Planamente, Chiara Plizzari, Emanuele Alberti, Barbara Caputo, WACV 2022

• Differentiated Learning for Multi-Modal Domain Adaptation - Jianming Lv, Kaijie Liu, Shengfeng He, MM 2021

• Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval - Jonathan Munro, Michael Wray, Diane Larlus, Gabriela Csurka, Dima Damen, 2021

• Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing - Aadarsh Sahoo, Rutav Shah, Rameswar Panda, Kate Saenko, Abir Das, NIPS 2021

• Learning Cross-modal Contrastive Features for Video Domain Adaptation - Donghyun Kim, Yi-Hsuan Tsai, Bingbing Zhuang, Xiang Yu, Stan Sclaroff, Kate Saenko, Manmohan Chandraker, ICCV 2021

• Spatio-temporal Contrastive Domain Adaptation for Action Recognition - Xiaolin Song, Sicheng Zhao, Jingyu Yang, Huanjing Yue, Pengfei Xu, Runbo Hu, Hua Chai, CVPR 2021

• Multi-Modal Domain Adaptation for Fine-Grained Action Recognition - Jonathan Munro, Dima Damen, CVPR 2020

Domain Generalization

• Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition - Mirco Planamente, Chiara Plizzari, Emanuele Alberti, Barbara Caputo, WACV 2022

Action Anticipation

Short-Term Action Anticipation

• Action Anticipation Using Pairwise Human-Object Interactions and Transformers - Debaditya Roy; Basura Fernando, TIP 2021

• Higher Order Recurrent Space-Time Transformer for Video Action Prediction - Tsung-Ming Tai, Giuseppe Fiameni, Cheng-Kuang Lee, Oswald Lanz, ArXiv 2021

• Anticipating Human Actions by Correlating Past With the Future With Jaccard Similarity Measures - Basura Fernando, Samitha Herath, CVPR 2021

• Towards Streaming Egocentric Action Anticipation - Antonino Furnari, Giovanni Maria Farinella, arXiv 2021

• Multimodal Global Relation Knowledge Distillation for Egocentric Action Anticipation - Y Huang, X Yang, C Xu, ACM 2021

• Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videos - Olga Zatsarynna, Yazan Abu Farha, Juergen Gall, CVPRW 2021

• Self-Regulated Learning for Egocentric Video Activity Anticipation - Zhaobo Qi; Shuhui Wang; Chi Su; Li Su; Qingming Huang; Qi Tian, T-PAMI 2021

• Anticipative Video Transformer - Rohit Girdhar, Kristen Grauman, ICCV 2021

• What If We Could Not See? Counterfactual Analysis for Egocentric Action Anticipation - T Zhang, W Min, J Yang, T Liu, S Jiang, Y Rui, IJCAI 2021

• Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video - Antonino Furnari, Giovanni Maria Farinella, T-PAMI 2020

• Knowledge Distillation for Action Anticipation via Label Smoothing - Guglielmo Camporese, Pasquale Coscia, Antonino Furnari, Giovanni Maria Farinella, Lamberto Ballan, ICPR 2020

• An Egocentric Action Anticipation Framework via Fusing Intuition and Analysis - Tianyu Zhang, Weiqing Min, Ying Zhu, Yong Rui, Shuqiang Jiang, ACM 2020

• What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attention - Antonino Furnari, Giovanni Maria Farinella, ICCV 2019 [code] [demo]

• Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video - Miao Liu, Siyu Tang, Yin Li, James M. Rehg, ECCV 2020. [project page]

• Leveraging the Present to Anticipate the Future in Videos - Antoine Miech, Ivan Laptev, Josef Sivic, Heng Wang, Lorenzo Torresani, Du Tran, CVPRW 2019

• Zero-Shot Anticipation for Instructional Activities - Fadime Sener, Angela Yao, ICCV 2019

Long-Term Action Anticipation

• Learning to Anticipate Egocentric Actions by Imagination - Yu Wu, Linchao Zhu, Xiaohan Wang, Yi Yang, Fei Wu, TIP 2021.

• On Diverse Asynchronous Activity Anticipation - He Zhao and Richard P. Wildes, ECCV 2020

• Time-Conditioned Action Anticipation in One Shot - Qiuhong Ke, Mario Fritz, Bernt Schiele, CVPR 2019

• When Will You Do What? - Anticipating Temporal Occurrences of Activities - Yazan Abu Farha, Alexander Richard, Juergen Gall, CVPR 2018

• Joint Prediction of Activity Labels and Starting Times in Untrimmed Videos - Tahmida Mahmud, Mahmudul Hasan, Amit K. Roy-Chowdhury, ICCV 2017

• First-Person Activity Forecasting with Online Inverse Reinforcement Learning - Nicholas Rhinehart, Kris M. Kitani, ICCV 2017. [project page] [video]

Future Gaze Prediction

• Unsupervised gaze prediction in egocentric videos by energy-based surprise modeling, Aakur, S.N., Bagavathi, A., ArXiv 2020

• Digging Deeper into Egocentric Gaze Prediction - Hamed R. Tavakoli and Esa Rahtu and Juho Kannala and Ali Borji, WACV 2019.

• Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transition - Huang, Y., Cai, M., Li, Z., & Sato, Y., ECCV 2018 [code]

• Deep future gaze: Gaze anticipation on egocentric videos using adversarial networks - Zhang, M., Teck Ma, K., Hwee Lim, J., Zhao, Q., & Feng, J., CVPR 2017. [code]

• Learning to predict gaze in egocentric video - Li, Yin, Alireza Fathi, and James M. Rehg, ICCV 2013.

Trajectory prediction

• Forecasting Action through Contact Representations from First Person Video - Eadom Dessalene; Chinmaya Devaraj; Michael Maynord; Cornelia Fermuller; Yiannis Aloimonos, T-PAMI 2021

• Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View With a Reachability Prior - Makansi, Osama and Cicek, Ozgun and Buchicchio, Kevin and Brox, Thomas, CVPR 2020. [demo] [code] [project page]

• Understanding Human Hands in Contact at Internet Scale - Dandan Shan, Jiaqi Geng, Michelle Shu, David F. Fouhey, CVPR 2020

• Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video - Miao Liu, Siyu Tang, Yin Li, James M. Rehg, ECCV 2020. [project page]

• How Can I See My Future? FvTraj: Using First-person View for Pedestrian Trajectory Prediction - Huikun Bi, Ruisi Zhang, Tianlu Mao, Zhigang Deng, Zhaoqi Wang, ECCV 2020. [presentation video] [summary video]

• Future Person Localization in First-Person Videos- Takuma Yagi; Karttikeya Mangalam; Ryo Yonetani; Yoichi Sato, CVPR 2018

• Egocentric Future Localization - Park, Hyun Soo and Hwang, Jyh-Jing and Niu, Yedong and Shi, Jianbo, CVPR 2016. [demo]

• Going deeper into first-person activity recognition - Ma, M., Fan, H., & Kitani, K. M., CVPR 2016.

Region prediction

• EGO-TOPO: Environment Affordances from Egocentric Video - Nagarajan, Tushar and Li, Yanghao and Feichtenhofer, Christoph and Grauman, Kristen, CVPR 2020. [project page] [demo]

• Forecasting human object interaction: Joint prediction of motor attention and egocentric activity - Liu, M., Tang, S., Li, Y., Rehg, J., arXiv 2019

• Forecasting Hands and Objects in Future Frames - Chenyou Fan, Jangwon Lee, Michael S. Ryoo, ECCVW 2018

• Next-active-object prediction from egocentric videos - Antonino Furnari, Sebastiano Battiato, Kristen Grauman, Giovanni Maria Farinella, JVCIR 2017

• First Person Action-Object Detection with EgoNet, G Bertasius, HS Park, SX Yu, J Shi, arXiv 2016

• Unsupervised Learning of Important Objects From First-Person Videos - Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi, ICCV 2017

Multi-Modalities

Audio-Visual

• Attention Bottlenecks for Multimodal Fusion, Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, Chen Sun, NIPS 2021

• Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition - Mirco Planamente, Chiara Plizzari, Emanuele Alberti, Barbara Caputo, WACV 2022

• With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition - Evangelos Kazakos, Jaesung Huh, Arsha Nagrani, Andrew Zisserman, Dima Damen, BMVC 2021

• Slow-Fast Auditory Streams For Audio Recognition - Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen, ICASSP 2021

• Multi-modal Egocentric Activity Recognition using Audio-Visual Features - Mehmet Ali Arabacı, Fatih Özkan, Elif Surer, Peter Jančovič, Alptekin Temizel, MTA 2020

• EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition - Kazakos, Evangelos and Nagrani, Arsha and Zisserman, Andrew and Damen, Dima, ICCV 2019. [code] [project page]

• Seeing and Hearing Egocentric Actions: How Much Can We Learn? - Alejandro Cartas, Jordi Luque, Petia Radeva, Carlos Segura, Mariella Dimiccoli, WICCV19

Depth

• Trear: Transformer-based RGB-D Egocentric Action Recognition - Xiangyu Li, Yonghong Hou, Pichao Wang, Zhimin Gao, Mingliang Xu, Wanqing Li, TCDS 2020

• First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations - Garcia-Hernando, Guillermo and Yuan, Shanxin and Baek, Seungryul and Kim, Tae-Kyun, CVPR 2018. [project page] [code]

• Multi-stream Deep Neural Networks for RGB-D Egocentric Action Recognition - Yansong Tang, Zian Wang, Jiwen Lu, Jianjiang Feng, Jie Zhou, TCSVT 2018

• Action recognition in RGB-D egocentric videos - Yansong Tang, Yi Tian, Jiwen Lu, Jianjiang Feng, Jie Zhou, ICIP 2017

• Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos - Rachel Luo, Ozan Sener, Silvio Savarese, 3DV 2017

• 3D Hand Pose Detection in Egocentric RGB-D Images - Grégory Rogez, Maryam Khademi, J. S. Supančič III, J. M. M. Montiel, Deva Ramanan, WECCV 2014

Thermal

• Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos - Rachel Luo, Ozan Sener, Silvio Savarese, 3DV 2017

Event

• E(GO)^2MOTION: Motion Augmented Event Stream for Egocentric Action Recognition - Chiara Plizzari, Mirco Planamente, Gabriele Goletto, Marco Cannici, Emanuele Gusso, Matteo Matteucci, Barbara Caputo, 2021

Temporal Segmentation (Action Detection)

• UnweaveNet: Unweaving Activity Stories - Will Price, Carl Vondrick, Dima Damen, 2021

• Temporal Action Segmentation from Timestamp Supervision - Zhe Li, Yazan Abu Farha, Jurgen Gall, CVPR 2021

• Personal-Location-Based Temporal Segmentation of Egocentric Video for Lifelogging Applications - A. Furnari, G. M. Farinella, S. Battiato, Journal of Visual Communication and Image Representation 2017 [demo] [project page]

• Temporal segmentation and activity classification from first-person sensing - Spriggs, Ekaterina H., Fernando De La Torre, and Martial Hebert, Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009.

Retrieval

• Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval - Jonathan Munro, Michael Wray, Diane Larlus, Gabriela Csurka, Dima Damen, 2021

• On Semantic Similarity in Video Retrieval - Michael Wray, Hazel Doughty, Dima Damen, CVPR 2021

• Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings - Michael Wray, Diane Larlus, Gabriela Csurka, Dima Damen, ICCV 2019

Video-Language

• Egocentric Video-Language Pretraining - Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou 2022

• Episodic Memory Question Answering - Samyak Datta, Sameer Dharur, Vincent Cartillier, Ruta Desai, Mukul Khanna, Dhruv Batra, Devi Parikh CVPR 2022

Few-Shot Action Recognition

• Unifying Few- and Zero-Shot Egocentric Action Recognition - Tyler R. Scott, Michael Shvartsman, Karl Ridgeway, CVPRW 2021

Gaze

• 1000 Pupil Segmentations in a Second Using Haar Like Features and Statistical Learning - Wolfgang Fuhl, Johannes Schneider, Enkelejda Kasneci, WICCV 2021

From Third-Person to First-Person

• Ego-Exo: Transferring Visual Representations From Third-Person to First-Person Videos - Yanghao Li, Tushar Nagarajan, Bo Xiong, Kristen Grauman, CVPR 2021

• Actor and Observer: Joint Modeling of First and Third-Person Videos - Gunnar A. Sigurdsson and Abhinav Gupta and Cordelia Schmid and Ali Farhadi and Karteek Alahari, CVPR 2018. [code]

• Making Third Person Techniques Recognize First-Person Actions in Egocentric Videos - Sagar Verma, Pravin Nagar, Divam Gupta, Chetan Arora, ICIP 2018

User Data from an Egocentric Point of View

• Dynamics-regulated kinematic policy for egocentric pose estimation - Zhengyi Luo, Ryo Hachiuma, Ye Yuan, Kris Kitani, NIPS 2021

• Estimating Egocentric 3D Human Pose in Global Space - Jian Wang, Lingjie Liu, Weipeng Xu, Kripasindhu Sarkar, Christian Theobalt, ICCV 2021

• Egocentric Pose Estimation From Human Vision Span - Hao Jiang, Vamsi Krishna Ithapu, ICCV 2021

• EgoRenderer: Rendering Human Avatars From Egocentric Camera Images - Tao Hu, Kripasindhu Sarkar, Lingjie Liu, Matthias Zwicker, Christian Theobalt, ICCV 2021

• Whose Hand Is This? Person Identification From Egocentric Hand Gestures - Satoshi Tsutsui, Yanwei Fu, David J. Crandall, WACV 2021.

• Recognizing Camera Wearer from Hand Gestures in Egocentric Videos - Daksh Thapar, Aditya Nigam, Chetan Arora, MM 2020, code

• You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions - Ng, Evonne and Xiang, Donglai and Joo, Hanbyul and Grauman, Kristen, CVPR 2020. [demo] [project page] [dataset] [code]

• Ego-Pose Estimation and Forecasting as Real-Time PD Control - Ye Yuan and Kris Kitani, ICCV 2019. [code] [project page] [demo]

• xR-EgoPose: Egocentric 3D Human Pose From an HMD Camera - Tome, Denis and Peluse, Patrick and Agapito, Lourdes and Badino, Hernan, ICCV 2019. [demo] [dataset]

• 3D Ego-Pose Estimation via Imitation Learning - Ye Yuan, Kris Kitani, ECCV 2018

Localization

• Egocentric Indoor Localization From Room Layouts and Image Outer Corners - Xiaowei Chen, Guoliang Fan, WICCV 2021

• Egocentric Activity Recognition and Localization on a 3D Map - Miao Liu, Lingni Ma, Kiran Somasundaram, Yin Li, Kristen Grauman, James M. Rehg, Chao Li, 2021

• Egocentric Shopping Cart Localization - E. Spera, A. Furnari, S. Battiato, G. M. Farinella, ICPR 2018.

• Recognizing personal locations from egocentric videos - Furnari, A., Farinella, G. M., & Battiato, S., IEEE Transactions on Human-Machine Systems 2017.

• Context-based vision system for place and object recognition - Torralba, A., Murphy, K. P., Freeman, W. T., & Rubin, M. A., ICCV 2003. [project page]

Privacy protection

• Anonymizing Egocentric Videos - Daksh Thapar, Aditya Nigam, Chetan Arora, ICCV 2021

• Mitigating Bystander Privacy Concerns in Egocentric Activity Recognition with Deep Learning and Intentional Image Degradation - Dimiccoli, M., Marín, J., & Thomaz, E., Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2018.

• Privacy-Preserving Human Activity Recognition from Extreme Low Resolution - Ryoo, M. S., Rothrock, B., Fleming, C., & Yang, H. J., AAAI 2017.

Social Interactions

• EgoCom: A Multi-person Multi-modal Egocentric Communications Dataset - Curtis G. Northcutt and Shengxin Zha and Steven Lovegrove and Richard Newcombe, PAMI 2020.

• Deep Dual Relation Modeling for Egocentric Interaction Recognition - Li, Haoxin and Cai, Yijun and Zheng, Wei-Shi, CVPR 2019.

• Recognizing Micro-Actions and Reactions from Paired Egocentric Videos - Yonetani, Ryo and Kitani, Kris M. and Sato, Yoichi, CVPR 2016.

• Social interactions: A first-person perspective - Fathi, A., Hodgins, J. K., & Rehg, J. M., CVPR 2012.

Multiple Egocentric Tasks

• Ego4D: Around the World in 3,000 Hours of Egocentric Video - Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Christian Fuegen, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, Jitendra Malik, arXiv. [Github] [project page] [video]

Activity-context

• Learning Visual Affordance Grounding from Demonstration Videos - Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, Dacheng Tao, 2021

• Shaping embodied agent behavior with activity-context priors from egocentric video - Tushar Nagarajan, Kristen Grauman, NIPS 2021

• EGO-TOPO: Environment Affordances from Egocentric Video - Tushar Nagarajan, Yanghao Li, Christoph Feichtenhofer, Kristen Grauman, CVPR 2020

Video summarization

• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language - Andy Zeng, Adrian Wong, Stefan Welker, Krzysztof Choromanski, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, Pete Florence, 2022

• Egocentric video summarisation via purpose-oriented frame scoring and selection - V. Javier Traver and Dima Damen, Expert Systems with Applications 2022

• Multi-stream dynamic video Summarization - Mohamed Elfeki, Liqiang Wang, Ali Borji, WACV 2022

• Together Recognizing, Localizing and Summarizing Actions in Egocentric Videos - Abhimanyu Sahu; Ananda S. Chowdhury, TIP 2021

• First person video summarization using different graph representations - Abhimanyu Sahu, Ananda S.Chowdhury, Pattern Recognition Letters 2021

• Summarizing egocentric videos using deep features and optimal clustering - Abhimanyu Sahu, Ananda S.Chowdhury, Neurocomputing 2020

• Text Synopsis Generation for Egocentric Videos - Aidean Sharghi; Niels da Vitoria Lobo; Mubarak Shah, ICPR 2020

• Shot Level Egocentric Video Co-summarization - Abhimanyu Sahu; Ananda S. Chowdhury, ICPR 2018

• Personalized Egocentric Video Summarization of Cultural Tour on User Preferences Input - Patrizia Varini; Giuseppe Serra; Rita Cucchiara, IEEE Transactions on Multimedia 2017

• Highlight Detection with Pairwise Deep Ranking for First-Person Video Summarization - Ting Yao; Tao Mei; Yong Rui, CVPR 2016

• Video Summarization with Long Short-term Memory - Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman, ECCV 2016

• Discovering Picturesque Highlights from Egocentric Vacation Videos - Vinay Bettadapura, Daniel Castro, Irfan Essa, arXiv 2016

• Spatial and temporal scoring for egocentric video summarization - Zhao Guo, Lianli Gao, Xiantong Zhen, Fuhao Zou, Fumin Shen, Kai Zheng, Neurocomputing 2016

• Gaze-Enabled Egocentric Video Summarization via Constrained Submodular Maximization - Jia Xu, Lopamudra Mukherjee, Yin Li, Jamieson Warner, James M. Rehg, Vikas Singh, CVPR 2015

• Predicting Important Objects for Egocentric Video Summarization - Yong Jae Lee & Kristen Grauman, IJCV 2015

• Video Summarization by Learning Submodular Mixtures of Objectives - Michael Gygli, Helmut Grabner, Luc Van Gool, CVPR 2015

• Storyline Representation of Egocentric Videos with an Applications to Story-Based Search - Bo Xiong; Gunhee Kim; Leonid Sigal, ICCV 2015

• Detecting Snap Points in Egocentric Video with a Web Photo Prior - Bo Xiong and Kristen Grauman, ECCV 2014

• Creating Summaries from User Videos - Michael Gygli, Helmut Grabner, Hayko Riemenschneider, and Luc Van Gool, ECCV 2014

• Quasi Real-Time Summarization for Consumer Videos - Bin Zhao, Eric P. Xing, CVPR 2014

• Story-Driven Summarization for Egocentric Video - Zheng Lu and Kristen Grauman, CVPR 2013 [project page]

• Discovering Important People and Objects for Egocentric Video Summarization - Yong Jae Lee, Joydeep Ghosh, and Kristen Grauman, CVPR 2012. [project page]

• Wearable hand activity recognition for event summarization - Mayol, W. W., & Murray, D. W., IEEE International Symposium on Wearable Computers, 2005.

Applications

• Wearable System for Personalized and Privacy-preserving Egocentric Visual Context Detection using On-device Deep Learning - Mina Khan, Glenn Fernandes, Akash Vaish, Mayank Manuja, Pattie Maes, UMAP 2021

• Learning Robot Activities From First-Person Human Videos Using Convolutional Future Regression - Jangwon Lee, Michael S. Ryoo, CVPR 2017

Human to Robot

• R3M: A Universal Visual Representation for Robot Manipulation - Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, Abhinav Gupta 2022

• Learning Robot Activities From First-Person Human Videos Using Convolutional Future Regression - Jangwon Lee, Michael S. Ryoo, CVPR 2017

• One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning - Tianhe Yu, Chelsea Finn, Annie Xie, Sudeep Dasari, Tianhao Zhang, Pieter Abbeel, Sergey Levine, RSS 2014

Asssitive Egocentric Vision

• A Computational Model of Early Word Learning from the Infant's Point of View - Satoshi Tsutsui, Arjun Chandrasekaran, Md Alimoor Reza, David Crandall, Chen Yu, CogSci 2020

• Preserved action recognition in children with autism spectrum disorders: Evidence from an EEG and eye-tracking study - Mohammad Saber Sotoodeh, Hamidreza Taheri-Torbati, Nouchine Hadjikhani, Amandine Lassalle, Psychophysiology 2020

Popular Architectures

2D

• [GSM] Gate-Shift Networks for Video Action Recognition - Swathikiran Sudhakaran, Sergio Escalera, Oswald Lanz, CVPR 2020. [code]
• [TSM] TSM: Temporal Shift Module for Efficient Video Understanding - Ji Lin, Chuang Gan, Song Han, ICCV 2019
• [TBN] EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition - Kazakos, Evangelos and Nagrani, Arsha and Zisserman, Andrew and Damen, Dima, ICCV 2019. [code] [project page]
• [TRN] Temporal Relational Reasoning in Videos - Bolei Zhou, Alex Andonian, Aude Oliva, Antonio Torralba, ECCV 2018. [project page]
• [R(2+1)] A Closer Look at Spatiotemporal Convolutions for Action Recognition - Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, Manohar Paluri, CVPR 2018
• [TSN] Temporal Segment Networks: Towards Good Practices for Deep Action Recognition - Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, Luc Van Gool, ECCV 2016

3D

• [SlowFast] SlowFast Networks for Video Recognition - Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming He, ICCV 2019
• [I3D] Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset - Joao Carreira, Andrew Zisserman, CVPR 2017

RNN

• [LSTA] LSTA: Long Short-Term Attention for Egocentric Action Recognition - Sudhakaran, Swathikiran and Escalera, Sergio and Lanz, Oswald, CVPR 2019. [code]
• [RULSTM] What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attention - Antonino Furnari, Giovanni Maria Farinella, ICCV 2019 [code] [demo]

Transformer

• [Ego-STAN]Building Spatio-temporal Transformers for Egocentric 3D Pose Estimation - Jinman Park, Kimathi Kaai, Saad Hossain, Norikatsu Sumi, Sirisha Rambhatla, Paul Fieguth WCVPR2022
• [XViT] Space-time Mixing Attention for Video Transformer - Adrian Bulat, Juan-Manuel Perez-Rua, Swathikiran Sudhakaran, Brais Martinez, Georgios Tzimiropoulos, NIPS 2021
• [ViViT] ViViT: A Video Vision Transformer Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, Cordelia Schmid, ICCV 2021
• [TimeSformer] Is Space-Time Attention All You Need for Video Understanding? - Gedas Bertasius, Heng Wang, Lorenzo Torresani, ICML 2021

Other EGO-Context

• Revisiting 3D Object Detection From an Egocentric Perspective - Boyang Deng, Charles R. Qi, Mahyar Najibi, Thomas Funkhouser, Yin Zhou, Dragomir Anguelov, NIPS 2021

• Learning by Watching - Jimuyang Zhang, Eshed Ohn-Bar, CVPR 2021

Datasets

• Assembly101 - Procedural activity dataset featuring 4321 videos of people assembling and disassembling 101 “take-apart” toy vehicles. - Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities - Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov, Kun He, Dipika Singhania, Robert Wang, Angela Yao, CVPR 2022. Assembly101

• EgoPAT3D - A large multimodality dataset of more than 1 million frames of RGB-D and IMU streams, with evaluation metrics based on high-quality 2D and 3D labels from semi-automatic annotation. - Egocentric Prediction of Action Target in 3D - Yiming Li, Ziang Cao, Andrew Liang, Benjamin Liang, Luoyao Chen, Hang Zhao, Chen Feng, CVPR 2022.

• EasyCom-Clustering - The first large-scale egocentric video face clustering dataset. - Self-supervised Video-centralised Transformer for Video Face Clustering - Yujiang Wang, Mingzhi Dong, Jie Shen, Yiming Luo, Yiming Lin, Pingchuan Ma, Stavros Petridis, Maja Pantic 2022

• AGD20K - Affordance dataset constructed by collecting and labeling over 20K images from 36 affordance categories. - Learning Affordance Grounding from Exocentric Images - Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, Dacheng Tao, CVPR 2022

• AssistQ - A new dataset comprising 529 question-answer samples derived from 100 newly filmed first-person videos. Each question should be completed with multi-step guidances by inferring from visual details (e.g., buttons' position) and textural details (e.g., actions like press/turn). - AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistant - Benita Wong, Joya Chen, You Wu, Stan Weixian Lei, Dongxing Mao, Difei Gao, Mike Zheng Shou, 2022

• HOI4D - A large-scale 4D egocentric dataset with rich annotations, to catalyze the research of category-level human-object interaction. HOI4D consists of 2.4M RGB-D egocentric video frames over 4000 sequences collected by 4 participants interacting with 800 different object instances from 16 categories over 610 different indoor rooms. Frame-wise annotations for panoptic segmentation, motion segmentation, 3D hand pose, category-level object pose and hand action have also been provided, together with reconstructed object meshes and scene point clouds. - HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction - Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, Li Yi, CVPR 2022

• EgoPW - A dataset captured by a head-mounted fisheye camera and an auxiliary external camera, which provides an additional observation of the human body from a third-person perspective. - Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision - Jian Wang, Lingjie Liu, Weipeng Xu, Kripasindhu Sarkar, Diogo Luvizon, Christian Theobalt CVPR 2022

• First2Third-Pose - A new paired synchronized dataset of nearly 2,000 videos depicting human activities captured from both first- and third-view perspectives. - Enhancing Egocentric 3D Pose Estimation with Third Person Views - Ameya Dhamanaskar, Mariella Dimiccoli, Enric Corona, Albert Pumarola, Francesc Moreno-Noguer, 2022

• Ego4D - 3,025 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 855 unique camera wearers from 74 worldwide locations and 9 different countries - Ego4d: Around the world in 3,000 hours of egocentric video. - K. Grauman, et al. CVPR 2022.

• EgoCom - A natural conversations dataset containing multi-modal human communication data captured simultaneously from the participants' egocentric perspectives. - EgoCom: A Multi-person Multi-modal Egocentric Communications Dataset - Curtis Northcutt, Shengxin Zha, Steven Lovegrove, Richard Newcombe TPAMI 2020

• TREK-100 - Object tracking in first person vision - Is First Person Vision Challenging for Object Tracking? - Matteo Dunnhofer, Antonino Furnari, Giovanni Maria Farinella, Christian Micheloni WICCV 2021

• MECCANO - 20 subject assembling a toy motorbike. - The MECCANO Dataset: Understanding Human-Object Interactions From Egocentric Videos in an Industrial-Like Domain - Francesco Ragusa, Antonino Furnari, Salvatore Livatino, Giovanni Maria Farinella WACV 2021

• EPIC-Kitchens 2020 - Subjects performing unscripted actions in their native environments. - Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 - D. Damen, H. Doughty, G.M. Farinella, A. Furnari, J. Ma, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W.Price, M. Wray IJCV 2021

• EPIC-Tent - 29 participants assembling a tent while wearing two head-mounted cameras. [paper] - EPIC-Tent: An Egocentric Video Dataset for Camping Tent Assembly - Youngkyoon Jang, Brian Sullivan, Casimir Ludwig, Iain Gilchrist, Dima Damen, Walterio Mayol-Cuevas ICCV 2019

• EGO-CH - 70 subjects visiting two cultural sites in Sicily, Italy. - EGO-CH: Dataset and fundamental tasks for visitors behavioral understanding using egocentric vision - Francesco Ragusa, Antonino Furnari, Sebastiano Battiato, Giovanni Signorello, Giovanni Maria Farinella; Pattern Recognition Letters 2020

• EPIC-Kitchens 2018 - 32 subjects performing unscripted actions in their native environments. - Scaling Egocentric Vision: The EPIC-KITCHENS Dataset - Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray ECCV 2018

• Charade-Ego - Paired first-third person videos.

• EGTEA Gaze+ - 32 subjects, 86 cooking sessions, 28 hours.

• ADL - 20 subjects performing daily activities in their native environments.

• CMU kitchen - Multimodal, 18 subjects cooking 5 different recipes: brownies, eggs, pizza, salad, sandwich.

• EgoSeg - Long term actions (walking, running, driving, etc.)

• First-Person Social Interactions - 8 subjects at disneyworld.

• UEC Dataset - Two choreographed datasets with different egoactions (walk, jump, climb, etc.) + 6 YouTube sports videos.

• JPL - Interaction with a robot.

• FPPA - Five subjects performing 5 daily actions.

• UT Egocentric - 3-5 hours long videos capturing a person's day.

• VINST/ Visual Diaries - 31 videos capturing the visual experience of a subject walking from metro station to work.

• Bristol Egocentric Object Interaction (BEOID) - 8 subjects, six locations. Interaction with objects and environment.

• Object Search Dataset - 57 sequences of 55 subjects on search and retrieval tasks.

• UNICT-VEDI - Different subjects visiting a museum.

• UNICT-VEDI-POI - Different subjects visiting a museum.

• Simulated Egocentric Navigations - Simulated navigations of a virtual agent within a large building.

• EgoCart - Egocentric images collected by a shopping cart in a retail store.

• Unsupervised Segmentation of Daily Living Activities - Egocentric videos of daily activities.

• Visual Market Basket Analysis - Egocentric images collected by a shopping cart in a retail store.

• Location Based Segmentation of Egocentric Videos - Egocentric videos of daily activities.

• Recognition of Personal Locations from Egocentric Videos - Egocentric videos clips of daily.

• EgoGesture - 2k videos from 50 subjects performing 83 gestures.

• EgoHands - 48 videos of interactions between two people.

• DoMSEV - 80 hours/different activities.

• DR(eye)VE - 74 videos of people driving.

• THU-READ - 8 subjects performing 40 actions with a head-mounted RGBD camera.

• EgoDexter - 4 sequences with 4 actors (2 female), and varying interactions with various objects and and cluttered background. [paper]

• First-Person Hand Action (FPHA) - 3D hand-object interaction. Includes 1175 videos belonging to 45 different activity categories performed by 6 actors. [paper]

• UTokyo Paired Ego-Video (PEV) - 1,226 pairs of first-person clips extracted from the ones recorded synchronously during dyadic conversations.

• UTokyo Ego-Surf - Contains 8 diverse groups of first-person videos recorded synchronously during face-to-face conversations.

• TEgO: Teachable Egocentric Objects Dataset - Contains egocentric images of 19 distinct objects taken by two people for training a teachable object recognizer.

• Multimodal Focused Interaction Dataset - Contains 377 minutes of continuous multimodal recording captured during 19 sessions, with 17 conversational partners in 18 different indoor/outdoor locations.

Challenges

• Ego4D - Episodic Memory, Hand-Object Interactions, AV Diarization, Social, Forecasting.

• Epic Kithchen Challenge - Action Recognition, Action Detection, Action Anticipation, Unsupervised Domain Adaptation for Action Recognition, Multi-Instance Retrieval

Devices

• GoPro
• Narrative clip
• Autographer5
• Microsoft SenseCam
• SMI eye-tracker
• ASL Mobile eye
• Tobii eye-tracker
• Pupil Invisible
• Microsoft Hololens 2
• Google Glass
• Vusix Blade
• Magic Leap
• Nreal Light
• Epson Moverio
• Realwear
• [TCL Smart Glasses Thunderbird]
• OrCam
• [Xiaomi Smart Glasses]
• Ray-Ban Stories
• dynaEdge
• [Apple Glass]
• Alpha Glass
• GWD HiiDii
• Spectacles

This is a work in progress.