awesome-MIM

Reading list for research topics in Masked Image Modeling(MIM).

We list the most popular methods for MIM, if we missed something, please submit a request. (Note: We show the date the first edition of the paper was submitted to arxiv, but the link to the paper may be up to date.)

Backbone models.

Date	Method	Conference	Title	Code
2020-xx-xx(maybe 2019)	iGPT	ICML 2020	Generative Pretraining from Pixels	iGPT
2020-10-22	ViT	ICLR 2021 (Oral)	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	ViT
2021-04-08	SiT	Arxiv 2021	SiT: Self-supervised vIsion Transformer	None
2021-06-10	MST	NeurIPS 2021	MST: Masked Self-Supervised Transformer for Visual Representation	None
2021-06-14	BEiT	ICLR 2022 (Oral)	BEiT: BERT Pre-Training of Image Transformers	BEiT
2021-11-11	MAE	Arxiv 2021	Masked Autoencoders Are Scalable Vision Learners	MAE
2021-11-15	iBoT	ICLR 2022	iBOT: Image BERT Pre-Training with Online Tokenizer	iBoT
2021-11-18	SimMIM	Arxiv 2021	SimMIM: A Simple Framework for Masked Image Modeling	SimMIM
2021-11-24	PeCo	Arxiv 2021	PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers	None
2021-11-30	MC-SSL0.0	Arxiv 2021	MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning	None
2021-12-16	MaskFeat	Arxiv 2021	Masked Feature Prediction for Self-Supervised Visual Pre-Training	None
2021-12-20	SplitMask	Arxiv 2021	Are Large-scale Datasets Necessary for Self-Supervised Pre-training?	None
2022-01-31	ADIOS	Arxiv 2022	Adversarial Masking for Self-Supervised Learning	None
2022-02-07	CAE	Arxiv 2022	Context Autoencoder for Self-Supervised Representation Learning	CAE
2022-02-07	CIM	Arxiv 2022	Corrupted Image Modeling for Self-Supervised Visual Pre-Training	None
2022-03-10	MVP	Arxiv 2022	MVP: Multimodality-guided Visual Pre-training	None
2022-03-23	AttMask	ECCV 2022	What to Hide from Your Students: Attention-Guided Masked Image Modeling	AttMask
2022-03-29	mc-BEiT	Arxiv 2022	mc-BEiT: Multi-choice Discretization for Image BERT Pre-training	None
2022-04-18	Ge2-AE	Arxiv 2022	The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training	None
2022-05-08	MCMAE	NeurIPS 2022	MCMAE: Masked Convolution Meets Masked Autoencoders	MCMAE
2022-05-20	UM-MAE	Arxiv 2022	Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Locality	UM-MAE
2022-05-26	GreenMIM	Arxiv 2022	Green Hierarchical Vision Transformer for Masked Image Modeling	GreenMIM
2022-05-26	MixMIM	Arxiv 2022	MixMIM: Mixed and Masked Image Modeling for Efficient Visual Representation Learning	Code is Opening
2022-05-28	SupMAE	Arxiv 2022	SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners	SupMAE
2022-05-30	HiViT	Arxiv 2022	HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling	None
2022-06-01	LoMaR	Arxiv 2022	Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction	LoMaR
2022-06-22	SemMAE	NeurIPS 2022	SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders	SemMAE
2022-08-11	MILAN	Arxiv 2022	MILAN: Masked Image Pretraining on Language Assisted Representation	MILAN
2022-11-14	EVA	Arxiv 2022	EVA: Exploring the Limits of Masked Visual Representation Learning at Scale	EVA
2022-11-28	AMT	Arxiv 2022	Good helper is around you: Attention-driven Masked Image Modeling	AMT
2022-03-04	PixMIM	Arxiv 2023	PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling	PixMIM

Others:

Object detection.

Date	Method	Conference	Title	Code
2022-04-06	MIMDet	Arxiv 2022	Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection	MIMDet

3D.

Date	Method	Conference	Title	Code
2021-11-29	Point-BERT	Arxiv 2021	Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling	Point-BERT
2022-03-28	Point-MAE	Arxiv 2022	Masked Autoencoders for Point Cloud Self-supervised Learning	Point-MAE

Image generation.

Date	Method	Conference	Title	Code
2022-02-08	MaskGIT	Arxiv 2022	MaskGIT: Masked Generative Image Transformer	None

Video.

Date	Method	Conference	Title	Code
2021-12-02	BEVT	Arxiv 2021	BEVT: BERT Pretraining of Video Transformers	BEVT
2022-03-23	VideoMAE	NeurIPS 2022	VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training	VideoMAE
2022-05-18	MAE_ST	NeurIPS 2022	Masked Autoencoders As Spatiotemporal Learners	MAE_ST

Multi-modal.

Date	Method	Conference	Title	Code
2022-04-04	MultiMAE	Arxiv 2022	MultiMAE: Multi-modal Multi-task Masked Autoencoders	MultiMAE
2022-05-27	M3AE	Arxiv 2022	Multimodal Masked Autoencoders Learn Transferable Representations	None
2022-08-03	xxx	Arxiv 2022	Masked Vision and Language Modeling for Multi-modal Representation Learning	None
2022-12-01	FLIP	Arxiv 2022	Scaling Language-Image Pre-training via Masking	None

Medical.

Date	Method	Conference	Title	Code
2022-03-10	MedMAE	Arxiv 2022	Self Pre-training with Masked Autoencoders for Medical Image Analysis	None

Analysis.

Date	Method	Conference	Title
2022-08-08	RelaxMIM	Arxiv 2022	Understanding Masked Image Modeling via Learning Occlusion Invariant Feature

Survey.

Date	Conference	Title
2022-07-30	Arxiv 2022	A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond