ML Papers

Table of contents

1. activation
2. active learning
3. adaptation
4. adversarial training
5. antialiasing
6. asr
7. attention
8. augmentation
9. autoregressive model
10. backbone
11. bayesian
12. bert
13. bias
14. calibration
15. causality
16. channel attention
17. chat
18. computation
19. continual learning
20. contrastive learning
21. convolution
22. dataset
23. ddpm
24. decoding
25. deep prior
26. differentiable operator
27. differentiable tree
28. discrete vae
29. disentangle
30. distillation
31. distributed training
32. domain adaptation
33. dropout
34. efficient attention
35. embedding
36. end2end
37. energy based model
38. ensemble
39. federated learning
40. few shot
41. finetuning
42. flow
43. fpn
44. gan
45. gan inversion
46. generalization
47. generative model
48. graph
49. hallucination
50. hypernetwork
51. hyperparameter
52. identifiability
53. image editing
54. image generation
55. img2img
56. implicit model
57. implicit representation
58. instance segmentation
59. interpolation
60. knowledge base
61. language generation
62. language model
63. layout
64. lightweight
65. line
66. lm
67. local attention
68. loss
69. loss surface
70. matting
71. memory
72. meta learning
73. metric learning
74. mixup
75. mlm
76. multimodal
77. multitask
78. nas
79. nerf
80. neural computer
81. neural ode
82. neural rendering
83. nlp
84. nmt
85. noise
86. non autoregressive
87. norm free
88. normalization
89. object detection
90. ocr
91. optimization
92. optimizer
93. oriented object detection
94. out of distribution
95. panoptic segmentation
96. perceptual loss
97. pooling
98. pose
99. positional encoding
100. pretraining
101. probabilistic model
102. pruning
103. qa
104. reasoning
105. regularization
106. reinforcement learning
107. rendering
108. representation
109. resampling
110. restoration
111. review
112. robustness
113. saliency
114. salient object detection
115. scale
116. score
117. self supervised
118. self supervised discovery
119. semantic factor
120. semantic segmentation
121. semi supervised learning
122. sgld
123. single image
124. speech
125. structure learning
126. style transfer
127. stylegan
128. super resolution
129. text generation
130. topic model
131. topology
132. tracking
133. training
134. transducer
135. transfer
136. transformer
137. tropical geometry
138. tts
139. unsupervised img2img
140. unsupervised nmt
141. vae
142. video
143. video transformer
144. vision
145. vision language
146. vision transformer
147. visual grounding
148. vit
149. vocoder
150. weak supervision
151. uncategorized

activation

1. 201019 Smooth activations and reproducibility in deep networks #stability

active learning

1. 200630 Similarity Search for Efficient Active Learning and Search of Rare

adaptation

1. 200129 Side-Tuning
2. 200130 Once for All #deploy

adversarial training

1. 200130 Adversarial Examples Improve Image Recognition
2. 200625 Smooth Adversarial Training

antialiasing

1. 201120 An Effective Anti-Aliasing Approach for Residual Networks
2. 201128 Truly shift-invariant convolutional neural networks

asr

1. 200220 Imputer #non-autoregressive #ctc
2. 200510 Listen Attentively, and Spell Once #non-autoregressive
3. 200516 Large scale weakly and semi-supervised learning for low-resource video ASR #weak_supervision #semi_supervised_learning
4. 200516 Reducing Spelling Inconsistencies in Code-Switching ASR using #ctc
5. 200516 Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition #non-autoregressive
6. 200518 Attention-based Transducer for Online Speech Recognition #transducer
7. 200518 Iterative Pseudo-Labeling for Speech Recognition
8. 200519 Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognition #ctc
9. 200519 Improved Noisy Student Training for Automatic Speech Recognition #semi_supervised_learning
10. 200729 Developing RNN-T Models Surpassing High-Performance Hybrid Models with #rnn_t
11. 201021 FastEmit #transducer #decoding
12. 201027 CASS-NAT #non-autoregressive
13. 201125 Streaming end-to-end multi-talker speech recognition #transducer
14. 210524 Unsupervised Speech Recognition #unsupervised_training

attention

1. 200122 Object Contextual Representations #semantic_segmentation
2. 200129 Empirical Attention
3. 200130 Axial Attention #generative_model
4. 200130 Criss-Cross Attention #semantic_segmentation
5. 200212 Capsules with Inverted Dot-Product Attention Routing #capsule
6. 200219 Tree-structured Attention with Hierarchical Accumulation #parse
7. 200226 Sparse Sinkhorn Attention #sparse_attention
8. 200317 Axial-DeepLab #panoptic_segmentation
9. 200404 Neural Architecture Search for Lightweight Non-Local Networks
10. 200421 Attention is Not Only a Weight #bert
11. 200423 Self-Attention Attribution #bert
12. 200428 Exploring Self-attention for Image Recognition
13. 200510 CTC-synchronous Training for Monotonic Attention Model #asr #ctc
14. 200516 Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory #asr #memory
15. 200519 Normalized Attention Without Probability Cage
16. 200519 Staying True to Your Word
17. 200626 Object-Centric Learning with Slot Attention
18. 201119 On the Dynamics of Training Attention Models #training
19. 210223 Linear Transformers Are Secretly Fast Weight Memory Systems #linear_attention #efficient_attention
20. 210225 LazyFormer #bert
21. 210517 Pay Attention to MLPs #mlp
22. 210524 Self-Attention Networks Can Process Bounded Hierarchical Languages #nlp

augmentation

1. 200122 FixMatch #semi_supervised_learning #manifold #mixup
2. 200220 Affinity and Diversity
3. 200621 AdvAug #mixup #nlp #adversarial_training
4. 200710 Meta-Learning Requires Meta-Augmentation #metalearning
5. 201117 Sequence-Level Mixed Sample Data Augmentation #nlp
6. 201125 Can Temporal Information Help with Contrastive Self-Supervised Learning #video #self_supervised
7. 201213 Simple Copy-Paste is a Strong Data Augmentation Method for Instance #instance_segmentation
8. 201214 Improving Panoptic Segmentation at All Scales #panoptic_segmentation
9. 210318 AlignMix #mixup
10. 210318 TrivialAugment
11. 210429 Ensembling with Deep Generative Views #ensemble #gan_inversion

autoregressive model

1. 200129 Semi Autorgressive Training
2. 201027 Scaling Laws for Autoregressive Generative Modeling #scale

backbone

1. 190724 MixNet #convolution
2. 200123 Antialiasing #invariance
3. 200128 Attentive Normalization
4. 200128 IBN-Net
5. 200128 Selective Kernel
6. 200128 SpineNet
7. 200128 Squeeze-Excitation
8. 200128 Switchable Normalization
9. 200128 Switchable Whitening
10. 200129 Assembled Techniques #regularization
11. 200129 DenseNet
12. 200129 Dual Path Networks
13. 200129 HarDNet
14. 200129 PyramidNet
15. 200129 SelecSLS
16. 200129 ShuffleNet V2 #efficiency
17. 200129 VoVNet
18. 200130 FishNet
19. 200130 HRNet
20. 200130 MixConv #convolution
21. 200330 Designing Network Design Spaces #hypernetwork
22. 200330 TResNet #antialiasing
23. 200419 ResNeSt
24. 200630 Deep Isometric Learning for Visual Recognition #normalization #resnet #cnn #norm_free
25. 200712 PSConv #cnn #multiscale
26. 201015 HS-ResNet #multiscale
27. 201221 FcaNet #channel_attention
28. 210226 Transformer in Transformer #vision_transformer
29. 210310 Involution #convolution #attention
30. 210312 Revisiting ResNets #resnet
31. 210317 Learning to Resize Images for Computer Vision Tasks #resizing
32. 210331 EfficientNetV2
33. 210408 SI-Score #robustness #vision_transformer
34. 210505 RepMLP #mlp
35. 210506 Do You Even Need Attention #mlp
36. 210510 ResMLP #mlp

bayesian

1. 200207 Bayes Posterior
2. 200210 Liberty or Depth #mean_field
3. 200220 Neural Bayes #representation #clustering
4. 200514 Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors #ensemble #variational_inference

bert

1. 200305 What the [MASK]
2. 200427 DeeBERT #lightweight
3. 200518 Audio ALBERT #audio #representation
4. 200601 Amnesic Probing
5. 200608 On the Stability of Fine-tuning BERT #finetuning
6. 200610 Revisiting Few-sample BERT Fine-tuning #finetuning

bias

1. 200519 Identifying Statistical Bias in Dataset Replication
2. 201202 Learning from others' mistakes #product_of_experts

calibration

1. 200221 Calibrating Deep Neural Networks using Focal Loss #loss
2. 200223 Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks #bayesian
3. 200620 Regression Prior Networks

causality

1. 200518 An Analysis of the Adaptation Speed of Causal Models

channel attention

1. 200129 GCNet

chat

1. 200630 PLATO-2 #text_gen #chatbot

computation

1. 200213 Training Large Neural Networks with Constant Memory using a New Execution Algorithm
2. 201204 Nimble

continual learning

1. 200508 Transforming task representations to perform novel tasks #multitask

contrastive learning

1. 200213 A Simple Framework for Contrastive Learning of Visual Representations #augmentation
2. 200309 Improved Baselines with Momentum Contrastive Learning
3. 200423 Supervised Contrastive Learning #metric_learning
4. 200511 Prototypical Contrastive Learning of Unsupervised Representations
5. 200520 What Makes for Good Views for Contrastive Learning
6. 200613 Bootstrap your own latent
7. 200630 Debiased Contrastive Learning
8. 200730 Contrastive Learning for Unpaired Image-to-Image Translation #img2img
9. 200803 LoCo
10. 201020 BYOL works even without batch statistics
11. 201109 Towards Domain-Agnostic Contrastive Learning #mixup #multimodal
12. 201116 AdCo #adversarial_training
13. 201117 Dense Contrastive Learning for Self-Supervised Visual Pre-Training
14. 201119 Heterogeneous Contrastive Learning
15. 201119 Propagate Yourself
16. 201121 Run Away From your Teacher
17. 201123 Boosting Contrastive Self-Supervised Learning with False Negative
18. 201126 Beyond Single Instance Multi-view Unsupervised Representation Learning #self_supervised #mixup
19. 201126 How Well Do Self-Supervised Models Transfer #self_supervised #transfer
20. 201127 Self-EMD
21. 201201 Towards Good Practices in Self-supervised Representation Learning #self_supervised
22. 201204 Seed the Views #mixup
23. 201212 Contrastive Learning for Label-Efficient Semantic Segmentation #semantic_segmentation
24. 201221 Online Bag-of-Visual-Words Generation for Unsupervised Representation #self_supervised #discrete_vae
25. 201226 Spatial Contrastive Learning for Few-Shot Classification #few_shot #attention
26. 210304 Barlow Twins #self_supervised #backbone
27. 210325 Rethinking Self-Supervised Learning #training
28. 210405 An Empirical Study of Training Self-Supervised Vision Transformers #vision_transformer
29. 210426 Multimodal Contrastive Training for Visual Representation Learning #multimodal
30. 210429 A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning #video
31. 210429 Emerging Properties in Self-Supervised Vision Transformers #saliency #vision_transformer #representation
32. 210429 With a Little Help from My Friends #knn
33. 210510 Self-Supervised Learning with Swin Transformers #vision_transformer
34. 210511 VICReg
35. 210517 Divide and Contrast #self_supervised #dataset #distillation
36. 210601 Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task

convolution

1. 200316 SlimConv
2. 210429 Decoupled Dynamic Filter Networks

dataset

1. 200509 Building a Manga Dataset
2. 201130 Image Quality Assessment for Perceptual Image Restoration #score
3. 201201 Weakly-Supervised Arbitrary-Shaped Text Detection with #ocr #weak_supervision
4. 210601 Comparing Test Sets with Item Response Theory

ddpm

1. 200619 Denoising Diffusion Probabilistic Models
2. 201214 Learning Energy-Based Models by Diffusion Recovery Likelihood #energy_based_model
3. 210506 DiffSinger #singing_voice_synthesis
4. 210511 Diffusion Models Beat GANs on Image Synthesis
5. 210528 Gotta Go Fast When Generating Data with Score-Based Models
6. 210531 On Fast Sampling of Diffusion Probabilistic Models

decoding

1. 200516 Layer-Wise Cross-View Decoding for Sequence-to-Sequence Learning

deep prior

1. 200408 Deep Manifold Prior

differentiable operator

1. 200220 Fast Differentiable Sorting and Ranking

differentiable tree

1. 200218 The Tree Ensemble Layer

discrete vae

1. 200518 Robust Training of Vector Quantized Bottleneck Models

disentangle

1. 200130 ID-GAN #GAN
2. 200130 MixNMatch #conditional_generative_model
3. 200515 Face Identity Disentanglement via Latent Space Mapping

distillation

1. 200129 Learning by Cheating
2. 200209 Understanding and Improving Knowledge Distillation
3. 200210 Subclass Distillation
4. 200219 Knapsack Pruning with Inner Distillation #pruning #lightweight
5. 200221 Residual Knowledge Distillation
6. 200309 Knowledge distillation via adaptive instance normalization #normalization
7. 200405 FastBERT #bert #lightweight
8. 200408 DynaBERT #bert #pruning
9. 200408 Improving BERT with Self-Supervised Attention #bert #self_supervised
10. 200412 XtremeDistil #bert #lightweight
11. 200521 Why distillation helps #calibration
12. 200629 An EM Approach to Non-autoregressive Conditional Sequence Generation #non-autoregressive
13. 200701 Go Wide, Then Narrow #lightweight
14. 200702 Interactive Knowledge Distillation

distributed training

1. 210510 GSPMD

domain adaptation

1. 200526 Keep it Simple

dropout

1. 200701 On Dropout, Overfitting, and Interaction Effects in Deep Neural Networks

efficient attention

1. 200410 Longformer
2. 200412 ProFormer
3. 200605 Masked Language Modeling for Proteins via Linearly Scalable Long-Context
4. 200608 Linformer
5. 210324 Finetuning Pretrained Transformers into RNNs
6. 210505 Beyond Self-attention
7. 210510 Poolingformer
8. 210603 Luna

embedding

1. 200424 All Word Embeddings from One Embedding
2. 200717 A Unifying Perspective on Neighbor Embeddings along the

end2end

1. 200605 End-to-End Adversarial Text-to-Speech #tts
2. 200608 FastSpeech 2 #tts
3. 201106 Wave-Tacotron #tts

energy based model

1. 200504 How to Train Your Energy-Based Model for Regression
2. 201124 Energy-Based Models for Continual Learning #continual_learning

ensemble

1. 200217 BatchEnsemble

federated learning

1. 210415 See through Gradients

few shot

1. 200228 AdarGCN #graph
2. 210524 True Few-Shot Learning with Language Models #lm

finetuning

1. 200214 AutoLR #pruning
2. 200426 Masking as an Efficient Alternative to Finetuning for Pretrained
3. 200709 Sample-based Regularization #transfer

flow

1. 200220 Regularized Autoencoders via Relaxed Injective Probability Flow
2. 200227 Woodbury Transformations for Deep Generative Flows

fpn

1. 200122 CARAFE #resampling
2. 200129 Mixture FPN
3. 200506 Scale-Equalizing Pyramid Convolution for Object Detection
4. 201201 Dynamic Feature Pyramid Networks for Object Detection
5. 201202 Dual Refinement Feature Pyramid Networks for Object Detection
6. 201202 Parallel Residual Bi-Fusion Feature Pyramid Network for Accurate
7. 201225 Implicit Feature Pyramid Network for Object Detection #equilibrium_model #implicit_model

gan

1. 170629 Do GANs actually learn the distribution
2. 191022 MelGAN #tts
3. 200129 Adversarial Lipschitz Regularization
4. 200129 GAN generalization metric
5. 200129 OneGAN
6. 200130 AttentionGAN #attention #img2img
7. 200130 Evaluation metrics of GAN #metric #evaluation #generative_model
8. 200130 Local GAN #attention
9. 200130 Noise Robust GAN #robustness
10. 200130 Small-GAN
11. 200130 Smoothness and Stability in GANs
12. 200206 Unbalanced GANs #vae
13. 200210 Unsupervised Discovery of Interpretable Directions in the GAN Latent #semantic_factor
14. 200211 Improved Consistency Regularization for GANs #augmentation #consistency_regularization
15. 200211 Smoothness and Stability in GANs #regularization
16. 200212 Image-to-Image Translation with Text Guidance #multimodal #multimodal_generation #img2img
17. 200212 Real or Not Real, that is the Question
18. 200214 Top-k Training of GANs #regularization
19. 200220 The Benefits of Pairwise Discriminators for Adversarial Training #regularization
20. 200223 GANHopper #img2img
21. 200224 When Relation Networks meet GANs #regularization
22. 200225 Freeze the Discriminator #finetuning #transfer
23. 200226 On Leveraging Pretrained GANs for Generation with Limited Data #finetuning #transfer
24. 200227 Topology Distance #topology #score
25. 200228 A U-Net Based Discriminator for Generative Adversarial Networks
26. 200304 Creating High Resolution Images with a Latent Adversarial Generator #generative_model #super_resolution
27. 200308 Perceptual Image Super-Resolution with Progressive Adversarial Network #super_resolution
28. 200312 Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling #energy_based_model #sampling
29. 200317 Blur, Noise, and Compression Robust Generative Adversarial Networks #noise
30. 200318 OpenGAN #metric_learning
31. 200325 Improved Techniques for Training Single-Image GANs #single_image
32. 200326 Image Generation Via Minimizing Fréchet Distance in Discriminator Feature Space
33. 200402 Controllable Orthogonalization in Training DNNs #regularization
34. 200404 Feature Quantization Improves GAN Training #discrete_vae
35. 200405 Discriminator Contrastive Divergence
36. 200407 Inclusive GAN
37. 200408 Attentive Normalization for Conditional Image Generation #attention
38. 200504 Transforming and Projecting Images into Class-conditional Generative #generative_model
39. 200518 Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization #audio_generation
40. 200519 CIAGAN
41. 200519 Regularization Methods for Generative Adversarial Networks #review #regularization
42. 200604 Image Augmentations for GAN Training #augmentation
43. 200611 Training Generative Adversarial Networks with Limited Data #augmentation
44. 200618 Differentiable Augmentation for Data-Efficient GAN Training #augmentation
45. 200618 Diverse Image Generation via Self-Conditioned GANs #generative_model
46. 200630 PriorGAN
47. 200708 InfoMax-GAN #regularization
48. 200713 Closed-Form Factorization of Latent Semantics in GANs #semantic_factor
49. 200729 Instance Selection for GANs
50. 200729 VocGAN #vocoder
51. 200730 Rewriting a Deep Generative Model
52. 200804 Open-Edit #image_editing
53. 200807 Improving the Speed and Quality of GAN by Adversarial Training #robustness
54. 201028 Training Generative Adversarial Networks by Solving Ordinary #neural_ode
55. 201109 Learning Semantic-aware Normalization for Generative Adversarial Networks #normalization
56. 201109 Towards a Better Global Loss Landscape of GANs #training
57. 201118 Style Intervention #semantic_factor
58. 201124 Adversarial Generation of Continuous Images #implicit_representation
59. 201125 How to train your conditional GAN #img2img #generative_model
60. 201125 Omni-GAN #generative_model
61. 201127 Image Generators with Conditionally-Independent Pixel Synthesis #implicit_representation
62. 201201 Refining Deep Generative Models via Discriminator Gradient Flow #sampling
63. 201201 pi-GAN #implicit_representation
64. 201203 Self-labeled Conditional GANs #unsupervised_training
65. 201204 A Note on Data Biases in Generative Models #bias #generative_model
66. 201208 You Only Need Adversarial Supervision for Semantic Image Synthesis #img2img
67. 210227 Ultra-Data-Efficient GAN Training #augmentation #few_shot
68. 210317 Training GANs with Stronger Augmentations via Contrastive Discriminator #contrastive_learning #augmentation
69. 210318 Drop the GAN #single_image #generative_model #patch
70. 210330 Dual Contrastive Loss and Attention for GANs #contrastive_learning
71. 210401 Partition-Guided GANs
72. 210407 Regularizing Generative Adversarial Networks under Limited Data #regularization
73. 210408 InfinityGAN
74. 210413 DatasetGAN #few_shot
75. 210413 Few-shot Image Generation via Cross-domain Correspondence #img2img #generative_model #few_shot
76. 210414 Aligning Latent and Image Spaces to Connect the Unconnectable
77. 210415 GANcraft #nerf
78. 210422 On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation #antialiasing
79. 210426 EigenGAN #semantic_factor

gan inversion

1. 200331 In-Domain GAN Inversion for Real Image Editing
2. 200703 Collaborative Learning for Faster StyleGAN Embedding

generalization

1. 200130 Fantastic Generalization Measures
2. 200225 Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

generative model

1. 190325 Implicit Generative and Generalization in Energy-Based Models #energy_based_model
2. 200129 Controlling Generative Model
3. 200129 Deep Automodulator
4. 200129 Frechet Joint Distance
5. 200129 Spot CNN generated image
6. 200130 BIVA
7. 200130 Glow #flow
8. 200130 IGEBM #energy_based_model
9. 200130 Neural Spline Flows #flow
10. 200130 VQ-VAE-2 #autoregressive_model
11. 200217 Augmented Normalizing Flows #flow
12. 200313 Semantic Pyramid for Image Generation #perceptual_loss #image_editing
13. 200616 Improved Techniques for Training Score-Based Generative Models #ncsn
14. 201117 DeepNAG
15. 201126 Score-Based Generative Modeling through Stochastic Differential #ddpm
16. 201202 Improved Contrastive Divergence Training of Energy Based Models #energy_based_model
17. 201204 Few-shot Image Generation with Elastic Weight Consolidation #few_shot #continual_learning
18. 201209 Positional Encoding as Spatial Inductive Bias in GANs #positional_encoding
19. 201224 Soft-IntroVAE #vae
20. 210223 Zero-Shot Text-to-Image Generation #discrete_vae #autoregressive_model #multimodal
21. 210302 Fixing Data Augmentation to Improve Adversarial Robustness #ddpm #augmentation
22. 210305 Fixing Data Augmentation to Improve Adversarial Robustness 2 #robustness #augmentation #ddpm
23. 210318 Few-shot Semantic Image Synthesis Using StyleGAN Prior #stylegan #few_shot

graph

1. 200129 Multi-Graph Transformer

hallucination

1. 210413 The Curious Case of Hallucinations in Neural Machine Translation #mt

hypernetwork

1. 200722 WeightNet #channel_attention

hyperparameter

1. 200425 Learning to Guide Random Search
2. 200521 HyperSTAR

identifiability

1. 200701 On Linear Identifiability of Learned Representations

image editing

1. 200515 Semantic Photo Manipulation with a Generative Image Prior
2. 201123 HistoGAN
3. 210318 Using latent space regression to analyze and leverage compositionality

image generation

1. 200426 Disentangled Image Generation Through Structured Noise Injection

img2img

1. 200130 FUNIT
2. 200305 SketchyCOCO
3. 200315 GMM-UNIT #multimodal_generation
4. 200319 High-Resolution Daytime Translation Without Domain Labels
5. 200330 Semi-supervised Learning for Few-shot Image-to-Image Translation #semi_supervised_learning #few_shot
6. 200406 Rethinking Spatially-Adaptive Normalization #lightweight
7. 200409 TuiGAN #few_shot #single_image
8. 200419 TriGAN #domain_adaptation
9. 200702 Deep Single Image Manipulation #single_image #image_editing
10. 200709 Improving Style-Content Disentanglement in Image-to-Image Translation #disentangle
11. 200714 COCO-FUNIT
12. 200715 Transformation Consistency Regularization- A Semi-Supervised Paradigm #augmentation #semi_supervised_learning
13. 200723 TSIT
14. 200724 The Surprising Effectiveness of Linear Unsupervised Image-to-Image
15. 201203 CoCosNet v2 #patch #pose
16. 201205 Spatially-Adaptive Pixelwise Networks for Fast Image Translation #implicit_representation

implicit model

1. 200615 Multiscale Deep Equilibrium Models

implicit representation

1. 210506 ACORN #positional_encoding

instance segmentation

1. 200129 BlendMask
2. 200129 COCO 2018 Instance Segmentation #challenge
3. 200129 Deep Snake
4. 200130 PointRend
5. 200311 Conditional Convolutions for Instance Segmentation
6. 200313 PointINS #dynamic_conv
7. 200722 Deep Variational Instance Segmentation
8. 200730 LevelSet R-CNN
9. 201119 DCT-Mask
10. 201119 Unifying Instance and Panoptic Segmentation with Dynamic Rank-1 #panoptic_segmentation #dynamic_conv
11. 201126 The Devil is in the Boundary
12. 201129 End-to-End Video Instance Segmentation with Transformers #end2end #detr #video
13. 201203 BoxInst #dataset #weak_supervision
14. 210503 ISTR #end2end
15. 210505 QueryInst #end2end

interpolation

1. 200804 Autoencoder Image Interpolation by Shaping the Latent Space

knowledge base

1. 200214 Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base

language generation

1. 200424 Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order #mlm
2. 200712 Do You Have the Right Scissors
3. 200729 Mirostat

language model

1. 200128 Scaling Laws for LM
2. 200205 K-Adapter #multitask #adapter
3. 200206 Consistency of a Recurrent Language Model With Respect to Incomplete #decoding #hallucination #language_generation
4. 200222 Training Question Answering Models From Synthetic Data #qa #bert
5. 200225 MiniLM #distillation #lightweight
6. 200406 Sparse Text Generation #language_generation #sampling
7. 200427 Recall and Learn #finetuning #continual_learning
8. 200505 Stolen Probability
9. 200516 MicroNet for Efficient Language Modeling #lightweight
10. 200518 Contextual Embeddings
11. 201015 Fine-Tuning Pre-trained Language Model with Weak Supervision #transfer #weak_supervision
12. 201023 Rethinking embedding coupling in pre-trained language models #regularization
13. 201201 How Can We Know When Language Models Know #qa #calibration
14. 201228 Universal Sentence Representation Learning with Conditional Masked #sentence_embedding #mlm
15. 210216 Non-Autoregressive Text Generation with Pre-trained Language Models #non-autoregressive #text_generation
16. 210318 GPT Understands, Too #finetuning #prompt
17. 210407 Revisiting Simple Neural Probabilistic Language Models
18. 210420 Carbon Emissions and Large Neural Network Training #nlp

layout

1. 210601 Incorporating Visual Layout Structures for Scientific Text Classification

lightweight

1. 200624 Neural Architecture Design for GPU-Efficient Networks
2. 201124 MicroNet
3. 210507 Pareto-Optimal Quantized ResNet Is Mostly 4-bit #quantization

line

1. 210601 Towards Real-time and Light-weight Line Segment Detection

lm

1. 210524 StructuralLM #layout
2. 210528 ByT5

local attention

1. 210323 Scaling Local Self-Attention for Parameter Efficient Visual Backbones

loss

1. 200712 It Is Likely That Your Loss Should be a Likelihood

loss surface

1. 210225 Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling

matting

1. 200401 Background Matting
2. 201123 Is a Green Screen Really Necessary for Real-Time Portrait Matting

memory

1. 200206 Product Kanerva Machines

meta learning

1. 200221 Learning to Continually Learn #continual_learning
2. 200312 Online Fast Adaptation and Knowledge Accumulation
3. 200401 Editable Neural Networks
4. 200402 Tracking by Instance Detection #tracking
5. 200706 Meta-Learning Symmetries by Reparameterization #group_equivariance

metric learning

1. 200319 A unifying mutual information view of metric learning

mixup

1. 201220 ResizeMix

mlm

1. 210502 Larger-Scale Transformers for Multilingual Masked Language Modeling #multilingual #scale

multimodal

1. 200401 Pixel-BERT
2. 200513 INFOTABS
3. 200514 Behind the Scene
4. 201130 Multimodal Pretraining Unmasked

multitask

1. 200625 MTAdam

nas

1. 200324 BigNAS
2. 200326 Are Labels Necessary for Neural Architecture Search #unsupervised_training
3. 200406 Network Adjustment
4. 200412 FBNetV2
5. 200428 Angle-based Search Space Shrinking for Neural Architecture Search
6. 200506 Local Search is State of the Art for Neural Architecture Search
7. 200507 Noisy Differentiable Architecture Search
8. 200512 Neural Architecture Transfer #transfer
9. 200602 FBNetV3 #hyperparameter #training #swa
10. 200720 NSGANetV2

nerf

1. 201014 NeRF++
2. 201125 Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
3. 201127 D-NeRF
4. 201203 Learned Initializations for Optimizing Coordinate-Based Neural #implicit_representation
5. 201203 pixelNeRF
6. 201215 Object-Centric Neural Scene Rendering
7. 210225 IBRNet
8. 210318 FastNeRF
9. 210318 GNeRF
10. 210318 MVSNeRF
11. 210318 NeMI
12. 210324 Mip-NeRF
13. 210325 KiloNeRF
14. 210325 PlenOctrees for Real-time Rendering of Neural Radiance Fields

neural computer

1. 200720 Distributed Associative Memory Network with Memory Refreshing Loss

neural ode

1. 200207 How to train your neural ODE
2. 200520 Neural Controlled Differential Equations
3. 200708 Learning Differential Equations that are Easy to Solve

neural rendering

1. 200226 Learning to Shadow Hand-drawn Sketches
2. 200427 Neural Hair Rendering
3. 200506 CONFIG
4. 201116 Stylized Neural Painting
5. 201119 Creative Sketch Generation
6. 201130 Animating Pictures with Eulerian Motion Fields #single_image
7. 210319 Paint by Word
8. 210512 Enhancing Photorealism Enhancement

nlp

1. 200129 Meena #dialog
2. 200518 (Re)construing Meaning in NLP
3. 200715 Towards Debiasing Sentence Representations #bias
4. 201117 Neural Semi-supervised Learning for Text Classification Under #self_supervised

nmt

1. 200207 A Multilingual View of Unsupervised Machine Translation #multilingual
2. 200427 Lexically Constrained Neural Machine Translation with Levenshtein Transformer
3. 200710 Learn to Use Future Information in Simultaneous Translation #simultaneous_translation
4. 201224 Why Neural Machine Translation Prefers Empty Outputs #hallucination

noise

1. 201223 Noisy Labels Can Induce Good Representations #representation

non autoregressive

1. 200403 Aligned Cross Entropy for Non-Autoregressive Machine Translation
2. 200415 Non-Autoregressive Machine Translation with Latent Alignments #nmt #ctc
3. 200422 A Study of Non-autoregressive Model for Sequence Generation
4. 201022 Parallel Tacotron #vae
5. 201025 Improved Mask-CTC for Non-Autoregressive End-to-End ASR #ctc
6. 201125 FBWave #vocoder #lightweight
7. 201207 EfficientTTS #tts

norm free

1. 200310 ReZero is All You Need #initialization

normalization

1. 200122 Group Norm, Weight Standardization
2. 200122 Moving Average Batch Normalization
3. 200122 StyleGAN 2 #GAN
4. 200130 Rethinking Normalization
5. 200130 Weight Standardization #weight
6. 200224 Batch Normalization Biases Residual Blocks Towards the Identity Function #optimization #norm_free #initialization
7. 200306 TaskNorm #meta_learning
8. 200406 Evolving Normalization-Activation Layers #nas #activation
9. 200427 A Batch Normalized Inference Network Keeps the KL Vanishing Away
10. 201128 Batch Normalization with Enhanced Linear Transformation

object detection

1. 191118 Anchor-Free
2. 191118 CenterMask #instance_segmentation #backbone #1stage
3. 191121 EfficientDet
4. 200103 BlendMask #instance_segmentation #1stage
5. 200122 SABL
6. 200129 AP Loss #loss
7. 200129 Backbone Reallocation for Detection #backbone #nas
8. 200129 Dense RepPoints
9. 200129 DetNAS #nas #backbone
10. 200129 IOU-aware single stage detector #1stage
11. 200130 ATSS #anchor #retinanet #fcos
12. 200130 AutoAugment #augmentation #search
13. 200130 EfficientDet #fpn
14. 200130 Keypoint Triplet #keypoint
15. 200130 Learning from Noisy Anchors
16. 200130 Multiple Anchor Learning #anchor
17. 200130 Objects as Points #keypoint
18. 200130 Soft Anchor-Point #anchor
19. 200211 Object Detection as a Positive-Unlabeled Problem #positive_unlabled #dataset
20. 200212 Solving Missing-Annotation Object Detection with Background #dataset #noise
21. 200218 Universal-RCNN #multi_dataset #graph
22. 200316 Frustratingly Simple Few-Shot Object Detection #few_shot
23. 200317 Revisiting the Sibling Head in Object Detector
24. 200319 Revisiting the Sibling Head in Object Detector #review
25. 200320 CentripetalNet #keypoint
26. 200413 Dynamic R-CNN
27. 200423 YOLOv4
28. 200511 Scope Head for Accurate Localization in Object Detection
29. 200526 End-to-End Object Detection with Transformers #end2end #matching
30. 200603 DetectoRS
31. 200611 Rethinking Pre-training and Self-training #semi_supervised_learning #transfer
32. 200706 LabelEnc #distillation
33. 200707 AutoAssign #anchor_free
34. 200714 AQD #quantization
35. 200715 Probabilistic Anchor Assignment with IoU Prediction for Object Detection #anchor #1stage
36. 200716 RepPoints V2 #1stage #anchor_free
37. 200723 PP-YOLO #tuning
38. 200723 The Devil is in Classification #longtail
39. 200727 Corner Proposal Network for Anchor-free, Two-stage Object Detection #anchor_free #2stage
40. 201116 Scaled-YOLOv4
41. 201117 UP-DETR #detr #end2end #pretraining
42. 201118 End-to-End Object Detection with Adaptive Clustering Transformer #detr #end2end #efficiency
43. 201121 Rethinking Transformer-based Set Prediction for Object Detection #detr #end2end #efficiency
44. 201124 Sparse R-CNN
45. 201128 Class-agnostic Object Detection
46. 201207 End-to-End Object Detection with Fully Convolutional Network #end2end
47. 201223 SWA Object Detection #swa
48. 201227 Towards A Category-extended Object Detector without Relabeling or #continual_learning
49. 210225 Simple multi-dataset detection #multi_dataset
50. 210316 You Only Look One-level Feature
51. 210325 USB #dataset
52. 210417 TransVG #visual_grounding
53. 210420 PP-YOLOv2 #yolo
54. 210426 MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding #detr #visual_grounding
55. 210601 You Only Look at One Sequence #vit

ocr

1. 191231 LayoutLM
2. 200217 Text Perceptron
3. 210415 Rethinking Text Line Recognition Models

optimization

1. 200221 The Break-Even Point on Optimization Trajectories of Deep Neural Networks #loss #training
2. 200224 The Early Phase of Neural Network Training
3. 200227 Using a thousand optimization tasks to learn hyperparameter search strategies #optimizer #hyperparameter
4. 200228 A Self-Tuning Actor-Critic Algorithm #reinforcement_learning #hyperparameter #meta_learning
5. 200316 Weak and Strong Gradient Directions
6. 200403 Gradient Centralization #training
7. 200508 An Investigation of Why Overparameterization Exacerbates Spurious #training
8. 200519 One Size Fits All

optimizer

1. 200130 LAMB #large_batch

oriented object detection

1. 200129 Modulated Loss
2. 200129 Oriented Objects as Middle Lines

out of distribution

1. 200509 Generalizing Outside the Training Set
2. 200519 Bridging the Gap Between Training and Inference for Spatio-Temporal Forecasting

panoptic segmentation

1. 200129 Bridge gap of traininfer Panoptic Segmentation
2. 200130 Panoptic-DeepLab
3. 200218 Towards Bounding-Box Free Panoptic Segmentation #box_free
4. 200404 Pixel Consensus Voting for Panoptic Segmentation
5. 200421 Panoptic-based Image Synthesis #neural_rendering
6. 201123 Scaling Wide Residual Networks for Panoptic Segmentation #scale
7. 201201 Fully Convolutional Networks for Panoptic Segmentation #dynamic_conv
8. 201201 MaX-DeepLab #detr #end2end
9. 201202 Single-shot Path Integrated Panoptic Segmentation #dynamic_conv

perceptual loss

1. 200206 Image Fine-grained Inpainting #inpainting
2. 200330 Exploiting Deep Generative Prior for Versatile Image Restoration and #gan_inversion
3. 200515 Enhancing Perceptual Loss with Adversarial Feature Matching for Super-Resolution
4. 200626 A Loss Function for Generative Neural Networks Based on Watson's
5. 201223 Focal Frequency Loss for Image Reconstruction and Synthesis #loss

pooling

1. 200325 What Deep CNNs Benefit from Global Covariance Pooling
2. 200330 Strip Pooling

pose

1. 200729 Unselfie #inpainting

positional encoding

1. 200628 Rethinking Positional Encoding in Language Pre-training
2. 210408 Modulated Periodic Activations for Generalizable Local Functional #periodic_activation #implicit_representation

pretraining

1. 190620 XLNet #language_model
2. 190729 RoBERTa #language_model
3. 200128 mBART #machine_translation #nlp
4. 200129 ImageBERT #multimodal
5. 200129 LM Pretraining #nlp
6. 200129 oLMpics #language_model #nlp
7. 200130 RoBERTa #language_model #nlp #transformer
8. 200130 T5 #nlp #transformer #seq2seq
9. 200130 ViLBERT #multimodal
10. 200210 Pre-training Tasks for Embedding-based Large-scale Retrieval #retrieval
11. 200217 Incorporating BERT into Neural Machine Translation #language_model #bert #nmt
12. 200219 CodeBERT #bert
13. 200228 UniLMv2 #language_model
14. 200317 Calibration of Pre-trained Transformers #calibration
15. 200405 Unsupervised Domain Clusters in Pretrained Language Models #domain
16. 200412 Pre-training Text Representations as Meta Learning #meta_learning #finetuning
17. 200413 Pretrained Transformers Improve Out-of-Distribution Robustness #out_of_distribution
18. 200419 Are we pretraining it right #multimodal
19. 200420 Adversarial Training for Large Neural Language Models #adversarial_training #language_model #finetuning
20. 200420 MPNet #language_model
21. 200423 Don't Stop Pretraining #domain
22. 200427 LightPAFF #distillation #finetuning
23. 200520 Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models #contrastive_learning #sentence_embedding
24. 200610 MC-BERT
25. 200615 To Pretrain or Not to Pretrain #nlp #finetuning
26. 200626 Pre-training via Paraphrasing #retrieval
27. 200703 Language-agnostic BERT Sentence Embedding #embedding #multilingual
28. 200713 An Empirical Study on Robustness to Spurious Correlations using #nlp #multitask
29. 200715 InfoXLM #nlp #cross_lingual
30. 200804 Taking Notes on the Fly Helps BERT Pre-training #nlp
31. 201020 Pushing the Limits of Semi-Supervised Learning for Automatic Speech #semi_supervised_learning #asr
32. 201021 Self-training and Pre-training are Complementary for Speech Recognition #self_supervised #asr
33. 201022 mT5 #language_model #multilingual
34. 201109 When Do You Need Billions of Words of Pretraining Data #language_model
35. 201127 Progressively Stacking 2.0 #efficiency
36. 201201 Pre-Trained Image Processing Transformer #contrastive_learning #vision_transformer #restoration
37. 201201 StructFormer #parse #attention #mlm
38. 201227 Syntax-Enhanced Pre-trained Model #language_model #syntax
39. 210225 SparseBERT #attention #sparse_attention #bert
40. 210318 All NLP Tasks Are Generation Tasks #language_model
41. 210324 Can Vision Transformers Learn without Natural Images #vision_transformer
42. 210402 Robust wav2vec 2.0 #asr
43. 210407 Pushing the Limits of Non-Autoregressive Speech Recognition #non-autoregressive #asr #ctc
44. 210413 Masked Language Modeling and the Distributional Hypothesis #language_model #mlm
45. 210417 mT6 #language_model
46. 210418 Data-Efficient Language-Supervised Zero-Shot Learning with #multimodal
47. 210422 ImageNet-21K Pretraining for the Masses #backbone
48. 210510 Are Pre-trained Convolutions Better than Pre-trained Transformers #nlp #convolution #transformer

probabilistic model

1. 200413 Einsum Networks
2. 200419 Roundtrip

pruning

1. 200130 Rethinking Pruning
2. 200218 Picking Winning Tickets Before Training by Preserving Gradient Flow #lottery_ticket
3. 200224 HRank #rank
4. 200305 Comparing Rewinding and Fine-tuning in Neural Network Pruning
5. 200424 Convolution-Weight-Distribution Assumption
6. 200514 Bayesian Bits #quantization #variational_inference
7. 200515 Movement Pruning
8. 200518 Joint Multi-Dimension Pruning
9. 200706 Lossless CNN Channel Pruning via Decoupling Remembering and Forgetting
10. 200710 To Filter Prune, or to Layer Prune, That Is The Question

qa

1. 200222 Unsupervised Question Decomposition for Question Answering

reasoning

1. 200129 Neural Arithmetic Units
2. 200409 Injecting Numerical Reasoning Skills into Language Models

regularization

1. 200130 DropAttention #dropout
2. 200219 Revisiting Training Strategies and Generalization Performance in Deep #metric_learning
3. 200225 On Feature Normalization and Data Augmentation #normalization #mixup
4. 200228 The Implicit and Explicit Regularization Effects of Dropout #dropout
5. 200331 Regularizing Class-wise Predictions via Self-knowledge Distillation #distillation #consistency_regularization
6. 200409 Orthogonal Over-Parameterized Training
7. 200424 Dropout as an Implicit Gating Mechanism For Continual Learning
8. 200427 Scheduled DropHead
9. 200506 RNN-T Models Fail to Generalize to Out-of-Domain Audio #transducer #out_of_distribution #domain #asr
10. 200513 Implicit Regularization in Deep Learning May Not Be Explainable by Norms #training #optimization
11. 200707 RIFLE #finetuning
12. 200707 Remix #imbalanced
13. 200721 Improving compute efficacy frontiers with SliceOut #efficient_training
14. 201122 Stable Weight Decay Regularization
15. 210603 When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentations #vit

reinforcement learning

1. 191120 Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
2. 200130 Mastering Atari, Go, Chess, Shogi
3. 200626 Critic Regularized Regression

rendering

1. 200130 Textured Neural Avatars

representation

1. 200412 Gradients as Features for Deep Representation Learning

resampling

1. 200512 Invertible Image Rescaling

restoration

1. 200402 Learning to See Through Obstructions
2. 200404 Deblurring by Realistic Blurring
3. 200406 Self-Supervised Scene De-occlusion
4. 200420 Bringing Old Photos Back to Life #vae
5. 201123 Cross-Camera Convolutional Color Constancy
6. 201123 Dissecting Image Crops

review

1. 191210 Thoughts on recent papers
2. 200130 Filter Response Normalization
3. 200227 A Primer in BERTology #bert
4. 200306 What is the State of Neural Network Pruning #pruning
5. 200311 Improved Baselines with Momentum Contrastive Learning #contrastive_learning
6. 200318 A Metric Learning Reality Check #metric_learning
7. 200323 Thoughts on recent papers
8. 200324 A Systematic Evaluation
9. 200325 Rethinking Few-Shot Image Classification #meta_learning
10. 200326 Thoughts on recent papers
11. 200403 Thoughts on recent papers
12. 200408 State of the Art on Neural Rendering #neural_rendering
13. 200409 EvoNorm
14. 200411 Thoughts on recent papers
15. 200428 Showing Your Work Doesn't Always Work
16. 200619 Augmentation for GANs
17. 200627 Denoising Diffusion Probabilistic Models Implementation
18. 200708 Thoughts on recent papers
19. 200717 Semantic factor of GANs
20. 200717 Thoughts on recent papers
21. 200725 Neighbor Embedding
22. 200726 Thoughts on recent papers
23. 200802 Thoughts on recent papers
24. 200821 Virtual Try On
25. 201016 Representation Learning via Invariant Causal Mechanisms
26. 201021 BYOL works even without batch statistics
27. 201108 Long Range Arena #attention #efficient_attention
28. 201112 Learning Semantic-aware Normalization for Generative Adversarial Networks
29. 201112 When Do You Need Billions of Words of Pretraining Data
30. 201118 Thoughts on recent papers
31. 201120 Thoughts on recent papers
32. 201125 Thoughts on recent papers
33. 201126 Thoughts on recent papers 1
34. 201126 Thoughts on recent papers 2
35. 201204 Thoughts on recent papers
36. 210121 Thoughts on recent papers
37. 210227 Thoughts on recent papers
38. 210305 Thoughts on recent papers
39. 210319 Thoughts on recent papers
40. 210323 Thoughts on recent papers
41. 210324 A Broad Study on the Transferability of Visual Representations with Contrastive Learning #contrastive_learning
42. 210325 Contrasting Contrastive Self-Supervised Representation Learning Models #contrastive_learning
43. 210326 Thoughts on recent papers
44. 210403 Thoughts on recent papers
45. 210412 Thoughts on recent papers
46. 210424 Thoughts on recent papers
47. 210429 Thoughts on recent papers
48. 210430 Thoughts on recent papers 1
49. 210430 Thoughts on recent papers 2
50. 210505 Thoughts on recent papers
51. 210508 Thoughts on recent papers
52. 210512 When Does Contrastive Visual Representation Learning Work #contrastive_learning #self_supervised #transfer

robustness

1. 200211 Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial #adversarial_training
2. 200304 A Closer Look at Accuracy vs. Robustness #adversarial_training
3. 200810 Informative Dropout for Robust Representation Learning
4. 210521 Intriguing Properties of Vision Transformers #vit

saliency

1. 200406 There and Back Again

salient object detection

1. 200518 U$^2$-Net

scale

1. 200712 Learning to Learn Parameterized Classification Networks for Scalable #hypernetwork
2. 201130 Towards Better Accuracy-efficiency Trade-offs

score

1. 200319 GIQA
2. 200426 Evaluation Metrics for Conditional Image Generation

self supervised

1. 200213 Automatically Discovering and Learning New Visual Categories with Ranking Statistics #weak_supervision
2. 200218 MAST #tracking
3. 200224 Self-Adaptive Training #noise #dataset
4. 200722 CrossTransformers #few_shot
5. 201015 Representation Learning via Invariant Causal Mechanisms #causality
6. 201224 Self-supervised Pre-training with Hard Examples Improves Visual #mixup

self supervised discovery

1. 200403 Self-Supervised Viewpoint Learning From Image Collections #viewpoint
2. 201127 Unsupervised part representation by Flow Capsules
3. 210429 MarioNette

semantic factor

1. 200307 StyleGAN2 Distillation for Feed-forward Image Manipulation #stylegan
2. 200308 PULSE #stylegan
3. 200406 GANSpace
4. 201127 Navigating the GAN Parameter Space for Semantic Image Editing #image_editing
5. 201222 Time-Travel Rephotography #restoration #stylegan

semantic segmentation

1. 200323 Learning Dynamic Routing for Semantic Segmentation
2. 200516 Single-Stage Semantic Segmentation from Image Labels
3. 200826 EfficientFCN
4. 210512 Segmenter

semi supervised learning

1. 200218 DivideMix #mixup #noise #dataset
2. 200306 Semi-Supervised StyleGAN for Disentanglement Learning #stylegan #mixup
3. 200323 Meta Pseudo Labels #meta_learning
4. 200627 Laplacian Regularized Few-Shot Learning #few_shot
5. 200724 Deep Co-Training with Task Decomposition for Semi-Supervised Domain #domain_adaptation
6. 201116 On the Marginal Benefit of Active Learning #active_learning #unsupervised_training
7. 201118 FROST

sgld

1. 200706 Kernel Stein Generative Modeling #svgd

single image

1. 200405 Structural-analogy from a Single Image Pair

speech

1. 200129 Speech Recognition
2. 200129 WaveFlow #conditional_generative_model

structure learning

1. 200518 Large-scale empirical validation of Bayesian Network structure learning

style transfer

1. 200318 A Content Transformation Block For Image Style Transfer
2. 200324 Deformable Style Transfer
3. 200710 Geometric Style Transfer

stylegan

1. 200803 Encoding in Style #gan_inversion
2. 210318 Labels4Free #unsupervised_segmentation

super resolution

1. 200129 ESRGAN+
2. 200323 Deep Unfolding Network for Image Super-Resolution

text generation

1. 200130 Unlikelihood Training
2. 200601 Cascaded Text Generation with Markov Transformers #decoding
3. 200605 CoCon

topic model

1. 200426 Neural Topic Modeling with Bidirectional Adversarial Training

topology

1. 200413 Topology of deep neural networks #theory

tracking

1. 200402 Tracking Objects as Points #keypoint
2. 200403 FairMOT
3. 200506 PeTra
4. 201215 Detecting Invisible People

training

1. 200702 Beyond Signal Propagation

transducer

1. 200519 A New Training Pipeline for an Improved Neural Transducer

transfer

1. 200130 BiT ResNet #resnet
2. 200711 Adversarially-Trained Deep Nets Transfer Better #adversarial_training
3. 200716 Do Adversarially Robust ImageNet Models Transfer Better #robust
4. 200721 Adversarial Training Reduces Information and Improves Transferability #adversarial_training
5. 201122 Ranking Neural Checkpoints

transformer

1. 200129 Are Transformers universal approximator
2. 200129 Product Key Memory #attention
3. 200129 Reformer #attention
4. 200130 Sparse Transformer #generative_model
5. 200130 Structured Pruning for LM #pruning
6. 200207 Transformer Transducer #asr #transducer
7. 200211 On Layer Normalization in the Transformer Architecture #normalization
8. 200212 GLU Variants Improve Transformer #activation
9. 200214 Transformer on a Diet #efficient_attention
10. 200214 Transformers as Soft Reasoners over Language #language
11. 200215 Fine-Tuning Pretrained Language Models #bert #finetuning
12. 200221 Addressing Some Limitations of Transformers with Feedback Memory #recurrent
13. 200305 Talking-Heads Attention #attention
14. 200424 Lite Transformer with Long-Short Range Attention #lightweight
15. 200515 Finding Experts in Transformer Models
16. 200515 JDI-T #tts
17. 200516 Conformer #asr
18. 200518 Weak-Attention Suppression For Transformer Based Speech Recognition #asr
19. 200605 Funnel-Transformer #efficient_attention
20. 200707 Do Transformers Need Deep Long-Range Memory #lm #attention
21. 200709 Fast Transformers with Clustered Attention #attention
22. 200715 AdapterHub #nlp #finetuning
23. 200727 Big Bird #attention
24. 200802 DeLighT #nlp
25. 201217 Taming Transformers for High-Resolution Image Synthesis #discrete_vae #generative_model #autoregressive_model
26. 201221 RealFormer #attention
27. 201227 SG-Net #syntax #attention
28. 210223 Do Transformer Modifications Transfer Across Implementations and
29. 210225 Evolving Attention with Residual Convolutions #attention
30. 210318 HiT #video #retrieval
31. 210318 Looking Beyond Two Frames #tracking
32. 210318 TFPose #pose
33. 210318 TransCenter #tracking
34. 210318 Transformer Trackin #tracking
35. 210407 Seeing Out of tHe bOx #multimodal #vision-language
36. 210409 Efficient Large-Scale Language Model Training on GPU Clusters #distributed_training
37. 210409 Not All Attention Is All You Need
38. 210410 UniDrop #regularization
39. 210417 Demystifying the Better Performance of Position Encoding Variants for #positional_encoding
40. 210420 RoFormer #positional_encoding
41. 210423 M3DeTR #3d
42. 210509 FNet #efficient_attention #fourier

tropical geometry

1. 200220 On the Decision Boundaries of Neural Networks

tts

1. 200512 Flowtron #flow

unsupervised img2img

1. 200310 Unpaired Image-to-Image Translation using Adversarial Consistency Loss
2. 200611 Rethinking the Truly Unsupervised Image-to-Image Translation
3. 201201 Unpaired Image-to-Image Translation via Latent Energy Transport

unsupervised nmt

1. 200422 When and Why is Unsupervised Neural Machine Translation Useless

vae

1. 200707 NVAE
2. 201119 Dual Contradistinctive Generative Autoencoder
3. 201120 Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them

video

1. 210325 An Image is Worth 16x16 Words, What is a Video Worth

video transformer

1. 210423 VidTr

vision

1. 200305 Optimizing JPEG Quantization for Classification Networks
2. 201127 Field of Junctions

vision language

1. 201212 MiniVLM
2. 201222 Seeing past words
3. 210407 Multimodal Fusion Refiner Networks

vision transformer

1. 201127 General Multi-label Image Classification with Transformers
2. 201223 A Survey on Visual Transformer
3. 201223 Training data-efficient image transformers & distillation through #distillation
4. 210223 Pyramid Vision Transformer
5. 210318 CrossViT
6. 210318 CvT
7. 210318 Multi-Scale Vision Longformer
8. 210319 ConViT
9. 210319 Scalable Visual Transformers with Hierarchical Pooling
10. 210324 Vision Transformers for Dense Prediction #fpn
11. 210325 Swin Transformer #local_attention
12. 210331 Going deeper with Image Transformers
13. 210402 LeViT
14. 210421 Token Labeling
15. 210422 Multiscale Vision Transformers
16. 210422 So-ViT
17. 210426 Improve Vision Transformers Training by Suppressing Over-smoothing
18. 210426 Visformer
19. 210427 ConTNet
20. 210428 Twins #local_attention #positional_encoding
21. 210509 Conformer
22. 210515 Are Convolutional Neural Networks or Transformers more like human vision #cnn #inductive_bias
23. 210517 Rethinking the Design Principles of Robust Vision Transformer #robustness

visual grounding

1. 210401 Towards General Purpose Vision Systems
2. 210510 Visual Grounding with Transformers

vit

1. 210526 Aggregating Nested Transformers #local_attention
2. 210529 Less is More
3. 210603 DynamicViT #sparse_attention

vocoder

1. 200512 FeatherWave
2. 201118 Universal MelGAN

weak supervision

1. 201126 SelfText Beyond Polygon #ocr

uncategorized

1. 200211 fastai
2. 210603 The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models