Papers,code etc for deep learning study group
https://arxiv.org/pdf/1605.06431v1.pdf - Deep nets are ensembles
https://arxiv.org/pdf/1602.08124v3.pdf - soa for parallelization
https://arxiv.org/pdf/1404.5997v2.pdf - parallel computation issues
http://www.wsdm-conference.org/2016/slides/WSDM2016-Jeff-Dean.pdf - distributed architecture
https://www.youtube.com/watch?v=sUzQpd-Ku4o - video of jeff dean talk
https://arxiv.org/pdf/1611.01578v1.pdf - RL for finding neural architectures
http://mlg.eng.cam.ac.uk/yarin/blog_2248.html - uncertainty in neural nets
https://arxiv.org/pdf/1611.01587.pdf - Joint Many-task model: Neural Net for multiple NLP Tasks - Socher
http://papers.nips.cc/paper/5773-deep-generative-image-models-using-a-laplacian-pyramid-of-adversarial-networks.pdf -GAN paper (recc by LeCun)
https://arxiv.org/pdf/1511.05440.pdf - GAN for video prediction
https://arxiv.org/abs/1703.02528 - Generative unadversarial networks
https://arxiv.org/pdf/1611.01578.pdf - Neural architecture search with RL - google brain
https://arxiv.org/pdf/1703.01041.pdf - Large-Scale Evolution of Image Classifiers - google brain
https://arxiv.org/pdf/1708.05866.pdf - Survey of reinforcement learning
https://arxiv.org/pdf/1710.10196.pdf - training improvements for GAN
https://arxiv.org/pdf/1704.00028v2.pdf - improved training for WGANs
https://openreview.net/forum?id=ry_WPG-A-¬eId=ry_WPG-A - rebuttal for IB theory
http://www.mit.edu/~adedieu/pdf/2048.pdf - deep reinforcement learning
https://arxiv.org/pdf/1710.10784.pdf - geometry of deep learning
https://arxiv.org/pdf/1706.00473.pdf - bayesian perspective
http://openaccess.thecvf.com/content_cvpr_2017/papers/Khoreva_Simple_Does_It_CVPR_2017_paper.pdf - weakly supervised segmentation
https://arxiv.org/pdf/1711.11585.pdf - High resolution image synthesis and semantic manipulation - Nvidia
https://github.com/NVIDIA/pix2pixHD - Synthesizing and manipulating 2048x1024 images with conditional GAN
https://arxiv.org/pdf/1505.05424.pdf - weight uncertainties
https://arxiv.org/pdf/1711.01297.pdf - weight uncertainties
https://arxiv.org/pdf/1802.03268.pdf - Efficient Neural architecture search
https://github.com/RedditSota/state-of-the-art-result-for-machine-learning-problems - SOTA algorithms
https://arxiv.org/pdf/1711.10925.pdf - deep image prior
https://arxiv.org/pdf/1711.03953.pdf - breaking softmax bottleneck
mixed scale deep convolution - PNAS Dec 26, 2017
http://lanl.arxiv.org/pdf/1803.05049v1 - Fractal AI
https://arxiv.org/abs/1802.05365 - Deep contextualized word representations
https://arxiv.org/pdf/1804.04241.pdf - capsule net for segmentation (improvement 95%)
https://arxiv.org/pdf/1704.00109.pdf - Snapshot ensembles
https://arxiv.org/pdf/1711.00937.pdf - Neural discrete representation learning
https://arxiv.org/find/cs/1/au:+Segler_M/0/1/0/all/0/1 - refs on chemical models
https://arxiv.org/pdf/1801.10130.pdf - spherical CNN
https://arxiv.org/pdf/1804.02958.pdf - GAN for extreme compression
https://arxiv.org/pdf/1703.05698.pdf - Neural Sketch Learning for Conditional Program Generation
Papers on Learning Rate Scheduling --
https://arxiv.org/pdf/1608.03983.pdf - SGD with warm restarts
https://arxiv.org/pdf/1506.01186.pdf - Cyclical learning rates
https://arxiv.org/pdf/1803.10122.pdf - World models - teaching simple world model prepartory to RL - schmidhuber
ICLR top papers - https://iclr.cc/Conferences/2018/Schedule?type=Oral
https://arxiv.org/abs/1703.06114 Deep Sets
https://arxiv.org/abs/1807.02443 Tangent Convolutions for Dense Prediction in 3D.
https://arxiv.org/pdf/1806.01261.pdf - deep mind graph paper
https://arxiv.org/pdf/1805.11604.pdf - How does Batch normalization work - it's not about covariate shift
https://arxiv.org/pdf/1802.05983.pdf - Disentangling by factorizing
https://arxiv.org/pdf/1808.00508.pdf - Neural arithmetic logic units
https://arxiv.org/pdf/1803.08660.pdf - A new activation function
https://arxiv.org/pdf/1803.05268.pdf - Interpretability in CNN
Here's what the cool kids in SF are looking at this week --
https://arxiv.org/abs/1809.05042 - "Hamiltonian Descent Methods"
https://arxiv.org/pdf/1812.11314.pdf - Meta Reinforcement Learning with Distribution of Exploration Parameters Learned by Evolution Strategies
https://arxiv.org/pdf/1812.11675.pdf - Soft Autoencoder and Its Wavelet Shrinkage Interpretation
https://arxiv.org/pdf/1901.01122.pdf - Machine Translation: A Literature Review
https://arxiv.org/pdf/1901.01010.pdf - A Joint Model for Multimodal Document Quality Assessment
https://arxiv.org/pdf/1901.00949.pdf - Machine Teaching in Hierarchical Genetic Reinforcement Learning: Curriculum Design of Reward Functions for Swarm Shepherding
https://arxiv.org/pdf/1901.00983.pdf - Brief Review of Computational Intelligence Algorithms
https://arxiv.org/pdf/1901.00898.pdf - Imminent Collision Mitigation with Reinfo rcement Learning and Vision
https://github.com/borisbanushev/stockpredictionai - predicting stock prices
https://arxiv.org/abs/1806.01261 - relational inductive bias in graph - deep mind
http://proceedings.mlr.press/v97/mahoney19a/mahoney19a.pdf - Traditional and heavy tailed self regularization in neural net models
https://openreview.net/pdf?id=HygQBn0cYm - Model predictive policy learning with uncertainty regularization for driving in dense traffic
https://arxiv.org/pdf/1906.07774.pdf - information matrices and generalization - bengio
https://arxiv.org/pdf/1710.10903.pdf - graph attention networks bengio
https://arxiv.org/pdf/1812.09430.pdf - dynamic graph representation learning via self attention networks
https://arxiv.org/pdf/1906.04358.pdf - weight agnostic neural networks
https://arxiv.org/pdf/1804.00222.pdf - Meta-Data update rules for unsupervised representation learning
https://arxiv.org/abs/1901.10430 - Pay less attention with lightweight and dynamic convolutions
https://arxiv.org/pdf/1806.03107.pdf - Temporal difference variational autoencoder
https://arxiv.org/pdf/1810.00826.pdf - How powerful are graph neural networks?
https://arxiv.org/abs/1906.07084 - Particle swarm optimization for great enhancement in semi-supervised retinal vessel segmentation with generative adversarial networks
https://arxiv.org/pdf/1908.03015.pdf - Augmenting VAE with sparse labels: A unified framework for supervised and semi-supervised learning.
https://arxiv.org/pdf/1911.06294.pdf - DEEP REINFORCEMENT LEARNING FOR ADAPTIVE TRAFFIC SIGNAL CONTROL
https://arxiv.org/pdf/1911.06615.pdf - Deep learning methods in speaker recognition: a review
https://arxiv.org/pdf/1911.06904.pdf - Improving Graph Neural Network Representations of Logical Formulae with Subgraph Pooling
https://arxiv.org/pdf/1911.07470.pdf - Graph Transformer for Graph-to-Sequence Learning
https://arxiv.org/pdf/1911.07532.pdf - Graph Neural Ordinary Differential Equations - modeling time varying graphs
https://arxiv.org/pdf/1911.08517.pdf - Generalizable Resource Allocation in Stream Processing via Deep Reinforcement Learning
https://arxiv.org/pdf/1812.11951.pdf - Learning to Design RNA
https://arxiv.org/pdf/1911.06105.pdf - PharML.Bind: Pharmacologic Machine Learning for Protein-Ligand Interactions
https://arxiv.org/pdf/1911.06107.pdf - EARTHMOVER-BASED MANIFOLD LEARNING FOR ANALYZING MOLECULAR
CONFORMATION SPACES
https://arxiv.org/pdf/1911.07125.pdf - Opportunities for artificial intelligence in advancing precision medicine
https://arxiv.org/pdf/1911.06198.pdf - Election control in social networks via edge addition and removal
https://arxiv.org/pdf/1911.05885.pdf - Deception through half-truths
https://arxiv.org/pdf/1911.05892.pdf - Reinforcement Learning for Market Making in Multi-agent Dealer Market
https://arxiv.org/pdf/1911.06193.pdf - Predicting Indian stock market using psycho-linguistic features of financial news
https://arxiv.org/pdf/1911.05952.pdf - Change point analysis in financial networks
https://arxiv.org/pdf/1911.05620.pdf - Neural networks for option pricing and hedging - a literature review
https://arxiv.org/pdf/1911.06126.pdf - Unveil stock correlation via a new tensor-based decomposition method
https://arxiv.org/pdf/1911.08647.pdf - Deep Reinforcement Learning in Cryptocurrency Market Making
https://arxiv.org/pdf/1912.09524.pdf - Evolving ab initio trading strategies in heterogeneous environments
https://arxiv.org/pdf/1912.10343.pdf - Design of High-Frequency Trading Algorithm Based on Machine Learning
https://arxiv.org/abs/1912.10806 - DP-LSTM: Differential Privacy-inspired LSTM for Stock Prediction Using Financial News
https://arxiv.org/pdf/1912.10813.pdf - MODEL UNCERTAINTY IN FINANCIAL FORECASTING
https://arxiv.org/pdf/1910.13675.pdf- Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly
https://arxiv.org/pdf/1802.08232.pdf- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
https://arxiv.org/pdf/2002.11089.pdf - Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement
https://www.osapublishing.org/DirectPDFAccess/C6D6B2C3-953C-4461-695B6E5E2F993943_415059/prj-7-8-823.pdf?da=1&id=415059&seq=0&mobile=no --Nanophotonic media for artificial neural inference
https://arxiv.org/pdf/1910.02789.pdf - Language is Power: Representing States Using Natural Language in Reinforcement Learning
https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-discovery - Protein folding paper.
https://arxiv.org/abs/2001.04451 Reformer, the efficient transformer
https://ai.googleblog.com/2020/01/reformer-efficient-transformer.html
https://arxiv.org/pdf/1906.05717.pdf - Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics
https://arxiv.org/pdf/1912.09524.pdf - Evolving ab initio trading strategies in heterogeneous environments
https://arxiv.org/pdf/1911.05892.pdf - Reinforcement Learning for Market Making in Multi-agent Dealer Market
https://www.nature.com/articles/s41586-019-1724-z.epdf?author_access_token=lZH3nqPYtWJXfDA10W0CNNRgN0jAjWel9jnR3ZoTv0PSZcPzJFGNAZhOlk4deBCKzKm70KfinloafEF1bCCXL6IIHHgKaDkaTkBcTEv7aT-wqDoG1VeO9-wO3GEoAMF9bAOt7mJ0RWQnRVMbyfgH9A%3D%3D
https://www.gwern.net/docs/rl/2019-vinyals.pdf
https://deepmind.com/blog/article/AlphaStar-Grandmaster-level-in-StarCraft-II-using-multi-agent-reinforcement-learning
https://arxiv.org/pdf/1911.04252.pdf - Self-training with Noisy Student improves ImageNet classification
https://arxiv.org/pdf/1910.12713.pdf - Few-shot video-video synthesis
https://arxiv.org/pdf/1906.11883.pdf - Unsupervised learning of Object Keypoints for Perception and Control
https://arxiv.org/pdf/1710.03748.pdf - Emergent Complexity via Multi-Agent Competition
https://openai.com/blog/competitive-self-play/
https://arxiv.org/pdf/1703.04908.pdf - Emergence of Grounded Compositional Language in Multi-Agent Populations
https://arxiv.org/pdf/1909.07528.pdf - Emergent tool use from multi agent autocurricula
https://openai.com/blog/emergent-tool-use/
https://arxiv.org/pdf/1901.00949.pdf - Machine Teaching in Hierarchical Genetic Reinforcement Learning: Curriculum Design of Reward Functions for Swarm Shepherding
https://arxiv.org/pdf/1812.01729.pdf - Boltzman Generators - Sampling equilibrium states of many body systems with deep learning
https://arxiv.org/pdf/1907.10599.pdf - Fine Grained Spectral Perspective on Neural Networks
https://arxiv.org/pdf/1906.08237.pdf - XLNet Generalized autoregressive pretraining for language understanding
https://arxiv.org/pdf/1905.09272.pdf - Data efficient image recognition with contrastive predictive coding.
https://arxiv.org/pdf/1904.10509.pdf - Generating long sequences with sparse transformers
https://arxiv.org/pdf/1807.03748.pdf - Representation learning with contrastive predictive coding.
https://arxiv.org/pdf/1906.08253.pdf - When to trust your model: model-based policy optimization
https://arxiv.org/pdf/1901.09321.pdf - Fixup initialization - residual learning without normalization
http://proceedings.mlr.press/v97/mahoney19a/mahoney19a.pdf - Traditional and heavy tailed self regularization in neural net models
https://arxiv.org/pdf/1804.08838.pdf - Measuring intrinsic dimension of objective landscapes
https://arxiv.org/abs/1810.09536 - Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
https://arxiv.org/pdf/1812.05159.pdf - An empirical study of example forgetting during neural network training.
https://arxiv.org/pdf/1812.00417.pdf - Snorkel Drybell - A case study in weak supervision at industrial scale
https://arxiv.org/pdf/1905.04981.pdf - Modelling instance level annotator reliability for natural language labelling
https://arxiv.org/pdf/1901.09321.pdf - Fixup Initialization: Residual Learning without Normalization
https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf - Language Models are Unsupervised Multitask Learners.
https://arxiv.org/pdf/1811.00995.pdf - Invertible Residual Networks
https://arxiv.org/pdf/1904.01681.pdf - Augmented Neural ODE's
https://arxiv.org/pdf/1901.00596.pdf - Comprehensive Survey of Graph Neural Nets
https://github.com/rusty1s/pytorch_geometric
https://arxiv.org/pdf/1901.00596.pdf - Comprehensive Survey of Graph Neural Nets
https://papers.nips.cc/paper/7539-optimal-algorithms-for-non-smooth-distributed-optimization-in-networks.pdf - nips award winner
https://papers.nips.cc/paper/8200-non-delusional-q-learning-and-value-iteration.pdf - Non-delusional Q-learning and Value Iteration
https://arxiv.org/pdf/1706.03762.pdf - attention is all you need - Vaswani
https://github.com/jadore801120/attention-is-all-you-need-pytorch - easier to read code
https://www.youtube.com/watch?v=S0KakHcj_rs
https://tdls.a-i.science/events/2018-10-22/
https://tdls.a-i.science/events/2019-02-04/
http://nlp.seas.harvard.edu/2018/04/03/attention.html
https://arxiv.org/pdf/1806.02643.pdf - Re-evalating Evaluation
https://arxiv.org/pdf/1812.11951.pdf - Learning to Design RNA
https://arxiv.org/pdf/1901.02860.pdf - Transformer XL - Attentive Language Models, Beyond a fixed length context
https://arxiv.org/pdf/1809.06646.pdf - Model Free Adaptive Optimal Control of Sequential Manufacturing Process Using Reinforcement Learning
https://arxiv.org/pdf/1806.07366.pdf - Neural Ordinary Differential Equations - Top paper NIPS2019
https://arxiv.org/pdf/1606.05312.pdf - Successor Features for Transfer in Reinforcement Learning
http://proceedings.mlr.press/v37/schaul15.pdf - Universal Value Function Approximators
http://proceedings.mlr.press/v80/barreto18a/barreto18a.pdf - Transfer in deep reinforcement learning using successor features and generalised policy improvement.
https://www.youtube.com/watch?v=YDCPHekLUI4&t=1053s - Tom Schaul
https://www.youtube.com/watch?v=OCHwXxSW70o - Tejas Kulkarni
https://arxiv.org/pdf/1812.07626.pdf - Universal Successor Features Approximators
https://arxiv.org/pdf/1810.12715.pdf - On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
https://openreview.net/pdf?id=S1x4ghC9tQ - Temporal Difference Variational Autoencoder
https://openreview.net/pdf?id=S1JHhv6TW - Boosting Dilated Convolution with Mixed Tensor Decompositions
https://arxiv.org/pdf/1712.01208.pdf - The case for learned index structures
https://arxiv.org/abs/1809.07402 - Generalization properties of nn - Socher
https://einstein.ai/research/blog/identifying-generalization-properties-in-neural-networks - blog for above paper
https://arxiv.org/pdf/1802.05983.pdf - Disentangling by Factorising
https://arxiv.org/pdf/1804.00104.pdf - Learning Disentangled Joint, Discrete and Continuous Representations
https://arxiv.org/pdf/1807.05520.pdf - Deep Clustering for Unsupervised Learning of Visual Features
https://github.com/1Konny/FactorVAE
https://github.com/paruby/FactorVAE
https://github.com/nicolasigor/FactorVAE
https://arxiv.org/pdf/1810.12894.pdf - Exploration by Random Network Distillation - OpenAI
https://arxiv.org/pdf/1810.04805.pdf - Pre-trainged bi directional transformers for language translation
https://arxiv.org/pdf/1801.02613.pdf - Characterizing Adversarial Examples using Local Intrinsic Dimensionality
https://arxiv.org/pdf/1808.06670.pdf - Learning Deep Representations by Mutual Estimation Estimation and Maximization - Hjelm, Bengio
https://arxiv.org/pdf/1802.04364.pdf - Junction Tree Variational Auto-Encoder for Molecular Graph Generation
http://snap.stanford.edu/proj/embeddings-www/files/nrltutorial-part2-gnns.pdf
https://arxiv.org/pdf/1808.06601.pdf - Video to video synthesis https://github.com/NVIDIA/vid2vid - code
https://arxiv.org/pdf/1807.03146.pdf - Discovery of 3d keypoints from 2d image
https://arxiv.org/abs/1709.02371 - PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume," by Deqing Sun et al. (CVPR 2018)
Phil Ferrier will present the paper and run though his code for us. Phil's code is on his github reop:
https://github.com/philferriere/tfoptflow
https://arxiv.org/pdf/1807.03247.pdf - Intriguing failure (and improvement) to CNN for determining rotations.
https://arxiv.org/pdf/1803.03324.pdf - Learning Deep Generative Models of Graphs
https://arxiv.org/abs/1709.10082 - Optimally decentralized multi-robot collision avoidance w reinforcement learning.
https://github.com/TensorSwarm/TensorSwarm - Andreas Pasternak code for above
https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/learning-dexterity/learning-dexterity-paper.pdf -Robot doing single hand manipulations.
https://www.theverge.com/2018/7/30/17621112/openai-robot-dexterity-dactyl-artificial-intelligence
https://arxiv.org/pdf/1711.03953.pdf - Breaking the softmax bottleneck
https://arxiv.org/pdf/1805.10829.pdf - SigSoftMax: Reanalyzing the softmax bottleneck
https://severelytheoretical.wordpress.com/2018/06/08/the-softmax-bottleneck-is-a-special-case-of-a-more-general-phenomenon/
https://arxiv.org/pdf/1807.01281.pdf - Human level performance in first person multiplayer games with population reinforcement learning.
https://deepmind.com/blog/capture-the-flag/
https://www.youtube.com/watch?v=steioHoiEms
https://arxiv.org/abs/1711.09846v2
https://arxiv.org/pdf/1611.05397.pdf
https://arxiv.org/pdf/1803.10122.pdf - schmidhuber paper on RL
https://deepmind.com/research/publications/neural-scene-representation-and-rendering/ - Rendering 3d scene
https://arxiv.org/pdf/1707.06347.pdf - Proximal Optimization Policies
https://openreview.net/pdf?id=BJOFETxR- - Learning to represent programs with graphs
https://openreview.net/pdf?id=BkisuzWRW - Zero Shot Visual Imitation - Reinforcement Learning
https://openreview.net/forum?id=HkL7n1-0b - Wasserstein Auto Encoders - one of ICLR top papers.
https://openreview.net/pdf?id=Hy7fDog0b - Ambient GAN - Generative Models from Lossy Measurements - ICLR top paper
https://arstechnica.com/science/2018/05/ai-trained-to-navigate-develops-brain-like-location-tracking/ - Grid representations in rat brain
https://deepmind.com/documents/200/Banino_at_al_final.pdf --
https://www.nature.com/articles/s41586-018-0102-6 --
https://arxiv.org/pdf/1712.06567.pdf - Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for
Training Deep Neural Networks for Reinforcement Learning
https://arxiv.org/pdf/1712.06560.pdf - Improving Exploration in Evolution Strategies for Deep Reinforcement
Learning via a Population of Novelty-Seeking Agents
https://eng.uber.com/deep-neuroevolution/ - Uber engineering blog post
https://arxiv.org/pdf/1801.10130.pdf - spherical CNN
https://arxiv.org/pdf/1710.07313.pdf - Using machine learning to replicate chaotic attractors
http://www.bmp.ds.mpg.de/tl_files/bmp/preprints/Zimmermann_Parlitz_preprint.pdf - paper to be published in "chaos"
https://www.quantamagazine.org/machine-learnings-amazing-ability-to-predict-chaos-20180418/ - blog post
https://arxiv.org/pdf/1711.10925.pdf - Deep Image Prior
https://dmitryulyanov.github.io/deep_image_prior - git hub from authors
https://box.skoltech.ru/index.php/s/ib52BOoV58ztuPM
http://mlexplained.com/2018/01/18/paper-dissected-deep-image-prior-explained/
http://fortune.com/2018/04/24/nvidia-artificial-intelligence-images/ - Article w video showing photo editing use
Finish Fractal AI
https://arxiv.org/pdf/1711.07971.pdf - non-local filtering
http://lanl.arxiv.org/pdf/1803.05049v1 - Fractal AI
https://arxiv.org/pdf/1803.04831.pdf - IndRNN longer deeper RNN's
https://arxiv.org/pdf/1711.10433.pdf - parallel wavenet
https://arxiv.org/pdf/1708.04552.pdf - regularizing convnet with cutout (desert paper)
http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf - will get short presentation on this one.
https://arxiv.org/pdf/1802.03268.pdf - Efficient Neural Architecture Search via Parameter Sharing
https://github.com/carpedm20/ENAS-pytorch
some related papers and reviews.
https://arxiv.org/pdf/1708.05344.pdf - One shot architecture search
https://openreview.net/forum?id=ByQZjx-0-
and
https://openreview.net/forum?id=rydeCEhs-
https://arxiv.org/abs/1703.10135 - tacotron - end-to-end speech synthesis
https://arxiv.org/pdf/1712.05884.pdf - tacotron 2
https://research.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html -
https://github.com/A-Jacobson/tacotron2 - pytorch code
http://research.baidu.com/deep-speech-3%EF%BC%9Aexploring-neural-transducers-end-end-speech-recognition/
https://arxiv.org/pdf/1705.09792.pdf - Deep Complex Networks
https://arxiv.org/pdf/1801.10308.pdf - Nested LSTM's
https://arxiv.org/pdf/1705.10142.pdf - KRU from Fair
https://github.com/hannw/nlstm - tf code for Nested LSTM
http://openaccess.thecvf.com/content_cvpr_2017/papers/Khoreva_Simple_Does_It_CVPR_2017_paper.pdf - Weakly Supervised Instance and Semantic Segmentation
https://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/weakly-supervised-learning/simple-does-it-weakly-supervised-instance-and-semantic-segmentation/
https://github.com/philferriere/tfwss - Phil Ferriere's code
https://drive.google.com/file/d/1wPHMA4PqygawvIxRiy-2ZMKcpUO447cz/view?usp=sharing - mehul's notebook on segmentation
https://arxiv.org/pdf/1511.06939.pdf - using rnn for recommendation system
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46488.pdf - latest paper on rnn for recommendation
https://arxiv.org/pdf/1709.04511.pdf - Empirical study of multi-agent RL
https://github.com/geek-ai/1m-agents - code
https://arxiv.org/pdf/1704.00028.pdf - Improvements in Wasserstein GAN training
https://arxiv.org/pdf/1710.02298.pdf - Combining improvements in deep reinforcement learning
https://openreview.net/pdf?id=HJWLfGWRb - follow-on to capsule network paper
https://www.youtube.com/watch?v=pPN8d0E3900
https://www.youtube.com/watch?v=2Kawrd5szHE
https://github.com/ageron/handson-ml/blob/master/extra_capsnets.ipynb
https://github.com/naturomics/CapsNet-Tensorflow
https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-ii-how-capsules-work-153b6ade9f66
https://arxiv.org/pdf/1710.09829.pdf - Dynamic routing between capsules - Hinton
https://arxiv.org/pdf/1701.01724.pdf - DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker
https://deepmind.com/documents/119/agz_unformatted_nature.pdf - alpha zero paper
https://webdocs.cs.ualberta.ca/~mmueller/talks/2016-LeeSedol-AlphaGo.pdf - some slides
https://arxiv.org/pdf/1703.10593.pdf - cycle consistent GANs
https://arxiv.org/pdf/1503.02406.pdf Naftali Tishby and Noga Zaslavsky. information bottleneck principle.
https://www.cs.huji.ac.il/labs/learning/Papers/allerton.pdf - Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method.
Mask R-CNN
https://arxiv.org/abs/1703.06870
And these are prerequisites (read at least Fast R-CNN and Faster R-CNN)
R-CNN
https://arxiv.org/abs/1311.2524
Fast R-CNN
https://arxiv.org/pdf/1504.08083.pdf
Faster R-CNN
https://arxiv.org/abs/1506.01497 Feature Pyramid Networks
https://arxiv.org/abs/1612.03144
https://arxiv.org/pdf/1703.00810.pdf - Opening the Black Box of Neural Nets via Information
https://www.youtube.com/watch?v=ekUWO_pI2M8
https://www.youtube.com/watch?v=bLqJHjXihK8
https://arxiv.org/pdf/1501.00092.pdf - super resolution first paper
https://arxiv.org/abs/1608.00367 - super resolution second paper
https://arxiv.org/abs/1604.03901 - Single-Image Depth Perception in the Wild
https://arxiv.org/pdf/1706.08947.pdf - Exploring generalization in deep networks.
https://arxiv.org/pdf/1705.02550.pdf - nvidia drone nav
https://github.com/NVIDIA-Jetson/redtail/wiki - code
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.365.5060&rep=rep1&type=pdf - hyperneat ref
https://arxiv.org/pdf/1609.09106.pdf - Hypernet ref
http://blog.otoro.net/2016/09/28/hyper-networks/ - blog on hypernet
https://www.youtube.com/watch?v=-8oyTYViuJ4 - vid on hyperNeat
http://eplex.cs.ucf.edu/hyperNEATpage/HyperNEAT.html - blog on hyperNeat
https://arxiv.org/pdf/1708.05344.pdf - SMASH: One-Shot Model Architecture Search through HyperNetworks https://www.youtube.com/watch?v=79tmPL9AL48 - youtube vid on SMASH
https://arxiv.org/pdf/1706.02515.pdf - Self Normalizing Neural Networks - Hochreiter
https://arxiv.org/pdf/1606.01541.pdf - Reinforcement Learning for Dialog Generation - Jurafsky
https://github.com/liuyuemaicha/Deep-Reinforcement-Learning-for-Dialogue-Generation-in-tensorflow - tensorflow code for same
https://github.com/jiweil/ - some related code
https://arxiv.org/pdf/1612.00563.pdf - self critical training for image captioning - RL for text prob.
Some papers referenced by Jurafsky paper
[1506.05869] A Neural Conversational Model - Vinyals and Le
https://arxiv.org/abs/1604.04562 - Dialogue generation system - Wen
https://arxiv.org/pdf/1705.04304.pdf - A Deep Reinforced Model for Abstractive Summarization - socher
https://arxiv.org/pdf/1706.01433.pdf - visual interaction networks - deep mind
https://arxiv.org/pdf/1706.01427.pdf - neural model for relational reasoning - deep mind
Guest Speaker - Using FPGA to speed CNN.
https://arxiv.org/pdf/1703.03130.pdf - A structured self-attentive sentence embedding - Lin and Bengio
https://github.com/dennybritz/deeplearning-papernotes/blob/master/notes/self_attention_embedding.md (review)
https://github.com/yufengm/SelfAttentive code
https://github.com/Diego999/SelfSent code
https://arxiv.org/pdf/1706.03762.pdf - attention is all you need - Vaswani
https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/models
https://github.com/jadore801120/attention-is-all-you-need-pytorch - easier to read code
https://arxiv.org/pdf/1607.06450.pdf - layer normalization paper - hinton
https://www.youtube.com/watch?v=nR74lBO5M3s - google translate paper - youtube video
https://arxiv.org/pdf/1609.08144.pdf - google translate paper -
https://arxiv.org/pdf/1706.03762.pdf - attention is all you need - Vaswani
https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/models
https://github.com/jadore801120/attention-is-all-you-need-pytorch - easier to read code
https://arxiv.org/pdf/1607.06450.pdf - layer normalization paper - hinton
http://www.machinelearning.org/proceedings/icml2006/047_Connectionist_Tempor.pdf - A. Graves, S. Fernandez, F. Gomez, and J. Schmidhuber
https://www.reddit.com/r/MachineLearning/comments/6jdi87/r_question_about_positional_encodings_used_in/
https://arxiv.org/pdf/1705.03122.pdf - convolutional sequence to sequence learning
https://arxiv.org/pdf/1706.03762.pdf - attention is all you need - Vaswani
http://www.machinelearning.org/proceedings/icml2006/047_Connectionist_Tempor.pdf - A. Graves, S. Fernandez, F. Gomez, and J. Schmidhuber
https://arxiv.org/pdf/1701.02720.pdf - RNN for end to end voice recognition
New reinforcement learning results -- Too cool for school. Watch the video and you'll be hooked.
https://www.youtube.com/watch?v=2vnLBb18MuQ&feature=em-subs_digest
http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/index.html - paper
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/HintonDengYuEtAl-SPM2012.pdf - comparison of RNN and HMM for speech recognition
https://arxiv.org/pdf/1412.6572.pdf - Explaining and Harnessing Adversarial Examples
https://arxiv.org/abs/1704.03453 - The Space of Transferable Adversarial Examples
https://discourse-production.oss-cn-shanghai.aliyuncs.com/original/3X/1/5/15ba4cef726cab390faa180eb30fd82b693469f9.pdf - Using TPU for data center
Reservoir Computing by Felix Grezes. http://www.gc.cuny.edu/CUNY_GC/media/Computer-Science/Student%20Presentations/Felix%20Grezes/Second_Exam_Survey_Felix_Grezes_9_04_2014.pdf
Slides by Felix Grezes: Reservoir Computing for Neural Networks
http://www.gc.cuny.edu/CUNY_GC/media/Computer-Science/Student%20Presentations/Felix%20Grezes/Second_Exam_Slides_Felix_Grezes_9-14-2014.pdf
(more at: http://speech.cs.qc.cuny.edu/~felix/ )
This is a short, very useful backgrounder on randomized projections,
here used for compressed sensing, in a blog post by Terence Tao
https://terrytao.wordpress.com/2007/04/13/compressed-sensing-and-single-pixel-cameras/
and the same story told with illustrations on the Nuit Blanche blog:
http://nuit-blanche.blogspot.com/2007/07/how-does-rice-one-pixel-camera-work.html
(BTW http://nuit-blanche.blogspot.com is a tremendous website.)
If we have time, we may discuss this paper:
Information Processing Using a Single Dynamical Node as Complex System.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3195233/pdf/ncomms1476.pdf
https://arxiv.org/pdf/1603.08678.pdf - Instance-sensitive Fully Convolutional Networks
https://arxiv.org/pdf/1611.07709.pdf - Fully Convolutional Instance-aware Semantic Segmentation
https://arxiv.org/pdf/1703.03864.pdf - Sutskever paper on using evolutionary systems for optimizing RL prob
http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wierstra14a.pdf - ES paper with algo used in Sutskever paper
Aurobindo Tripathy will reprise a talk he's going to give at Embedded Summit this year. His talk will survey recent progress in object detection from RCNN to Single Shot MultiBox Detector and Yolo 9000.
https://arxiv.org/pdf/1612.05424.pdf - Unsupervised Pixel-level domain adaptation with generative adversarial networks
https://arxiv.org/pdf/1701.06547.pdf - adversarial learning for neural dialog generation
https://arxiv.org/pdf/1612.02699.pdf - Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
Zeeshan's slides are in the folder with his name on it. Along with his descriptions of his own ground-breaking work, he gives an excellent history of efforts to identify 3d objects from 2d images.
https://arxiv.org/pdf/1506.07285.pdf - Ask me anything - Socher
https://github.com/YerevaNN/Dynamic-memory-networks-in-Theano - Code and implementation notes.
https://www.youtube.com/watch?v=FCtpHt6JEI8&t=27s - Socher presentation of material
https://arxiv.org/pdf/1701.06538v1.pdf - Outrageously large neural networks
https://arxiv.org/pdf/1505.00387v2.pdf - Highway networks
https://arxiv.org/pdf/1507.06228.pdf - Also highway networks - different examples
https://arxiv.org/pdf/1607.03474v3.pdf - Recurrent Highway Networks
https://arxiv.org/pdf/1603.03116v2.pdf - Low-rank pass-through RNN's follow-on to unitary rnn https://github.com/Avmb/lowrank-gru - theano code
https://arxiv.org/abs/1612.03242 - Stack Gan Paper
https://github.com/hanzhanggit/StackGAN - Code
https://arxiv.org/pdf/1511.06464v4.pdf - Unitary Evolution RNN https://github.com/amarshah/complex_RNN - theano code
Cheuksan Edward Wang Talk
https://arxiv.org/pdf/1612.04642v1.pdf - rotation invariant cnn
https://github.com/deworrall92/harmonicConvolutions - tf code for harmonic cnn
http://visual.cs.ucl.ac.uk/pubs/harmonicNets/index.html - blog post by authors
https://arxiv.org/pdf/1602.02218v2.pdf - using typing to improve RNN behavior
http://jmlr.org/proceedings/papers/v37/jozefowicz15.pdf - exploration of alternative LSTM architectures
https://arxiv.org/pdf/1611.01576.pdf - Socher qRnn paper
https://arxiv.org/pdf/1604.02135v2.pdf - latest segmentation fair
https://github.com/MarvinTeichmann/tensorflow-fcn - code for segmenter
https://arxiv.org/pdf/1506.06204.pdf - Object segmentation
https://arxiv.org/pdf/1603.08695v2.pdf - refinement of above segmentation paper
https://code.facebook.com/posts/561187904071636/segmenting-and-refining-images-with-sharpmask/ - blog post
https://github.com/facebookresearch/deepmask - torch code for deepmask
https://arxiv.org/pdf/1506.01497v3.pdf
people.eecs.berkeley.edu/~rbg/slides/rbg-defense-slides.pdf - Girshick thesis slides
Check edge boxes and selective search
https://arxiv.org/pdf/1406.4729v4.pdf - key part of architecture
https://github.com/smallcorgi/Faster-RCNN_TF - excellent code
https://people.eecs.berkeley.edu/~rbg/papers/r-cnn-cvpr.pdf - RCNN
https://arxiv.org/pdf/1504.08083v2.pdf - RCNN - first in series
https://arxiv.org/pdf/1506.01497v3.pdf - Faster R-CNN
http://techtalks.tv/talks/rich-feature-hierarchies-for-accurate-object-detection-and-semantic-segmentation/60254/ - video of Girshick talk
https://arxiv.org/pdf/1506.02025v3.pdf - Spatial transformer networks
https://github.com/daviddao/spatial-transformer-tensorflow - tf code for above
https://github.com/jazzsaxmafia/show_attend_and_tell.tensorflow - tf code for attention-captioning http://cs.stanford.edu/people/karpathy/densecap/ - karpathy captioning https://arxiv.org/pdf/1412.2306v2.pdf - earlier karpathy captioning paper
https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html - Deep dive into reinforcement learning - Sutton and Barto - Chapters 1 and 2.
https://arxiv.org/pdf/1608.06993v1.pdf - DenseNet. New reigning champion image classifier
https://github.com/liuzhuang13/DenseNet - lua code
The DenseNet paper is straight-forward, so we're also going to start on image captioning
http://www.cs.toronto.edu/~zemel/documents/captionAttn.pdf
http://kelvinxu.github.io/projects/capgen.html
http://people.ee.duke.edu/~lcarin/Yunchen9.25.2015.pdf - slides for caption attention
collections of captioning papers.
https://github.com/kjw0612/awesome-deep-vision#image-captioning - images
https://github.com/kjw0612/awesome-deep-vision#video-captioning - video
http://www.mit.edu/~dimitrib/NDP_Encycl.pdf - (early) Bersekas paper on RL, policy and value iteration
http://www.nervanasys.com/demystifying-deep-reinforcement-learning/?imm_mid=0e2d7e&cmp=em-data-na-na-newsltr_20160420 - blog post on RL. Nice coverage of value iteration
https://github.com/carpedm20/pixel-rnn-tensorflow - tensorflow code for pixel rnn (and cnn)
https://arxiv.org/pdf/1606.05328v2.pdf - Conditional Image Generation with PixelCNN decoders
https://arxiv.org/pdf/1601.06759v3.pdf - Pixel RNN
https://drive.google.com/file/d/0B3cxcnOkPx9AeWpLVXhkTDJINDQ/view - wavenet Generative Audio
https://deepmind.com/blog/wavenet-generative-model-raw-audio/ - wavenet blog
http://www.gitxiv.com/posts/fepYG4STYaej3KSPZ/densely-connected-convolutional-netowork-densenet
http://arxiv.org/pdf/1410.3916v11.pdf - original memory networks
https://arxiv.org/pdf/1606.03126v1.pdf - key/value memory augmented nn
http://www.thespermwhale.com/jaseweston/icml2016/icml2016-memnn-tutorial.pdf#page=87 - tutorial on memory networks in language understanding
https://arxiv.org/pdf/1410.5401v2.pdf - Neural Turing Machines
https://github.com/carpedm20/NTM-tensorflow
https://www.youtube.com/watch?v=_H0i0IhEO2g - Alex Graves presentation at microsoft research
http://www.robots.ox.ac.uk/~tvg/publications/talks/NeuralTuringMachines.pdf - slides for ntm
http://arxiv.org/pdf/1410.3916v11.pdf - original memory networks
https://arxiv.org/pdf/1606.03126v1.pdf - key/value memory augmented nn
http://www.thespermwhale.com/jaseweston/icml2016/icml2016-memnn-tutorial.pdf#page=87 - tutorial on memory networks in language understanding
https://arxiv.org/pdf/1605.07648v1.pdf - fractal net - alternative to resnet for ultra-deep convolution
https://github.com/edgelord/FractalNet - tf code
http://www.gitxiv.com/posts/ibA8QEu8bvBJSDxr9/fractalnet-ultra-deep-neural-networks-without-residuals
https://arxiv.org/pdf/1602.01783v2.pdf - new RL architecture - deep mind
Code:
https://github.com/Zeta36/Asynchronous-Methods-for-Deep-Reinforcement-Learning - tf
https://github.com/miyosuda/async_deep_reinforce - tf
https://github.com/coreylynch/async-rl - keras (tf)
https://github.com/muupan/async-rl - chainer (good discussion)
https://arxiv.org/pdf/1607.02533v1.pdf - Hardening deep networks to adversarial examples.
http://www.gitxiv.com/posts/HQJ3F9YzsQZ3eJjpZ/model-free-episodic-control - deep mind gitxiv paper and code on github https://github.com/sudeepraja/Model-Free-Episodic-Control - other code https://github.com/ShibiHe/Model-Free-Episodic-Control
https://arxiv.org/pdf/1406.2661.pdf - originating paper on generative adversarial net (gan) - goodfellow, bengio
http://arxiv.org/pdf/1511.06434v2.pdf - deep cnn gan - radford
https://github.com/Newmu/dcgan_code - theano code for cnn gan - radford
http://www.gitxiv.com/posts/HQJ3F9YzsQZ3eJjpZ/model-free-episodic-control - deep mind gitxiv paper and code on github
Papers -
https://drive.google.com/file/d/0B8Dg3PBX90KNWG5KQXNQOFlBLU1JWWVONkN1UFpnbUR6Y0cw/view?pref=2&pli=1 - Using Stochastic RNN for temporal anomaly detection
https://home.zhaw.ch/~dueo/bbs/files/vae.pdf - cover math
https://arxiv.org/pdf/1401.4082v3.pdf - Rezende - Other Original VAE paper
Code Review -
https://github.com/oduerr/dl_tutorial/blob/master/tensorflow/vae/vae_demo.ipynb
https://github.com/oduerr/dl_tutorial/blob/master/tensorflow/vae/vae_demo-2D.ipynb
Papers:
http://arxiv.org/pdf/1410.5401v2.pdf - Neural Turing Machines - Graves et. al.
https://arxiv.org/pdf/1605.06065v1.pdf - One Shot Learning - DeepMind
Code:
http://icml.cc/2016/reviews/839.txt
https://github.com/brendenlake/omniglot
https://github.com/tristandeleu/ntm-one-shot
https://github.com/MLWave/extremely-simple-one-shot-learning
Papers - Using VAE for anomaly detection
https://arxiv.org/pdf/1411.7610.pdf - Stochastic Recurrent Networks
https://drive.google.com/file/d/0B8Dg3PBX90KNWG5KQXNQOFlBLU1JWWVONkN1UFpnbUR6Y0cw/view?pref=2&pli=1 - Using Stochastic RNN for temporal anomaly detection
Papers to read:
http://www.thespermwhale.com/jaseweston/ram/papers/paper_16.pdf
http://snowedin.net/tmp/Hochreiter2001.pdf -
Comments / Code
http://icml.cc/2016/reviews/839.txt
https://github.com/brendenlake/omniglot
https://github.com/tristandeleu/ntm-one-shot
https://github.com/MLWave/extremely-simple-one-shot-learning
https://www.periscope.tv/hugo_larochelle/1ypJdnPRYEoKW
Papers to read:
http://arxiv.org/pdf/1312.6114v10.pdf - variational autoencoders - U of Amsterdam - Kingma and Welling
http://arxiv.org/pdf/1310.8499v2.pdf - deep autoregressive networks - deep mind
https://arxiv.org/abs/1606.05908 - tutorial on vae
Commentaries/Code
https://jmetzen.github.io/2015-11-27/vae.html - metzen - code and discussion
http://blog.keras.io/building-autoencoders-in-keras.html - chollet - discusses different autoencoders, gives keras code.
Recurrent network for image generation - Deep Mind
https://arxiv.org/pdf/1502.04623v2.pdf
Background and some references cited
http://blog.evjang.com/2016/06/understanding-and-implementing.html - blog w. code for VAE
http://arxiv.org/pdf/1312.6114v10.pdf - Variational Auto Encoder
https://jmetzen.github.io/2015-11-27/vae.html - tf code for variational auto-encoder
https://www.youtube.com/watch?v=P78QYjWh5sM
https://arxiv.org/pdf/1401.4082.pdf - stochastic backpropagation and approx inference - deep mind
http://www.cs.toronto.edu/~fritz/absps/colt93.html - keep neural simple by minimizing descr length - hinton
https://github.com/vivanov879/draw - code
Recurrent models of visual attention - Deep Mind
https://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention.pdf
http://arxiv.org/pdf/1410.5401v2.pdf - Neural Turing Machines - Graves et. al.
https://arxiv.org/pdf/1605.06065v1.pdf - One Shot Learning - DeepMind
http://www.shortscience.org/paper?bibtexKey=journals/corr/1605.06065 - Larochell comments on One-Shot paper
https://github.com/shawntan/neural-turing-machines - Code
https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cp4ecce - schmidhuber's comments
http://www.thespermwhale.com/jaseweston/ram/papers/paper_16.pdf
http://snowedin.net/tmp/Hochreiter2001.pdf -
Reviews:
http://icml.cc/2016/reviews/839.txt
Code
https://github.com/brendenlake/omniglot
https://github.com/tristandeleu/ntm-one-shot
https://github.com/MLWave/extremely-simple-one-shot-learning
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning:
http://arxiv.org/pdf/1602.07261v1.pdf
Visualizing and Understanding RNN:
https://arxiv.org/pdf/1506.02078v2.pdf
Google inception paper - origin of 1x1 convolution layers
http://arxiv.org/pdf/1409.4842v1.pdf
Image segmentation with deep encoder-decoder
https://arxiv.org/pdf/1511.00561.pdf
Compressed networks, reducing flops by pruning
https://arxiv.org/pdf/1510.00149.pdf
http://arxiv.org/pdf/1602.07360v3.pdf
Word2Vec meets LDA:
http://arxiv.org/pdf/1605.02019v1.pdf - Paper
https://twitter.com/chrisemoody - Chris Moody's twiter with links to slides etc.
http://qpleple.com/topic-coherence-to-evaluate-topic-models/ - writeup on topic coherence
https://arxiv.org/pdf/1603.05027v2.pdf - Update on microsoft resnet - identity mapping
http://gitxiv.com/posts/MwSDm6A4wPG7TcuPZ/recurrent-batch-normalization - batch normalization w. RNN
Go playing DQN - AlphaGo
https://gogameguru.com/i/2016/03/deepmind-mastering-go.pdf
https://m.youtube.com/watch?sns=em&v=pgX4JSv4J70 - video of slide presentation on paper
https://en.m.wikipedia.org/wiki/List_of_Go_games#Lee.27s_Broken_Ladder_Game - Handling "ladders" in alphgo
https://en.m.wikipedia.org/wiki/Ladder_(Go) - ladders in go
The Paper
http://arxiv.org/pdf/1512.03385v1.pdf
References:
http://arxiv.org/pdf/1603.05027v2.pdf - Identity mapping paper
Code:
https://keunwoochoi.wordpress.com/2016/03/09/residual-networks-implementation-on-keras/ - keras code
https://github.com/ry/tensorflow-resnet/blob/master/resnet.py - tensorflow code
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/skflow/resnet.py
The Paper
https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
http://gitxiv.com/posts/MwSDm6A4wPG7TcuPZ/recurrent-batch-normalization - Batch Normalization for RNN
The Paper https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf)
Related references:
This adds 'soft' and 'hard' attention and the 4 frames are replaced with an LSTM layer:
http://gitxiv.com/posts/NDepNSCBJtngkbAW6/deep-attention-recurrent-q-network
http://home.uchicago.edu/~arij/journalclub/papers/2015_Mnih_et_al.pdf - Nature Paper
http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html - videos at the bottom of the page
http://llcao.net/cu-deeplearning15/presentation/DeepMindNature-preso-w-David-Silver-RL.pdf - David Silver's slides
http://www.cogsci.ucsd.edu/~ajyu/Teaching/Cogs118A_wi09/Class0226/dayan_watkins.pdf
http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html - David Silver
Implementation Examples:
http://www.danielslater.net/2016/03/deep-q-learning-pong-with-tensorflow.html
The Paper
"Gated RNN" (http://arxiv.org/pdf/1502.02367v4.pdf
-Background Material
http://arxiv.org/pdf/1506.00019v4.pdf - Lipton's excellent review of RNN
http://www.nehalemlabs.net/prototype/blog/2013/10/10/implementing-a-recurrent-neural-network-in-python/ - Discussion of RNN and theano code for Elman network - Tiago Ramalho
http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf - Hochreiter's original paper on LSTM
https://www.youtube.com/watch?v=izGl1YSH_JA - Hinton video on LSTM
-Skylar Payne's GF RNN code
https://github.com/skylarbpayne/hdDeepLearningStudy/tree/master/tensorflow
-Slides
https://docs.google.com/presentation/d/1d2keyJxRlDcD1LTl_zjS3i45xDIh2-QvPWU3Te29TuM/edit?usp=sharing
https://github.com/eadsjr/GFRNNs-nest/tree/master/diagrams/diagrams_formula
http://www.computervisionblog.com/2016/06/deep-learning-trends-iclr-2016.html
https://indico.io/blog/iclr-2016-takeaways/