See the latest book content here.

References

Key papers in the development of deep learning (1958 - 2017)

1958 (Perceptron): The perceptron: a probabilistic model for information storage and organization in the brain, by Frank Rosenblatt.
1982 (Hopfield nets): Neural networks and physical systems with emergent collective computational abilities, by John Hopfield.
1998: (Convolutional nets): Gradient based learning applied to document recognition, by Yann Lecun, Leon Bottou, Yoshua Bengio and Patrick Haffne.
2012 (AlexNet): ImageNet Classification with Deep Convolutional Neural Networks, by Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton.
2013: (Inner layers): Visualizing and Understanding Convolutional Networks, by Matthew Zeiler and Rob Fergus
2013: (Deep RL): Playing Atari with Deep Reinforcement Learning by Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra and Martin Riedmiller.
2014: (ADAM): Adam: A method for stochastic optimization, by Diederik Kingma and Jimmy Ba.
2014: (GAN): Generative adversarial nets, by Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville and Yoshua Bengio.
2015 (VGG): Very deep convolutional networks for large-scale image recognition, by Karen Simonyan and Andrew Zisserman.
2015 (Inception): Going deeper with convolutions, by Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke and Andrew Rabinovich.
2015 (ResNet): Deep residual learning for image recognition, by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun.
2016 (Deep RL vs. Go): Mastering the game of Go with deep neural networks and tree search, by David Silver, Aja Huang, Chris Maddison and others.
2017 (Transformers): Attention is all you need, by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser and Illia Polosukhin.

Key deep learning books and online resources

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
Practical Deep Learning for Coders by fast.ai.
Neural networks and deep learning by Charu Aggarwal.
Neural networks and deep learning by Michael Nielsen.

Futher deep learning papers

2009: A Survey on Transfer Learning by Sinno Jialin Pan and Qiang Yang.
2012: Practical recommendations for gradient-based training of deep architectures by Yoshua Bengio.
2012: Random Search for Hyper-Parameter Optimization by Bergstra and Bengio.
2013: Overfeat: Integrated recognition, localization and detection using convolutional network by Pierre Sermanet, David Eigen, Xiang Zhang, Micha{"e}l Mathieu, Rob Fergus, and Yann LeCun.
2013: The Synthetic Minority Over-sampling TEchnique by Rok Blagus and Lara Lusa.
2013: Speech recognition with deep recurrent neural network by Alex Graves, Abdel-rahman Mohamed and Geoffrey Hinton.
2014: How transferable are features in deep neural networks? by Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson.
2014: Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation by Cho et al.
2014: Sequence to Sequence Learning with Neural Networks by Sutskever et al.
2014: On the Properties of Neural Machine Translation: Encoder-Decoder Approaches" y Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau and Yoshua Bengio.
2014: Rich feature hierarchies for accurate object detection and semantic segmentation by Ross Girshick, Jeff Donahue, Trevor Darrell and Jitendra Malik.
2015: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun.
2015: Unsupervised representation learning with deep convolutional generative adversarial networks by Alec Radford, Luke Metz and Soumith Chintala.
2015: Unsupervised Domain Adaptation by Backpropagation by Ganin and Lemppitsky.
2015: A neural algorithm of artistic style by Leon Gatys, Alexander Ecker and Matthias Bethge.
2015: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. by Ioffe, Sergey, and Christian Szegedy.
2014: Dropout: a simple way to prevent neural networks from overfitting by Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Salakhutdinov, Ruslan.
2014: Conditional Generative Adversarial Nets by Mehdi Mirza and Simon Osindero.
2015: An Empirical Exploration of Recurrent Network Architectures) by Jozefowicz et al.
2015: Deep Learning by Yann LeCun, Yoshua Bengio and Geoffrey Hinton.
2016: Neural Machine Translation by Jointly Learning to Align and Translate by Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio.
2016: Unsupervised representation learning with deep convolutional generative adversarial networks, by Alec Radford, Luke Metz, and Soumit Chintala.
2016: You only look once: Unified, real-time object detection by Joseph Redmon, Santosh Divvala, Ross Girshick and Ali Farhadi.
2016: Improved techniques for training gans by Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.
2017: Image-to-image translation with conditional adversarial networks by Phillip Isola, Jun-Yan Zhu, Tinghui Zhou and Alexei Efros.
2017: A deep learning framework for financial time series using stacked autoencoders and long-short term memory by Wei Bao,Jun Yue and Yulei Rao.
2017: Gans trained by a two time-scale update rule converge to a local nash equilibrium by Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.
2017: Wasserstein generative adversarial networks by Martin Arjovsky, Soumith Chintala and L{'e}on Bottou.
2017: Why does deep and cheap learning work so well? by Henry Lin, Max Tegmark, and David Rolnick.
2018: Large scale GAN training for high fidelity natural image synthesis by Andrew Brock, Jeff Donahue, and Karen Simonyan.
2018: Synthesizing robust adversarial examples by Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok.
2018: Recurrent Neural Network for Predicting Transcription Factor Binding Sites by Zhen Shen, Wenzheng Bao and De-Shuang Huang.
2018: How does batch normalization help optimization? by Santurkar, Shibani, et al.
2018: Are GANs Created Equal? A Large-Scale Study by Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly and Olivier Bousquet.
2028: The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation by Chen et al.
2019: A style-based generator architecture for generative adversarial networks by Tero Karras, Samuli Laine and Timo Aila.
2019: Attention in Natural Language Processing by Andrea Galassi, Marco Lippi and Paolo Torroni.
2020: Universal differential equations for scientific machine learning by Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan and Alan Edelman.
2020: A Survey of Deep Learning Techniques for Neural Machine Translation by Shuoheng Yang, Yuxin Wang and Xiaowen Chu.

Other useful literature (not focusing just on deep learning)

Data Science and Machine Learning: Mathematical and Statistical Methods by Dirk Kroese, Zdravko Botev, Thomas Taimre and Slava Vaisman.
Statistics with Julia: Fundamentals for Data Science, Machine Learning and Artificial Intelligence by Yoni Nazarathy and Hayden Klok.
The R Software: Fundamentals of Programming and Statistical Analysis by Pierre Lafaye de Micheaux, , Rémy Drouilhet and Benoit Liquet.
Algorithms for optimization by Mykel Kochenderfer and Tim Wheeler.

More references

Boyd, Stephen, and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511804441.

François-Lavet, Vincent, Peter Henderson, Riashat Islam, Marc G. Bellemare, and Joelle Pineau. 2018. “An Introduction to Deep Reinforcement Learning.” Foundations and Trends® in Machine Learning 11 (3-4): 219–354. https://doi.org/10.1561/2200000071.

Kochenderfer, Mykel J., and Tim A. Wheeler. 2019. Algorithms for Optimization. The MIT Press.

Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. “Playing Atari with Deep Reinforcement Learning.” http://arxiv.org/abs/1312.5602.

Nesterov, Yurii. 2004. Introductory Lectures on Convex Optimization. Vol. 87. Applied Optimization. Kluwer Academic Publishers, Boston, MA. https://doi.org/10.1007/978-1-4419-8853-9.

Puterman, Martin L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley & Sons, Inc., New York.

Sundaram, Rangarajan K. 1996. A First Course in Optimization Theory. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511804526.

Watkins, Christopher J. C. H., and Peter Dayan. 1992. “Q-Learning” 8(3): 279–92.

Page built: 2021-03-04 using R version 4.0.3 (2020-10-10)

See the latest book content here.

References

Key papers in the development of deep learning (1958 - 2017)Copy link

Key deep learning books and online resourcesCopy link

Futher deep learning papersCopy link

Other useful literature (not focusing just on deep learning)Copy link

More referencesCopy link

Key papers in the development of deep learning (1958 - 2017)

Key deep learning books and online resources

Futher deep learning papers

Other useful literature (not focusing just on deep learning)

More references