Spring 2019:
MA 59800 Mathematical Aspects of Neural Networks
Instructor: Greg Buzzard
MWF 2:30-3:20p
Note: Mon and Fri classes will be in WALC 1121. Wed classes will be in WTHR 320 in conjunction with the Machine Learning Seminar.
![[photo]](AI.jpg)
|
Overview: Neural networks have dramatically changed the capabilities of computers to perform tasks such as image classification and processing, image generation, voice recognition, game playing, and many others. I will give an overview of the development of neural networks, describe the primary architectures and training methods, and describe some of the mathematical results associated with neural networks. Required work will include programming projects, oral presentations on research papers, and a final group project that includes a written paper and a neural network implementation. Some prior programming experience is required, as well as solid understanding of both linear algebra and probability at the level of a good undergraduate course. | |
| Image source: Proceedings of the IEEE |
Course policies
Office hours:
Monday and Thursday, 3:30-4:30, MATH 830
Text: We'll primarily follow these two books:
- Deep Learning, Ian Goodfellow, et al. Overview of many topics in neural networks.
- Deep Learning with Python, Francois Chollet. Introduction to programming neural networks using Keras. The code from this book is available here on github. This code is in the form of iPython/Jupyter notebooks. The easiest way to run them with GPU support is on the Google colaboratory. This requires a Google account. If you are unwilling to get a Google account, you will need to find some other way to set up an environment for running Jupyter notebooks. See e.g., this link.
Each Wednesday we'll participate in the Machine Learning Reading Group. Working in groups of two, each student will be required to give a 20 minute presentation on a paper related to neural networks.
Required work and approximate weighting:
See here for homework assignments.
- One presentation during the Wednesday seminar - 20 minutes, done in pairs (25%).
- Due most Fridays: brief summary of Wednesday presentations/papers - individual (45%).
- Due periodically: Programming assignments (30%).
Discussion forum: Please enroll in the Piazza discussion forum for this class. This is forum will not be moderated by the instructor but should be useful for clarifying ideas, pointing to resources, etc. Please be respectful and helpful to one another. If you haven't used Piazza before, you can go to Q&A along the top, then browse by topic along the left.
Approximate schedule (subject to change):
- Week 1: Overview of deep learning. Linear algebra (Goodfellow, Chs 1 and 2, Chollet, Chs 1 and 2).
- Week 2: Basic probability (Goodfellow, Ch 3)
- Week 3: Basics of machine learning (Goodfellow, Ch 5 and Chollet, Ch 3)
- Week 4: Basics of deep learning (Goodfellow, Ch 6 and Chollet, Ch 4)
- Week 5: Convolutional networks (Goodfellow, Ch 9 and Chollet, Ch 5)
- Week 6: Convolutional networks (Goodfellow, Ch 9 and Chollet, Ch 5)
- Week 7: Recurrent networks (Goodfellow, Ch 10 and Chollet, Chs 6 and 7)
- Week 8: Recurrent networks (Goodfellow, Ch 10 and Chollet, Chs 6 and 7)
- Week 9: Autoencoders (Goodfellow, Ch 14 and Chollet, Section 8.4)
- Week 10: Autoencoders (Goodfellow, Ch 14 and Chollet, Section 8.4)
- Week 11: Generative models (Goodfellow, Ch 20, Chollet, Ch 8)
- Week 12: Generative models (Goodfellow, Ch 20, Chollet, Ch 8)
- Week 13: Reinforcement learning (Aggarwal, Ch 9, Stanford CS 231n slides)
- Week 14: Reinforcement learning (Aggarwal, Ch 9, Stanford CS 231n slides)
- Week 15: TBA
- Week 2: Basic probability (Goodfellow, Ch 3)
Goals:
- Understand the primary elements of neural networks and the mathematical ideas underlying them.
- Develop the ability to implement neural networks using Keras.
- Develop the ability to read, understand, evaluate, and present research papers.