Bayesian Statistics
šŸ“Š

Bayesian Statistics

Created
May 2, 2023 04:42 PM
Author
Haoli Yin

Motivation:

Taking this course provides an opportunity to build a strong foundation in Bayesian statistics, which is crucial for deep learning research due to its emphasis on uncertainty quantification and principled decision-making. By learning Bayesian methods, researchers can develop more robust and interpretable models that better reflect real-world scenarios. Bayesian techniques allow for the incorporation of prior knowledge, resulting in improved performance and generalization, especially in situations with limited data. As you embark on this Bayesian deep learning odyssey, you'll be crafting trailblazing models that master intricate challenges, propelling the AI revolution forward.
Ā 

Table of Contents:

Ā 

Week 1: Introduction to Bayesian Statistics

Day 1: Difference between frequentist and Bayesian statistics
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 1
Day 2: Bayes' theorem and conditional probability
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 2
Day 3: Bayesian inference and Bayesian updating
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 3
Day 4: Prior and posterior distributions
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 4
Day 5: Conjugate priors and hierarchical models
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 5
Day 6-7: Review and practice problems
  • Reading: "Bayesian Data Analysis" by Gelman et al., review Chapters 1-5 and associated exercises
Ā 

Week 2: Bayesian Modeling

Day 1: Model selection and Bayesian model averaging
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 6
Day 2: Markov Chain Monte Carlo (MCMC) methods
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 11
Day 3: Gibbs sampling and Metropolis-Hastings algorithm
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 12
Day 4: Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS)
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 14
Day 5: Approximate Bayesian Computation (ABC)
  • Reading: "Bayesian Data Analysis" by Gelman et al., Chapter 15
Day 6-7: Review and practice problems
  • Reading: "Bayesian Data Analysis" by Gelman et al., review Chapters 6, 11, 12, 14, and 15, and associated exercises
Ā 

Week 3: Bayesian Methods in Machine Learning

Day 1: Bayesian linear regression and Bayesian logistic regression
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 3
Day 2: Gaussian processes
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 6
Day 3: Bayesian neural networks
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 5
Day 4: Bayesian model selection and Occam's razor
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 3.5
Day 5: Bayesian optimization
  • Reading: "Practical Bayesian Optimization of Machine Learning Algorithms" by Jasper Snoek, Hugo Larochelle, and Ryan P. Adams
Day 6-7: Review and practice problems
  • Review chapters and resources from Week 3
  • Practice implementing the concepts in Python using libraries like PyMC3 and GPy
Ā 

Week 4: Bayesian Methods in Deep Learning

Day 1: Introduction to variational inference
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 10
Day 2: Mean-field variational inference
  • Reading: "Pattern Recognition and Machine Learning" by Christopher Bishop, Chapter 10.1
Day 3: Variational autoencoders (VAEs)
  • Reading: "Auto-Encoding Variational Bayes" by Diederik P. Kingma and Max Welling
Day 4: Bayesian deep learning and uncertainty quantification
  • Reading: "Bayesian Deep Learning" by Yarin Gal
Day 5: Advanced topics and applications of Bayesian deep learning
  • Reading: "Deep and Hierarchical Implicit Models" by Dustin Tran, Rajesh Ranganath, and David Blei
Day 6-7: Review and practice problems
  • Review chapters and resources from Week 4
  • Practice implementing the concepts in Python using libraries like TensorFlow, PyMC3, and Keras
Ā