A list of more advanced resources for graphical models is also available.

Online Textbooks & Tutorials

• The Elements of Statistical Learning. T. Hastie, R. Tibshirani, & J. Friedman, Springer, 2009.
• Information Theory, Inference, and Learning Algorithms. D. MacKay, Cambridge University Press, 2003.
• Convex Optimization. S. Boyd & L. Vandenberghe, Cambridge University Press, 2004.
• A Brief Introduction to Graphical Models and Bayesian Networks. Kevin Murphy, University of British Columbia, 1998.
• Probabilistic Models for Unsupervised Learning. Zoubin Ghahramani and Sam Roweis, NIPS 1999.

Reference Print Textbooks

• Pattern Recognition and Machine Learning. C. Bishop, Springer, 2007.
• Pattern Classification. R. Duda, P. Hart, & D. Stork, Wiley, 2001.
• Machine Learning. T. Mitchell, McGraw Hill, 1997.
• Introduction to Probability. D. Bertsekas & J. Tsitsiklis, Athena Scientific, 2008.
• Bayesian Data Analysis. A. Gelman, J. Carlin, H. Stern, & D. Rubin, CRC Press, 2003.
• Monte Carlo Statistical Methods. C. Robert & G. Casella, Springer, 2004.

Courses

• Probabilistic and Unsupervised Learning. Gatsby Unit, Yee Whye Teh and Maneesh Sahani, 2009.
• Machine Learning. MIT 6.867, Tommi Jaakkola and Michael Collins, Fall 2010.
• Machine Learning. Stanford University CS 229, Andrew Ng, Fall 2010.
• Machine Learning. University of Pennsylvania CIS 520, Ben Taskar, Fall 2010.
• Statistical Machine Learning. CMU 10-702, John Lafferty and Larry Wasserman, Spring 2010.
• Statistical Learning Theory: Graphical Models. UC Berkeley EECS 281a, Martin Wainwright, Fall 2008.

Software

• Matlab Machine Learning Toolboxes. Especially the PMTK, Kevin Murphy, University of British Columbia.
• mloss.org. Machine learning open source software.
• Weka. Data mining software in Java.
• Bayesian Modeling and Monte Carlo Methods. Radford Neal, University of Toronto.
• Lightspeed Matlab Toolbox. Tom Minka, Microsoft Research.