Note : I regularly update this list.
Machine Learning 101:
I. Introduction to Machine Learning
- http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/MLPAGES/mltut.htm
- http://jeremykun.com/2012/08/04/machine-learning-introduction/
- http://www.omidrouhani.com/research/machinelearning/html/machinelearning.htm
- http://www.youtube.com/playlist?list=PLD63A284B7615313A (cal tech class)
II. Linear Regression
- http://en.wikipedia.org/wiki/Linear_regression
- http://www.youtube.com/watch?v=ExVhaN36jBs
- http://en.wikipedia.org/wiki/Simple_linear_regression
- http://www.youtube.com/watch?v=ocGEhiLwDVc
III) Linear Algebra
- http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/Syllabus/
- https://www.khanacademy.org/math/linear-algebra
- online text
- http://joshua.smcvt.edu/linearalgebra/book.pdf
- – see http://joshua.smcvt.edu/linearalgebra/ for usage rights
V) Linear Regression with Multiple Variables
– Gradient Descent
- http://en.wikipedia.org/wiki/Gradient_descent
- http://www.youtube.com/watch?v=umAeJ7LMCfU (discusses above wiki article)
- http://www.youtube.com/watch?v=Dgn1ssi2p40
– Optimization
IV) Octave Tutorial
VI) Logistic Regression (LR)
- http://en.wikipedia.org/wiki/Logistic_regression
- http://alias-i.com/lingpipe/demos/tutorial/logistic-regression/read-me.html
- http://www.ats.ucla.edu/stat/sas/library/logistic.pdf
- http://www.youtube.com/watch?v=-Z2a_mzl9LM&feature=c4-overview&playnext=1&list=TLIxwITi7ngG0 (refers to LR as a classifier)
VII) Regularization
- http://en.wikipedia.org/wiki/Regularization_(mathematics)
- http://solon.cma.univie.ac.at/regul.html
- http://www.di.ens.fr/~fbach/ecml2010tutorial/ecml_tutorial_part1.pdf
overview using advanced math
VIII and IX) Neural Networks
- http://www.youtube.com/watch?v=KuPai0ogiHk
- http://www.youtube.com/watch?v=Ih5Mr93E-2c&list=PLD63A284B7615313A&index=10
– backpropagation
- http://www.youtube.com/watch?v=aVId8KMsdUU
- http://www.speech.sri.com/people/anand/771/html/node37.html
- http://blog.zabarauskas.com/backpropagation-tutorial/
XI) Machine Learning System Design
Precision, recall, accuracy, …
- http://en.wikipedia.org/wiki/Precision_and_recall
- https://en.wikipedia.org/wiki/Accuracy_and_precision
- http://stats.stackexchange.com/questions/34193/how-to-choose-an-error-metric-when-evaluating-a-class…
- http://www.cs.cornell.edu/courses/cs578/2003fa/performance_measures.pdf
XII) Support Vector Machines
- http://www.cs.ucf.edu/courses/cap6412/fall2009/papers/Berwick2003.pdf
- http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf
- http://www.youtube.com/watch?v=eHsErlPJWUU
- http://web.mit.edu/zoya/www/SVM.pdf
XIII) Clustering
- http://en.wikipedia.org/wiki/Cluster_analysis
- http://en.wikipedia.org/wiki/K-means_clustering
- http://www.youtube.com/watch?v=0MQEt10e4NM&feature=c4-overview&playnext=1&list=TLT3EED0Azl4Y
XIV) Dimensionality Reduction
- http://en.wikipedia.org/wiki/Dimensionality_reduction
- http://research.cs.tamu.edu/prism/lectures/iss/iss_l10.pdf
- http://www.math.uwaterloo.ca/~aghodsib/courses/f06stat890/readings/tutorial_stat890.pdf
- http://www.youtube.com/watch?v=EHIZ7Pk1XVY
- http://www.youtube.com/watch?v=mz618Tesra4
XV) Anomaly Detection
– Google Analytics http://www.google.com/analytics/
– anomaly detection with Google Analytics (example)
Must purchase this article (I did not purchase but appears to be good) http://www.sciencedirect.com/science/article/pii/S138912860700062X
– Gaussian distribution
- http://www.youtube.com/watch?v=4uiJoYVPmMw (no math)
- https://en.wikipedia.org/wiki/Normal_distribution
- http://www.r-tutor.com/elementary-statistics/probability-distributions/normal-distribution
- https://en.wikipedia.org/wiki/Multivariate_normal_distribution
XVI) Recommender Systems
- http://pages.cs.wisc.edu/~beechung/icml11-tutorial/
- http://ijcai-11.iiia.csic.es/files/proceedings/Tutorial%20IJCAI%202011%20Gesamt.pdf
- http://muricoca.github.io/crab/tutorial.html (using Python)
– Collaborative Filtering
- www.cs.cmu.edu/~wcohen/collab-filtering-tutorial.ppt
XVII) Large Scale Machine Learning
- http://i.stanford.edu/~ullman/pub/ch12.pdf
- http://www.sanjivk.com/EECS6898/ (introduction to class)
- (lectures) http://www.sanjivk.com/EECS6898/lectures.html
- http://techtalks.tv/talks/introduction-5/57923/
– stochastic gradient descent
- http://en.wikipedia.org/wiki/Stochastic_gradient_descent
- http://www.youtube.com/watch?v=HvLJUsEc6dw (visualization)
- http://work.caltech.edu/library/101.html
- http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-832-underactuated-robotics-…
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-832-underactuated-robotics-…
– parallelized stochastic gradient descent
– recursive partitioning:
XVIII) Reinforcement Learning
Machine Learning 201:
- Advanced Machine Learning Course (CMU)
- Lecture 1: Machine Learning With Scikit-Learn
- Lecture 2: Machine Learning With Scikit-Learn
- Lecture 3: Machine Learning from the Boston Python User Group
- Andrew Ng’s Standford ML Class
- An Introduction to Machine Learning
- Andrew Ng’s Coursera Class Wiki
- Koller’s PGM course on Coursera (requires solid prob. background)
- The Machine Learning Library
- JMLR
- CMU Google Slides
- NN Course
- Statistical Machine Learning Course by Ryan Tibshirani & Larry Wasserman (NEW)
Online Lectures:
- Andrew Ng’s Coursera Class Wiki
- Koller’s PGM course on Coursera (requires solid probability background)
- Pedro Domingo’s Coursera Class
- Recommender Systems at Coursera
- Mining Massive Dataset at Coursera
- Hinton’s NN lectures at Coursera
- Scalable Machine Learning by Alex Smola
- Advance Optimization and Randomized Methods by Alex Smola
- Neural Networks by
- Harvard Advance Machine Learning
Deep Learning:
- Deep Learning – Very wide grasp resource about everything
- Juergen Schmidhuber’s home page – Different perspectives of NNs with theoretical view as well
- Lecun’s Lecture Materials
- Home Page of Geoffrey Hinton – And the Father of DL
- Neural Network FAQ, part 1 of 7: Introduction – General sense NN FAQ
- Page on lear.inrialpes.fr – INRIA Deep Learning Notes tutorial
- Page on nyu.edu:21991 – very detailed examples on real datasets
- Hinton’s NN lectures at Coursera
- Neural Networks by
- Practical Deep Learning talk by Adam Gibson for DeepLearning4J
- Deep Learning talk by Andrew Ng.
- Very Good Introductory Material for the Basics of Deep Learning with a example code.
- Topic-wise Deep Learning Bibliography by memkite (new)
- IPAM deep learning and feature learning summer school (new)
Sparse Coding:
Useful for Kaggle
Some good articles on working with the command line:
- command line nuggets for data science (article focuses on unix but all will work in linux bash)
- intro to the command line
- 7 Command Line Tools for Data Scientists
Jacobian Iteration for Singular Value Decomposition:
Mathematics, Statistical Theory and Probability Theory:
Methods of Optimization:
- Gradient Descent
- Basic Steepest Decent
- Newton’s Method in Optimization
- CRAN Optimization and Mathematical Programming Task View
- MIT OCW Optimization Methods
- Boyd Optimization
- Boyd Solutions Manual
- Convex Optimization in R
Theoretical Computer Science:
Random but Important Things:
- A Little Stats Cheat Sheet. Pretty basic stuff but it is a nice quick reference.
- Onto Data Scaling and Standardization (new)
- A great list of Machine Learning Tools (new)
- A Cheat Sheet for Distance Measures
- Proof wiki list of symbols with LaTex code!!
- Large set of free ML related e-books (new)
R:
Python:
Fortran:
- Fortran for Beginners
- Fortran 77 Stanford Tutorial
- Professional Programmer’s Guide to Fortran 77
- BLAS
- Fortran 77 Intrinsic Functions
Miscellaneous Links:
- Awesome Machine Learning – includes large set of ML related tools divided into programming languages.
- Top 100 Video ML Lectures Talk
Credits goes to Resources
I added some of my places to that list as well.