Answer by Franck Dernoncourt:
Let me add two pictures to Paul Hobbs answer, taken from http://work.caltech.edu/l
ectures… (Yaser Abu-Mostafa, Caltech):
Big picture of what learning is:
Answer by Franck Dernoncourt:
Let me add two pictures to Paul Hobbs answer, taken from http://work.caltech.edu/l
ectures… (Yaser Abu-Mostafa, Caltech):
Big picture of what learning is:
1. Deep learning [5] seems to be getting the most press right now. It is a form of a Neural Network (with many neurons/layers). Articles are currently being published in the New Yorker [1] and the New York Times[2] on Deep Learning.
2. Combining Support Vector Machines (SVMs) and Stochastic Gradient Decent (SGD) is also interesting. SVMs are really interesting and useful because you can use the kernel trick [10] to transform your Continue Reading
Sigmoid unit :
Tanh unit:
Rectified linear unit (ReLU):
we call;
as stepped sigmoid
as softplus function
The softplus function can be approximated by max function (or hard max ) ie
. The max function is Continue Reading
I’ve gathered the following from online research so far:
I’ve used Armadillo a little bit, and found the interface to be intuitive enough, and it was easy to locate binary packages for Ubuntu (and I’m assuming other Linux distros). I haven’t compiled it from source, but my hope is that it wouldn’t be too difficult. It meets most of my design criteria, and uses dense linear algebra. It can call LAPACK or MKL routines.
I’ve heard good things about Eigen, but haven’t used it. It claims to be fast, uses templating, and supports dense linear algebra. It doesn’t have LAPACK or BLAS as a dependency, but appears to be able to do everything that LAPACK can do (plus some things LAPACK can’t). A lot of projects use Eigen, Continue Reading
Using Stochastic Gradient instead of Batch Gradient
Stochastic Gradient:
Batch Gradient:
Shuffling Examples
Transformation of Inputs
What is anomaly detection? It is the way of detecting a outlier data point among the other points that have a some kind of logical distribution. Outlier one is also anomalous point (Figure 1)
What are the applications?
K-means is the most primitive and easy to use clustering algorithm (also a Machine Learning algorithm).
There are 4 basic steps of K-means:
Caveats for K-means:
Here is the basic animation to show the intuition of K-means.