machine learning Archives | Page 7 of 8 | A Blog From Human-engineer-being A Blog From Human-engineer-being

Posts tagged with: machine learning

Sep 20, 2013 |

1 comment

What is special about rectifier neural units used in NN learning?

Sigmoid unit :

$f(x) = frac{1}{1+exp(-x)}$

Tanh unit:

$f(x) = tanh(x)$

Rectified linear unit (ReLU):

$f(x) = \sum_{i=1}^{\inf} \sigma (x - i + 0.5) \approx log(1 + e^{x})$

we call;

$\sum_{i=1}^{inf} sigma (x - i + 0.5)$

as stepped sigmoid

$log(1 + e^{x})$

as softplus function

The softplus function can be approximated by max function (or hard max ) ie

$max( 0, x + N(0,1))$

. The max function is Continue Reading

Aug 22, 2013 |

0 comments

Some possible Matrix Algebra libraries based on C/C++

I’ve gathered the following from online research so far:

I’ve used Armadillo a little bit, and found the interface to be intuitive enough, and it was easy to locate binary packages for Ubuntu (and I’m assuming other Linux distros). I haven’t compiled it from source, but my hope is that it wouldn’t be too difficult. It meets most of my design criteria, and uses dense linear algebra. It can call LAPACK or MKL routines.

I’ve heard good things about Eigen, but haven’t used it. It claims to be fast, uses templating, and supports dense linear algebra. It doesn’t have LAPACK or BLAS as a dependency, but appears to be able to do everything that LAPACK can do (plus some things LAPACK can’t). A lot of projects use Eigen, Continue Reading

C linear algebra machine learning

Nov 1, 2012 |

0 comments

Some possible ways to faster Neural Network Backpropagation Learning #1

Using Stochastic Gradient instead of Batch Gradient
Stochastic Gradient:

faster
more suitable to track changes in each step
often results with better solution – it may finds different ways to different local minimums on cost function due to it fluctuation on weights –
Most common way to implement NN learning.

Batch Gradient:

Analytically more tractable for the way of its convergence
Many acceleration techniques are suited to Batch L.
More accurate convergence to local min. – again because of the fluctuation on weights in Stochastic method –

Shuffling Examples

give the more informative instance to algorithm next as the learning step is going further – more informative instance means causing more cost or being unseen –
Do not give successively instances from same class.

Transformation of Inputs

Mean normalization of input variables around zero mean
Scale input variables so that covariances are about the same unit length
Diminish correlations between features as much as possible – since two correlated input may result to learn same function by different units that is redundant –

machine learning neural network

Oct 28, 2012 |

0 comments

Data-Driven Enhancement of Facial Attractiveness

computer vision machine learning video

Oct 16, 2012 |

0 comments

ruby script – download all the Machine Learning Coursera Slides

require 'net/http'

no = [1,2,3,4,5,6,7,8,9]
no.each do |n|
    Net::HTTP.start('d19vezwu8eufl6.cloudfront.net') do |http|
        resp = http.get("/ml/docs%2Fslides%2FLecture#{n}.pptx")
        open("Lecture#{n}.pptx", "w") do |file|
            file.write(resp.body)
        end
    end
end

coursera machine learning ruby script

Oct 15, 2012 |

0 comments

Anomaly detection and a simple algorithm with probabilistic approach.

What is anomaly detection? It is the way of detecting a outlier data point among the other points that have a some kind of logical distribution. Outlier one is also anomalous point (Figure 1)

Figure 1

What are the applications?

Fraud user activity detection – it is a way of detecting hacker activities on web applications or network connections by considering varying attributes of the present status. For example , an application can keep track of the user’s inputs to website and the work load that he proposes to system. Considering these current attribute values detection system decide a particular fraud action and kick out the user if there is.
Data center monitoring – You might be governing a data center with vast amount of computers so it is really hard to check each computer regularly for any flaw. A anomaly detection system might be working by considering network connection parameters of the computers, CPU and Memory Loads, it detect any problem on computer. Continue Reading

algorithm anomaly detection machine learning tutorial

Oct 4, 2012 |

0 comments

Perceptron learning rule intuition (my post on Quora)

Read Quote of Eren Golge’s answer to Machine Learning: What is the intuition behind Perceptron weight update formula w = w + yx? on Quora

machine learning neural network perceptron

Sep 24, 2012 |

0 comments

How K-means clustering works

K-means is the most primitive and easy to use clustering algorithm (also a Machine Learning algorithm).
There are 4 basic steps of K-means:

Choose K different initial data points on instance space (as initial centroids) – centroid is the mean points of the clusters that overview the attributes of the classes-.
Assign each object to the nearest centroid.
After all the object are assigned, recalculate the centroids by taking the averages of the current classes (clusters)
Do 2-3 until centroid are stabilized.

Caveats for K-means:

Although it can be proved that the procedure will always terminate, the k-means algorithm does not necessarily find the most optimal configuration, corresponding to the global objective function minimum.
The algorithm is also significantly sensitive to the initial randomly selected cluster centres. The k-means algorithm can be run multiple times to reduce this effect.

Here is the basic animation to show the intuition of K-means.

algorithm animation gif k-means machine learning

Sep 11, 2012 |

0 comments

What is Bayes' Theorem?

machine learning probability video

Sep 11, 2012 |

0 comments

What are the needs of Machine Learning?

–Convexity, including convex optimization and formulation of problems as convex programs. Two important subsets of this are linear programming and proximal gradient-style optimization algorithms and formulations, which have a ridiculously vast array of applications for industrial engineering and machine learning.
–Probabilistic modeling and inference: Graphical models and max-entropy models are the most important, and have a vast array of applications in machine learning and more structured statistical modeling. Markov Chain Monte Carlo is a terrific and amazing algorithm with a great special case called Gibbs sampling – they both present almost generic methods of Continue Reading

machine learning

Pages:‹1 2 3 4 5 678 ›