A successful AI agent should communicate. It is all about language. It should understand and explain itself in words in order to communicate us. All of these spark with the “meaning” of words which the atomic part of human-wise communication. This is one of the fundamental problems of Natural Language Processing (NLP).
“meaning” is described as “the idea that is represented by a word, phrase, etc. How about representing the meaning of a word in a computer. The first attempt is to use some kind of hardly curated taxonomies such as WordNet. However such hand made structures not flexible enough, need human labor to elaborate and do not have semantic relations between words other then the carved rules. It is not what we expect from a real AI agent.
Then NLP research focused to use number vectors to symbolize words. The first use is to donate words with discrete (one-hot) representations. That is, if we assume a vocabulary with 1K words then we create a 1K length 0 vector with only one 1 representing the target word. Continue Reading