Probabilistic Topic Models

Mixture of Unigram Language Models.

UNDERSTANDING FROM THE VERY BASIC

While more and more texts are available online, we simply do not have the human power to read and study them. We need new computational tools to help organize, search, and understand these vast amounts of information. To this end, machine learning researchers have developed probabilistic topic modeling, a suite of algorithms that aim to discover and annotate large archives of documents with thematic information. Topic modeling algorithms are statistical methods that analyze the words of the original texts to discover the themes.These algorithms do not require any prior annotations or labeling of the documents the topics emerge from the analysis of the original texts. Topic modeling enables us to organize and summarize electronic archives at a scale that would be impossible by human annotation.

Lets consider a document “Text mining paper” denoted by d . the document consists a plethora of vocabulary .There are some words which uniquely represents or tell us about the document content like “text”, “association”, “clustering ”, “computer” and some very common words like determiners “the”, “a”, “an”, “this” , “that” ,“those” which apart from grammatical point of view ,are not that important for our document .So our aim is to separate those unique words from common words.

Lets start with assigning each word a probability of :

There are a number of probability distributions that can be used to generate the probability of each word in the document, however here the probability is computed by , dividing each word’s count by the total count of all words in the document.

SO HOW TO GET RID OF COMMON WORDS ?

Well lets consider two distributions and name them as “background topic ” and “topic” and imagine them as two bags. Each distribution contains both topic as well as common words with their respective probabilities but the only difference is one distribution assigns a higher probability to a particular word as compared to other distribution .What this means is “text” is…

Related posts:

The British first entered India in 1599 AD and started their company after they received the royal charter from Queen Elizabeth I on December 31 1600 AD permitting them to trade in the east. In 1608…

Era o terceiro round e as pessoas que assistiam estavam todas em pé, tentando ver por ângulos melhores o que o combate proporcionava. Um deles tentava seguir uma estratégia de longa distância…

I fell into the rabbit hole of professional sports banter. This mainly consisted of NBA discussion, but I ended up reading NFL disagreements. I used twitter as my medium. I think this is a good…