By examining close relationship between new classes and probability vectors of multiple labeling of the documents, we obtain probability distribution function to each label of documents. With the assumption of multinomial distribution over words, we apply EM algorithm to obtain the distribution. Then we apply clustering to label probabilities to mine classes.