![]() ![]() ![]() Similarities-on apparent semantic relatedness of their texts (words). Unlike modern search engines, here we only concentrate on a single aspect of possible Like to sort our nine corpus documents in decreasing order of relevance to this query. Now suppose a user typed in the query “Human computer interaction”. If you’re interested, you can read more about LSI here: Latent Semantic Indexing: Our LSI space is two-dimensional ( num_topics = 2) so there are two topics, but this is arbitrary. Second, the benefit of LSI is that enables identifying patterns and relationships between terms (in our case, words in a document) and topics. LsiModel ( corpus, id2word = dictionary, num_topics = 2 )įor the purposes of this tutorial, there are only two things you need to know about LSI.įirst, it’s just another transformation: it transforms vectors from one space to another. Dictionary ( texts ) corpus = įrom gensim import models lsi = models. split ()) texts = for document in documents ] # remove words that appear only once frequency = defaultdict ( int ) for text in texts : for token in text : frequency += 1 texts = > 1 ] for text in texts ] dictionary = corpora. From collections import defaultdict from gensim import corpora documents = # remove common words and tokenize stoplist = set ( 'for a of the and to in'.
0 Comments
Leave a Reply. |