Skip to content

An extremely simple Python library to perform TF-IDF document comparison.

License

Notifications You must be signed in to change notification settings

neineizhang/python-tf-idf

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 

Repository files navigation

The simplest TF-IDF library imaginable.

Add your documents as two-element lists [doc_name, [list_of_words_in_the_document]] with addDocument(doc_name, list_of_words).

table.addDocument("foo", ["alpha", "bravo", "charlie", "delta", "echo", "foxtrot", "golf", "hotel"])

Get a list of all the [doc_name, similarity_score] pairs relative to a list of words by calling similarities([list_of_words]). Resulting similarities will always be between 0.0 and 1.0, inclusive.

table.similarities(["alpha", "bravo", "charlie"])

So, for example:

import tfidf

table = tfidf.tfidf()
table.addDocument("foo", ["alpha", "bravo", "charlie", "delta", "echo", "foxtrot", "golf", "hotel"])
table.addDocument("bar", ["alpha", "bravo", "charlie", "india", "juliet", "kilo"])
table.addDocument("baz", ["kilo", "lima", "mike", "november"])

print table.similarities (["alpha", "bravo", "charlie"]) # => [['foo', 0.6875], ['bar', 0.75], ['baz', 0.0]]

Disclaimer: This library is a pretty clean example of how TF-IDF operates. However, it's totally unconcerned with efficiency (it's really just an exercise to brush up my Python skills), so you probably don't want to be using it in production. If you're looking for a more heavy-duty Python library to do information retrieval and topic modelling, I'd suggest taking a look at Gensim.

About

An extremely simple Python library to perform TF-IDF document comparison.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%