Popularity
3.3
Growing
Activity
0.0
Stable
16
1
6

Monthly Downloads: 65
Programming language: Elixir
License: The Unlicense
Latest version: v0.1.2

tfidf alternatives and similar packages

Based on the "Algorithms and Data structures" category.
Alternatively, view tfidf alternatives based on common mentions on social networks and blogs.

Do you think we are missing an alternative of tfidf or a related project?

Add another 'Algorithms and Data structures' Package

README

Travis CI Build Status

Tfidf

An Elixir implementation of tf-idf

Based on the blog post by Steven Loria

What is tf-idf?

tf–idf, short for term frequency–inverse document frequency, is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. It is often used as a weighting factor in information retrieval and text mining.

tf-idf on Wikipedia

Installation

defp deps do
  [{:tfidf, "~> 0.1.0"}]
end

Usage

Tfidf.calculate(word, text, corpus, tokenize_fn \\ &tokenize(&1))

Calculates the tf-idf for a given word within a text and a corpus (List) of texts.

iex> Tfidf.calculate("dog", "nice dog dog", ["dog hat", "dog", "cat mat", "duck"])
0.19178804830118723

An optional tokenizer function can be passed as the last argument to replace the default tokenizer:

iex> Tfidf.calculate("dog", "nice,dog,dog", ["dog,hat", "dog", "cat,mat", "duck"], &String.split(&1, ","))
0.19178804830118723

=====

Tfidf.calculate(word, tokenized_text, corpus)

Calculates the tf-idf for a given word within a pre-tokenized list and a corpus comprised of pre-tokenized lists.

iex> Tfidf.calculate("dog", ["nice", "dog", "dog"], [["dog", "hat"], ["dog"], ["cat", "mat"], ["duck"]])
0.19178804830118723

=====

Tfidf.calculate_all(text, corpus, tokenize_fn \\ &tokenize(&1))

Calculates the tf-idf for all words in a given text, returns a list of {word, score} tuples.

iex> Tfidf.calculate_all("nice dog", ["dog hat", "dog", "cat mat", "duck"])
[{"nice", 0.6931471805599453}, {"dog", 0.14384103622589042}]

As with Tfidf.calculate/4 an optional tokenizer function can be passed as the last argument. This will be used in place of the default tokenizer.

iex> Tfidf.calculate_all("nice,dog", ["dog,hat", "dog", "cat,mat", "duck"], &String.split(&1, ","))
[{"nice", 0.6931471805599453}, {"dog", 0.14384103622589042}]