paratize alternatives and similar packages
Based on the "Algorithms and Data structures" category.
Alternatively, view paratize alternatives based on common mentions on social networks and blogs.
-
matrex
A blazing fast matrix library for Elixir/Erlang with C implementation using CBLAS. -
simple_bayes
A Naive Bayes machine learning implementation in Elixir. -
exconstructor
An Elixir library for generating struct constructors that handle external data with ease. -
erlang-algorithms
Implementations of popular data structures and algorithms -
remodel
:necktie: An Elixir presenter package used to transform map structures. "ActiveModel::Serializer for Elixir" -
MapDiff
Calculates the difference between two (nested) maps, and returns a map representing the patch of changes. -
aja
Extension of the Elixir standard library focused on data stuctures, data manipulation and performance -
merkle_tree
:evergreen_tree: Merkle Tree implementation in pure Elixir -
bloomex
:hibiscus: A pure Elixir implementation of Scalable Bloom Filters -
DeepMerge
Deep (recursive) merge for maps, keywords and others in Elixir -
the_fuzz
String metrics and phonetic algorithms for Elixir (e.g. Dice/Sorensen, Hamming, Jaccard, Jaro, Jaro-Winkler, Levenshtein, Metaphone, N-Gram, NYSIIS, Overlap, Ratcliff/Obershelp, Refined NYSIIS, Refined Soundex, Soundex, Weighted Levenshtein) -
exmatrix
Elixir library implementing a parallel matrix multiplication algorithm and other utilities for working with matrices. Used for benchmarking computationally intensive concurrent code. -
ecto_materialized_path
Tree structure & hierarchy for ecto models -
dataframe
Package providing functionality similar to Python's Pandas or R's data.frame() -
Conrex
An Elixir implementation of the CONREC algorithm for topographic or isochrone maps. -
murmur
:speech_balloon: An implementation of the non-cryptographic hash Murmur3 -
bitmap
Bitmap implementation in Elixir using binaries and integers. Fast space efficient data structure for lookups -
Closure Table
Closure Table for Elixir - a simple solution for storing and manipulating complex hierarchies.
Collect and Analyze Billions of Data Points in Real Time
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.
Do you think we are missing an alternative of paratize or a related project?
Popular Comparisons
README
Paratize
Elixir library providing some handy parallel processing facilities that supports configuring number of workers and timeout.
This library is inspired by Parex.
Documentation
API documentation is available at http://hexdocs.pm/paratize
Adding Paratize To Your Project
To use Paratize with your projects, edit your mix.exs
file and add it as a dependency:
defp deps do
[
{:paratize, "~> x.x.x"},
]
end
Examples
Paratize is designed to run slow tasks in parallel. There are two processor implementatons, first the chunk based implementation Paratize.Chunk
and the second the worker pool based implementation Paratize.Pool
. Both modules have the same API.
parallel_exec(fun_list, task_options)
parallel_map(arg_list, fun, task_options)
parallel_each(arg_list, fun, task_options)
To execute a list of functions in parallel,
import Paratize.Pool
function_list = [
fn -> Math.fib(40) end,
fn -> :timer.sleep(5000) end,
fn -> HTTPotion.get("http://wwww.reddit.com") end
]
parallel_exec(function_list) # => [102334155, :ok, %HTTPotion.Response{body...}]
function_keyword_list = [
fib: fn -> Math.fib(40) end,
hang: fn -> :timer.sleep(5000) end,
web_request: fn -> HTTPotion.get("http://wwww.reddit.com") end
]
parallel_exec(function_keyword_list) # => [fib: 102334155, hang: :ok, web_request: %HTTPotion.Response{body...}]
To execute a map
in parallel,
(useful when results are needed for further processing)
import Paratize.Pool
slow_func = fn arg -> :timer.sleep(1000); arg + 1 end
workload = 1..100
{time, result} = :timer.tc fn -> workload |> parallel_map(slow_func) |> Enum.join(", ") end
time # => 13034452 (8 CPU cores system, running 8 workers)
To execute a each
in parallel,
(useful when resultset is large, and can be processed individually to prevent memory hog)
import Paratize.Pool
lots_of_urls |> parallel_each(fn url ->
HTTPotion.get(url) |> parse_page |> save_meta_data
end)
Task Options
Each function accepts task options to customize the parallel processing.
- size - the number of parallel workers, defaults to the number of system cores given by
:erlang.system_info(:schedulers)
- timeout - in milliseconds, the minimum time given for a function to complete, defaults to
5000
. If timeout happens, the entire parallel processing crashes withexit(:timeout,...)
. To disable timeout, use:infinity
.
Considerations
To achieve maximum parallelism, %Paratize.TaskOptions{}
size should be set to size of your workload,
alias Paratize.Pool
slow_func = fn arg -> :timer.sleep(1000); arg + 1 end
workload = 1..100
{time, result} = :timer.tc fn ->
workload |> Pool.parallel_map(slow_func, size: Enum.count(workload)) |> Enum.join(", ")
end
time # => 1004370 (Running 100 workers)
The %Paratize.TaskOptions{}
timeout
should not be relied upon for precise timing out of each workload, because it is not strictly enforced. It is an implementation detail that reasonably crashes the processor if no further work is completed after the timeout period has lapsed.
LICENSE
This software is licensed under [MIT LICENSE](LICENSE.md).
*Note that all licence references and agreements mentioned in the paratize README section above
are relevant to that project's source code only.