benchee v0.6.0 Release Notes
Release Date: 2016-11-30 // over 7 years ago-
๐ One of the biggest releases yet. Great stuff in here - more elixir like API for
Benchee.run/2
with the jobs as the primary argument and the optional options as the second argument and now also as the more idiomatic keyword list!The biggest feature apart from that is the possibility to use multiple inputs - which you all should do now as quite many functions behave differently with bigger, smaller or differently shaped inputs. Apart from that a bulk of work has gone into making and supporting benchee_html!
๐ Features (User Facing)
- New
:inputs
configuration key that allows you to specify a map from input name to input value so that each defined benchmarking job is then executed with this input. For this to work the benchmarking function is called with the appropriateinput
as an argument. Seesamples/multiple_inputs.exs
for an example. #21 - ๐ The highlevel
Benchee.run/2
is now more idiomatic elixir and takes the map of jobs as the first argument and a keyword list of options as the second (and last) argument. The old way of passing config as a map as the first argument and the jobs as the second argument still works, but might be deprecated later on #47 - Along with that
Benchee.init/1
now also accepts keyword lists of course
๐ฅ Breaking Changes (User Facing)
- ๐ The old way of providing the jobs as a list of tuples now removed, please switch to using a map from string to functions
๐ Features (Plugins)
Benchee.Utility.FileCreation
module to help with creating files from a map of multiple inputs (or other descriptors) mapping to input and aninterleave
function that spits out the correct file names especially if the:__no_input
marker is usedBenchee.System
is available to retrieve Elixir and Erlang versions but it's also already added to the suite duringBenchee.run/2
๐ฅ Breaking Changes (Plugins)
- ๐ท The structure of the output from
Benchee.Benchmark.measure/1
toBenchee.Statistics.statistics/1
has changed to accommodate the new inputs feature there is now an additional level where in a map the input name then points to the appropriate results of the jobs. When there were no inputs the key is the value returned byBenchee.Benchmark.no_input/0
.
๐ Bugfixes
- ๐ prewarming (discarding the first result due to some timer issues) during run time was removed, as it should already happen during the warmup period and would discard actual useful results especially for longer running macro benchmarks.
- ๐ท when the execution time of the benchmarking job exceeds the given
:time
it will now execute exactly once (used to be 2) #49 - ๐
run_times
are now in the order as recorded (used to be reverse) - important when wants to graph them/look at them to see if there are any anomalies during benchmarking - โ Remove elixir 1.4.0-rc.0 warnings
- New