Changelog History
Page 1
-
v1.1.0 Changes
December 09, 2020Enhancements
- Add
Flow.map_batch/2
- Add
Bug fixes
- Do not leak flows while enumerating
-
v1.0.0 Changes
February 03, 2020- Enhancements
- Require GenStage v1.0.0
- Enhancements
-
v0.15.0 Changes
October 28, 2019Enhancements
- Add on_init callback to MapReducer
- Deprecate Flow.each to avoid pitfalls
- Remove previously deprecated code
Bug fixes
- Set demand to accumulate before producer_consumer subscribe
-
v0.14.3 Changes
October 25, 2018- Bug fixes
- Don't fuse mappers into enumerables (#62)
- Trap exits to ensure event completion on shutdown
- Fix
flat_map
followed byemit_and_reduce
(#68)
- Bug fixes
-
v0.14.2 Changes
July 24, 2018- Bug fixes
- Make sure consumers added via
into_specs/3
restart the flow in case of failures
- Make sure consumers added via
- Bug fixes
-
v0.14.1 Changes
July 17, 2018Deprecations
Flow.filter_map/3
is deprecated in favor of filter+mapFlow.from_stage/2
is deprecated in favor ofFlow.from_stages/2
Flow.merge/2
is deprecated in favor ofFlow.partition/2
orFlow.shuffle/2
Enhancements
- Add
Flow.shuffle/2
to shuffle the stages into new ones - Add
Flow.through_stages/3
for hookingproducer_consumer
s into the flow - Add
Flow.from_specs/2
,Flow.through_specs/3
andFlow.into_specs/3
to start stages in the same supervision tree as the flow
- Add
-
v0.14.0 Changes
June 10, 2018๐ This release includes a redesign of how triggers and the reducing accumulator works.
Prior to this version, the
Flow
module was responsible for traversing events in the mapper stage and to accumulate the state in reducing stages. When working with unbound data, theFlow.Window
was used to control exactly when to emit data from the reducing stages and when to reset the partition state.๐ This approach meant that understanding which data is emitted and when the state was reset was hard because the logic was spread in multiple places. To make matters worse, if you wanted to have your own rules for emitting events, such as user session or sliding windows, it was only possible to achieve it via custom window implementations.
This design limitation caused many users to drop Flow and use GenStage, as GenStage provides the necessary abstractions for tackling those problems. However, since Flow is built on top of GenStage, why not expose it directly through Flow? That's what v0.14.0 does.
v0.14.0 introduces two new functions:
emit_and_reduce/3
andon_trigger/2
which gives developers explicit control of when to emit data. Theon_trigger/2
function also allows developers to fully control the state that is kept in the reducing stage after the trigger.Unfortunately this change is incompatible (or rather, fully replaces) the following functionalities:
each_state/2
andmap_state/2
- those two functions were only invoked when there was a trigger and they have now been replaced by a more explicitly namedon_trigger/2
functionThe
:keep
and:reset
argument to windows and triggers have been removed as you control the behaviour onon_trigger/2
For example, if you used
map_state/2
(oreach_state/2
) and a:reset
trigger, like this:|> Flow.map_state(fn acc -> do_something(acc) end)
You can now replace this code by:
|> Flow.on_trigger(fn acc -> {do_something(acc), []} end)
Where the first element of the tuple returned by
on_trigger
is the data to emit and the second element is the new accumulator of the reducer stage. Similarly, if you were usingmap_state/2
(oreach_state/2
) and a:keep
trigger, like this:|> Flow.map_state(fn acc -> do_something(acc) end)
You can now replace this code by:
|> Flow.on_trigger(fn acc -> {do_something(acc), acc} end)
Note that
on_trigger/2
can only be called once per partition. In case you were callingmap_state/2
andeach_state/2
multiple times, you can simply inline all calls inside the sameon_trigger/2
.We believe
emit_and_reduce/3
andon_trigger/2
provide a conceptually simpler module to reason about flows while being more powerful.This release also deprecates
Flow.Window.session/3
as developers can trivially roll their own with more customization power and flexibility usingemit_and_reduce/3
andon_trigger/2
.Notes
Enhancements
use Flow
now defines achild_spec/1
to be used under supervision- Added
emit_and_reduce/3
andon_trigger/2
- Use
DemandDispatcher
when there is one stage in partition
Deprecations
- Session windows are deprecated in favor of
Flow.emit_and_reduce/3
andFlow.on_trigger/2
- Session windows are deprecated in favor of
Backwards incompatible changes
Flow.map_state/2
was removed in favor ofFlow.on_trigger/2
Flow.each_state/2
was removed in favor ofFlow.on_trigger/2
- Passing
:keep
or:reset
to triggers was removed in favor of explicit control viaFlow.on_trigger/2
. If you are passing or matching on those atoms, those entries can be removed
-
v0.13.0 Changes
January 23, 2018Enhancements
- Expose a timeout parameter for start_link and into_stages
- Allow shutdown time for stages to be configured
Bug fixes
- Ensure proper shutdown propagation on start_link, into_stages and friends (#40)
- Ensure proper shutdown order in Flow (#35)
-
v0.12.0 Changes
Enhancements
- Allow late subscriptions to Flow returned by
Flow.into_stages
- Allow late subscriptions to Flow returned by
Bug fixes
- Cancel timer when termination is triggered on periodic window. This avoid invoking termination callbacks twice.
-
v0.11.1 Changes
Enhancements
- Add the ability to emit only certain events in a trigger
Bug fixes
- Add
:gen_stage
to the applications list - Ensure we handle supervisor exits on flow coordinator
- Ensure we do not unecessary partition when fusing producer+streams
- Add