Changelog History
Page 3
-
v2.5.0 Changes
February 26, 2021Top of the Minute Cron
โฑ Rather than scheduling cron evaluation 60 seconds after the server starts, โฑ evaluation is now scheduled at the top of the next minute. This yields several ๐ improvements:
- ๐ท More predictable timing for cron jobs now that they are inserted at the top of the minute. Note that jobs may execute later depending on how busy queues are.
- ๐ท Minimize contention between servers inserting jobs, thanks to the transaction lock acquired by each plugin.
- ๐ท Prevent duplicate inserts for jobs that omit the
completed
state (when server start time is staggered the transaction lock has no effect)
๐ Repeater Plugin for Transaction Pooled Databases
Environments that can't make use of PG notifications, i.e. because they use ๐ท PgBouncer with transaction pooling, won't process available jobs reliably. The ๐ new
Repeater
plugin provides a workaround that simulates polling functionality for producers.๐ Simply include the
Repeater
in your plugin list:config :my_app, Oban, plugins: [ Oban.Plugins.Pruner, Oban.Plugins.Stager, Oban.Plugins.Repeater ], ...
๐ Include Perform Result in Job Telemetry Metadata
๐ The metadata for
[:oban, :job, :stop]
telemetry events now include the job's ๐ทperform/1
return value. That makes it possible to extract job output from other processes:def handle_event([:oban, :job, :stop], _timing, meta, _opts) do IO.inspect(meta.result, label: "return from #{meta.job.worker}") :ok end
โ Added
๐ [Oban] Support a changeset function in addition to a changeset in
insert_all/4
. Inserting jobs within a multi is even more powerful.๐ท [Oban.Queue.Executor] Stash a timed job's pid to enable inner process messaging, notably via telemetry events.
๐ Changed
โฌ๏ธ [Oban] Upgrade minimum Elixir version from 1.8 to 1.9.
๐ [Oban.Plugins.Cron] Individually validate crontab workers and options, providing more descriptive errors at the collection and entry level.
๐ท [Oban] Simplify the job cancelling flow for consistency. Due to race conditions it was always possible for a job to complete before it was cancelled, now that flow is much simpler.
[Oban.Queue.Producer] Replace dispatch cooldown with a simpler debounce mechanism.
๐ [Oban.Plugins.Stager] Limit the number of notify events to one per queue, rather than one per available job.
๐ Fixed
[Oban.Queue.Producer] Handle unclean exits without an error and stack, which prevents "zombie" jobs. This would occur after multiple hot upgrades when the worker module changed, as the VM would forcibly terminate any processes running the old modules.
[Oban] Specify that the
changeset_wrapper
type allows keys other than:changesets
, fixing a dialyzer type mismatch.
โ Removed
- ๐ [Oban] Remove the undocumented
version/0
function in favor of using the standardApplication.spec/2
-
v2.4.3 Changes
February 07, 2021๐ Fixed
- ๐ท [Oban.Telemetry] Use
conf
rather thanconfig
in meta for:job
and:circuit
telemetry events.
- ๐ท [Oban.Telemetry] Use
-
v2.4.2 Changes
January 28, 2021๐ Fixed
- ๐ [Oban.Plugins.Stager] Notify queues of all available jobs, not only jobs that were most recently staged. In some circumstances, such as reboot or retries, jobs are available without previously being scheduled.
-
v2.4.1 Changes
January 27, 2021๐ Fixed
[Oban.Migrations] Correctly migrate up between sequential versions. The V10 migration adds check constraints, which can't be re-run safely. An attempted work-around prevented re-runs, but broke sequential upgrades like 9->10.
[Oban.Queue.Producer] Kick off an initial dispatch to start processing available jobs on startup, to compensate for a lack of incoming jobs or scheduling.
โฑ [Oban.Plugins.Stager] Keep rescheduling staging messages after the initial run.
-
v2.4.0 Changes
January 26, 2021๐ Centralized Stager Plugin
โฑ Queue producers no longer poll every second to stage scheduled jobs. Instead, ๐ the new
Oban.Plugins.Stager
plugin efficiently handles staging from a single location. This reduces overall load on the BEAM and PostgreSQL, allowing apps to easily run hundreds of queues simultaneously with little additional overhead.In a test with a single Ecto connection and 512 queues the CPU load went from 60.0% BEAM / 21.5% PostgreSQL in v2.3.4 to 0.5% BEAM / 0.0% PostgreSQL.
๐ The stager plugin is automatically injected into Oban instances and there isn't ๐ง any additional configuration necessary. However, if you've set a
poll_interval
๐ for Oban or an individual queue you can safely remove it.Overhauled Cron
The CRON parser is entirely rewritten to be simpler and dramatically smaller. ๐ The previous parser was built on
nimble_parsec
and while it was fast, the ๐ compiled parser added ~5,500LOC to the code base. Thanks to a suite of property โ tests we can be confident that the new parser behaves identically to the ๐ previous one, and has much clearer error messages when parsing fails.๐ Along with a new parser the
crontab
functionality is extracted into the ๐Oban.Plugins.Cron
plugin. For backward compatibility, top-levelcrontab
and ๐timezone
options are transformed into a plugin declaration. If you'd like to ๐ง start configuring the plugin directly change your config from:config :my_app, Oban, crontab: [{"* * * * *", MyApp.Worker}], timezone: "America/Chicago" ...
To this:
config :my_app, Oban, plugins: [ {Oban.Plugins.Cron, crontab: [{"* * * * *", MyApp.Worker}], timezone: "America/Chicago"} ]
๐ The plugin brings a cleaner implementation, simplified supervision tree, and ๐ง eliminates the need to set
crontab: false
in test configuration.๐ Improved Indexes for Unique Jobs
๐ท Applications may have thousands to millions of completed jobs in storage. As the ๐ table grows the performance of inserting unique jobs can slow drastically. A new V10 migration adds necessary indexes, and paired with improved query logic it alleviates insert slowdown entirely.
๐ท For comparison, a local benchmark showed that in v2.3.4 inserting a unique job ๐ท into a table of 1 million jobs runs around 4.81 IPS. In v2.4.0 it runs at 925.34 IPS, nearly 200x faster.
The V10 migration is optional. If you decide to migrate, first create a new migration:
mix ecto.gen.migration upgrade_oban_jobs_to_v10
Next, call
Oban.Migrations
in the generated migration:defmodule MyApp.Repo.Migrations.UpdateObanJobsToV10 do use Ecto.Migration def up do Oban.Migrations.up(version: 10) end def down do Oban.Migrations.down(version: 10) end end
โ Added
[Oban] Add
[:oban, :supervisor, :init]
event emitted when an Oban supervisor starts.๐ [Oban.Telemetry] Wrap built-in plugins with telemetry spans and consistently include
conf
in all telemetry events.[Oban.Config] Improve the error messages raised during initial validation. Also, the
Config
module is now public with light documentation.โ [Oban.Testing] Support specifying a default prefix for all test assertions. This comes with improved assertion messages that now include the prefix.
[Oban.Repo] Add
delete/3
as a convenient wrapper aroundc:Ecto.Repo.delete/2
.๐ท [Oban.Job] New check constraints prevent inserting jobs with an invalid
priority
, negativemax_attempts
or anattempt
number beyond the maximum.
๐ Changed
- ๐ [Oban.Telemetry] Deprecate and replace
span/3
usage in favor of the official:telemetry.span/3
, which wasn't available whenspan/3
was implemented.
๐ Fixed
- [Oban] Inspect the error reason for a failed
insert!/2
call before it is raised as an error. Wheninsert!/2
was called in a transaction the error could be:rollback
, which wasn't a valid error.
-
v2.3.4 Changes
December 02, 2020๐ Fixed
- โก๏ธ [Oban.Worker] Update
from_string/1
to correctly work with modules that weren't loaded.
- โก๏ธ [Oban.Worker] Update
-
v2.3.3 Changes
November 10, 2020๐ Changed
- ๐ท [Oban.Migration] Conditionally skip altering
oban_job_state
if thecancelled
state is already present. This allows for a smoother upgrade path for users running PG < 12. See the notes for v2.3.0 for more details.
- ๐ท [Oban.Migration] Conditionally skip altering
-
v2.3.2 Changes
November 06, 2020๐ Fixed
โช [Oban.Migration] Restore indexes possibly removed by changing
oban_job_state
. This only applies to PG older than version 12.๐ [Oban.Plugins.Pruner] Prune
cancelled
jobs along withcompleted
ordiscarded
.
-
v2.3.1 Changes
November 06, 2020๐ Changed
- ๐ท [Oban.Migration] Conditionally alter
oban_job_state
if the PG version is 12 or greater. This is vastly faster than the renaming, adding and dropping required for older PG versions.
- ๐ท [Oban.Migration] Conditionally alter
-
v2.3.0 Changes
November 06, 2020Migration Required (V9)
๐ท Migration V9 brings the new
cancelled
state, acancelled_at
column, and jobmeta
.Older versions of PostgreSQL, prior to version 12, don't allow altering an enum within a transaction. If you're running an older version and want to prevent a table rewrite you can add the new cancelled at state before running the V9 migration.
Create a migration with
@disable_ddl_transaction true
declared and run the ๐ท commandALTER TYPE oban_job_state ADD VALUE IF NOT EXISTS 'cancelled'
. The V9 ๐ migration will see that the cancelled value exists and won't attempt to modify the enum.After you've sorted adding the
cancelled
state (or ignored the issue, because ๐ท you're either running PG >= 12 or don't have many jobs retained), generate a new migration:mix ecto.gen.migration upgrade_oban_jobs_to_v9
Next, call
Oban.Migrations
in the generated migration:defmodule MyApp.Repo.Migrations.UpdateObanJobsToV9 do use Ecto.Migration def up do Oban.Migrations.up(version: 9) end def down do Oban.Migrations.down(version: 9) end end
โฌ๏ธ Oban will manage upgrading to V9 regardless of the version your application is currently using, and it will roll back a single version.
โ Added
๐ท [Oban.Job] Add new
meta
field for storing arbitrary job data that isn't appropriate asargs
.๐ท [Oban.Job] Introduce a
cancelled
state, along with a newcancelled_at
timestamp field. Cancelling a job viaOban.cancel_job
(or via Oban Web) now marks the job ascancelled
rather thandiscarded
.๐ท [Oban.Worker] Add
from_string/1
for improved worker module resolution.๐ [Oban.Telemetry] Pass the full
job
schema in telemetry metadata, not only select fields. Individual fields such asargs
,worker
, etc. are still passed for backward compatibility. However, their use is deprecated and they are no longer documented.
๐ Fixed
[Oban.Notifier] Fix resolution of
Oban.Notifier
child process inOban.Notifier.listen/2
.๐ท [Oban.Queue.Producer] Fix cancelling jobs without a supervised process. In some circumstances, namely a hot code upgrade, the job's process could terminate without the producer tracking it and leave the job un-killable.
[Oban] Only convert invalid changesets into
ChangesetError
frominsert!
. This prevents unexpected errors wheninsert!
is called within a transaction after the transaction has rolled back.