All Versions
59
Latest Version
Avg Release Cycle
26 days
Latest Release
569 days ago

Changelog History
Page 3

  • v2.5.0 Changes

    February 26, 2021

    Top of the Minute Cron

    โฑ Rather than scheduling cron evaluation 60 seconds after the server starts, โฑ evaluation is now scheduled at the top of the next minute. This yields several ๐Ÿ‘Œ improvements:

    • ๐Ÿ‘ท More predictable timing for cron jobs now that they are inserted at the top of the minute. Note that jobs may execute later depending on how busy queues are.
    • ๐Ÿ‘ท Minimize contention between servers inserting jobs, thanks to the transaction lock acquired by each plugin.
    • ๐Ÿ‘ท Prevent duplicate inserts for jobs that omit the completed state (when server start time is staggered the transaction lock has no effect)

    ๐Ÿ”Œ Repeater Plugin for Transaction Pooled Databases

    Environments that can't make use of PG notifications, i.e. because they use ๐Ÿ‘ท PgBouncer with transaction pooling, won't process available jobs reliably. The ๐Ÿ†• new Repeater plugin provides a workaround that simulates polling functionality for producers.

    ๐Ÿ”Œ Simply include the Repeater in your plugin list:

    config :my_app, Oban,
      plugins: [
        Oban.Plugins.Pruner,
        Oban.Plugins.Stager,
        Oban.Plugins.Repeater
      ],
      ...
    

    ๐Ÿ“‡ Include Perform Result in Job Telemetry Metadata

    ๐Ÿ“‡ The metadata for [:oban, :job, :stop] telemetry events now include the job's ๐Ÿ‘ท perform/1 return value. That makes it possible to extract job output from other processes:

    def handle_event([:oban, :job, :stop], _timing, meta, _opts) do
      IO.inspect(meta.result, label: "return from #{meta.job.worker}")
    
      :ok
    end
    

    โž• Added

    • ๐Ÿ‘ [Oban] Support a changeset function in addition to a changeset in insert_all/4. Inserting jobs within a multi is even more powerful.

    • ๐Ÿ‘ท [Oban.Queue.Executor] Stash a timed job's pid to enable inner process messaging, notably via telemetry events.

    ๐Ÿ”„ Changed

    • โฌ†๏ธ [Oban] Upgrade minimum Elixir version from 1.8 to 1.9.

    • ๐Ÿ”Œ [Oban.Plugins.Cron] Individually validate crontab workers and options, providing more descriptive errors at the collection and entry level.

    • ๐Ÿ‘ท [Oban] Simplify the job cancelling flow for consistency. Due to race conditions it was always possible for a job to complete before it was cancelled, now that flow is much simpler.

    • [Oban.Queue.Producer] Replace dispatch cooldown with a simpler debounce mechanism.

    • ๐Ÿ”Œ [Oban.Plugins.Stager] Limit the number of notify events to one per queue, rather than one per available job.

    ๐Ÿ›  Fixed

    • [Oban.Queue.Producer] Handle unclean exits without an error and stack, which prevents "zombie" jobs. This would occur after multiple hot upgrades when the worker module changed, as the VM would forcibly terminate any processes running the old modules.

    • [Oban] Specify that the changeset_wrapper type allows keys other than :changesets, fixing a dialyzer type mismatch.

    โœ‚ Removed

    • ๐Ÿšš [Oban] Remove the undocumented version/0 function in favor of using the standard Application.spec/2
  • v2.4.3 Changes

    February 07, 2021

    ๐Ÿ›  Fixed

    • ๐Ÿ‘ท [Oban.Telemetry] Use conf rather than config in meta for :job and :circuit telemetry events.
  • v2.4.2 Changes

    January 28, 2021

    ๐Ÿ›  Fixed

    • ๐Ÿ”Œ [Oban.Plugins.Stager] Notify queues of all available jobs, not only jobs that were most recently staged. In some circumstances, such as reboot or retries, jobs are available without previously being scheduled.
  • v2.4.1 Changes

    January 27, 2021

    ๐Ÿ›  Fixed

    • [Oban.Migrations] Correctly migrate up between sequential versions. The V10 migration adds check constraints, which can't be re-run safely. An attempted work-around prevented re-runs, but broke sequential upgrades like 9->10.

    • [Oban.Queue.Producer] Kick off an initial dispatch to start processing available jobs on startup, to compensate for a lack of incoming jobs or scheduling.

    • โฑ [Oban.Plugins.Stager] Keep rescheduling staging messages after the initial run.

  • v2.4.0 Changes

    January 26, 2021

    ๐Ÿ”Œ Centralized Stager Plugin

    โฑ Queue producers no longer poll every second to stage scheduled jobs. Instead, ๐Ÿ”Œ the new Oban.Plugins.Stager plugin efficiently handles staging from a single location. This reduces overall load on the BEAM and PostgreSQL, allowing apps to easily run hundreds of queues simultaneously with little additional overhead.

    In a test with a single Ecto connection and 512 queues the CPU load went from 60.0% BEAM / 21.5% PostgreSQL in v2.3.4 to 0.5% BEAM / 0.0% PostgreSQL.

    ๐Ÿ”Œ The stager plugin is automatically injected into Oban instances and there isn't ๐Ÿ”ง any additional configuration necessary. However, if you've set a poll_interval ๐Ÿšš for Oban or an individual queue you can safely remove it.

    Overhauled Cron

    The CRON parser is entirely rewritten to be simpler and dramatically smaller. ๐Ÿ“œ The previous parser was built on nimble_parsec and while it was fast, the ๐Ÿ“œ compiled parser added ~5,500LOC to the code base. Thanks to a suite of property โœ… tests we can be confident that the new parser behaves identically to the ๐Ÿ“œ previous one, and has much clearer error messages when parsing fails.

    ๐Ÿ“œ Along with a new parser the crontab functionality is extracted into the ๐Ÿ”Œ Oban.Plugins.Cron plugin. For backward compatibility, top-level crontab and ๐Ÿ”Œ timezone options are transformed into a plugin declaration. If you'd like to ๐Ÿ”ง start configuring the plugin directly change your config from:

    config :my_app, Oban,
      crontab: [{"* * * * *", MyApp.Worker}],
      timezone: "America/Chicago"
      ...
    

    To this:

    config :my_app, Oban,
      plugins: [
        {Oban.Plugins.Cron,
         crontab: [{"* * * * *", MyApp.Worker}],
         timezone: "America/Chicago"}
      ]
    

    ๐Ÿ”Œ The plugin brings a cleaner implementation, simplified supervision tree, and ๐Ÿ”ง eliminates the need to set crontab: false in test configuration.

    ๐Ÿ‘Œ Improved Indexes for Unique Jobs

    ๐Ÿ‘ท Applications may have thousands to millions of completed jobs in storage. As the ๐ŸŽ table grows the performance of inserting unique jobs can slow drastically. A new V10 migration adds necessary indexes, and paired with improved query logic it alleviates insert slowdown entirely.

    ๐Ÿ‘ท For comparison, a local benchmark showed that in v2.3.4 inserting a unique job ๐Ÿ‘ท into a table of 1 million jobs runs around 4.81 IPS. In v2.4.0 it runs at 925.34 IPS, nearly 200x faster.

    The V10 migration is optional. If you decide to migrate, first create a new migration:

    mix ecto.gen.migration upgrade_oban_jobs_to_v10
    

    Next, call Oban.Migrations in the generated migration:

    defmodule MyApp.Repo.Migrations.UpdateObanJobsToV10 do
      use Ecto.Migration
    
      def up do
        Oban.Migrations.up(version: 10)
      end
    
      def down do
        Oban.Migrations.down(version: 10)
      end
    end
    

    โž• Added

    • [Oban] Add [:oban, :supervisor, :init] event emitted when an Oban supervisor starts.

    • ๐Ÿ”Œ [Oban.Telemetry] Wrap built-in plugins with telemetry spans and consistently include conf in all telemetry events.

    • [Oban.Config] Improve the error messages raised during initial validation. Also, the Config module is now public with light documentation.

    • โœ… [Oban.Testing] Support specifying a default prefix for all test assertions. This comes with improved assertion messages that now include the prefix.

    • [Oban.Repo] Add delete/3 as a convenient wrapper around c:Ecto.Repo.delete/2.

    • ๐Ÿ‘ท [Oban.Job] New check constraints prevent inserting jobs with an invalid priority, negative max_attempts or an attempt number beyond the maximum.

    ๐Ÿ”„ Changed

    • ๐Ÿ—„ [Oban.Telemetry] Deprecate and replace span/3 usage in favor of the official :telemetry.span/3, which wasn't available when span/3 was implemented.

    ๐Ÿ›  Fixed

    • [Oban] Inspect the error reason for a failed insert!/2 call before it is raised as an error. When insert!/2 was called in a transaction the error could be :rollback, which wasn't a valid error.
  • v2.3.4 Changes

    December 02, 2020

    ๐Ÿ›  Fixed

    • โšก๏ธ [Oban.Worker] Update from_string/1 to correctly work with modules that weren't loaded.
  • v2.3.3 Changes

    November 10, 2020

    ๐Ÿ”„ Changed

    • ๐Ÿ‘ท [Oban.Migration] Conditionally skip altering oban_job_state if the cancelled state is already present. This allows for a smoother upgrade path for users running PG < 12. See the notes for v2.3.0 for more details.
  • v2.3.2 Changes

    November 06, 2020

    ๐Ÿ›  Fixed

    • โช [Oban.Migration] Restore indexes possibly removed by changing oban_job_state. This only applies to PG older than version 12.

    • ๐Ÿ”Œ [Oban.Plugins.Pruner] Prune cancelled jobs along with completed or discarded.

  • v2.3.1 Changes

    November 06, 2020

    ๐Ÿ”„ Changed

    • ๐Ÿ‘ท [Oban.Migration] Conditionally alter oban_job_state if the PG version is 12 or greater. This is vastly faster than the renaming, adding and dropping required for older PG versions.
  • v2.3.0 Changes

    November 06, 2020

    Migration Required (V9)

    ๐Ÿ‘ท Migration V9 brings the new cancelled state, a cancelled_at column, and job meta.

    Older versions of PostgreSQL, prior to version 12, don't allow altering an enum within a transaction. If you're running an older version and want to prevent a table rewrite you can add the new cancelled at state before running the V9 migration.

    Create a migration with @disable_ddl_transaction true declared and run the ๐Ÿ‘ท command ALTER TYPE oban_job_state ADD VALUE IF NOT EXISTS 'cancelled'. The V9 ๐Ÿ‘€ migration will see that the cancelled value exists and won't attempt to modify the enum.

    After you've sorted adding the cancelled state (or ignored the issue, because ๐Ÿ‘ท you're either running PG >= 12 or don't have many jobs retained), generate a new migration:

    mix ecto.gen.migration upgrade_oban_jobs_to_v9
    

    Next, call Oban.Migrations in the generated migration:

    defmodule MyApp.Repo.Migrations.UpdateObanJobsToV9 do
      use Ecto.Migration
    
      def up do
        Oban.Migrations.up(version: 9)
      end
    
      def down do
        Oban.Migrations.down(version: 9)
      end
    end
    

    โฌ†๏ธ Oban will manage upgrading to V9 regardless of the version your application is currently using, and it will roll back a single version.

    โž• Added

    • ๐Ÿ‘ท [Oban.Job] Add new meta field for storing arbitrary job data that isn't appropriate as args.

    • ๐Ÿ‘ท [Oban.Job] Introduce a cancelled state, along with a new cancelled_at timestamp field. Cancelling a job via Oban.cancel_job (or via Oban Web) now marks the job as cancelled rather than discarded.

    • ๐Ÿ‘ท [Oban.Worker] Add from_string/1 for improved worker module resolution.

    • ๐Ÿ“‡ [Oban.Telemetry] Pass the full job schema in telemetry metadata, not only select fields. Individual fields such as args, worker, etc. are still passed for backward compatibility. However, their use is deprecated and they are no longer documented.

    ๐Ÿ›  Fixed

    • [Oban.Notifier] Fix resolution of Oban.Notifier child process in Oban.Notifier.listen/2.

    • ๐Ÿ‘ท [Oban.Queue.Producer] Fix cancelling jobs without a supervised process. In some circumstances, namely a hot code upgrade, the job's process could terminate without the producer tracking it and leave the job un-killable.

    • [Oban] Only convert invalid changesets into ChangesetError from insert!. This prevents unexpected errors when insert! is called within a transaction after the transaction has rolled back.