oban v2.4.0 Release Notes

Release Date: 2021-01-26 // over 1 year ago
  • ๐Ÿ”Œ Centralized Stager Plugin

    โฑ Queue producers no longer poll every second to stage scheduled jobs. Instead, ๐Ÿ”Œ the new Oban.Plugins.Stager plugin efficiently handles staging from a single location. This reduces overall load on the BEAM and PostgreSQL, allowing apps to easily run hundreds of queues simultaneously with little additional overhead.

    In a test with a single Ecto connection and 512 queues the CPU load went from 60.0% BEAM / 21.5% PostgreSQL in v2.3.4 to 0.5% BEAM / 0.0% PostgreSQL.

    ๐Ÿ”Œ The stager plugin is automatically injected into Oban instances and there isn't ๐Ÿ”ง any additional configuration necessary. However, if you've set a poll_interval ๐Ÿšš for Oban or an individual queue you can safely remove it.

    Overhauled Cron

    The CRON parser is entirely rewritten to be simpler and dramatically smaller. ๐Ÿ“œ The previous parser was built on nimble_parsec and while it was fast, the ๐Ÿ“œ compiled parser added ~5,500LOC to the code base. Thanks to a suite of property โœ… tests we can be confident that the new parser behaves identically to the ๐Ÿ“œ previous one, and has much clearer error messages when parsing fails.

    ๐Ÿ“œ Along with a new parser the crontab functionality is extracted into the ๐Ÿ”Œ Oban.Plugins.Cron plugin. For backward compatibility, top-level crontab and ๐Ÿ”Œ timezone options are transformed into a plugin declaration. If you'd like to ๐Ÿ”ง start configuring the plugin directly change your config from:

    config :my_app, Oban,
      crontab: [{"* * * * *", MyApp.Worker}],
      timezone: "America/Chicago"
      ...
    

    To this:

    config :my_app, Oban,
      plugins: [
        {Oban.Plugins.Cron,
         crontab: [{"* * * * *", MyApp.Worker}],
         timezone: "America/Chicago"}
      ]
    

    ๐Ÿ”Œ The plugin brings a cleaner implementation, simplified supervision tree, and ๐Ÿ”ง eliminates the need to set crontab: false in test configuration.

    ๐Ÿ‘Œ Improved Indexes for Unique Jobs

    ๐Ÿ‘ท Applications may have thousands to millions of completed jobs in storage. As the ๐ŸŽ table grows the performance of inserting unique jobs can slow drastically. A new V10 migration adds necessary indexes, and paired with improved query logic it alleviates insert slowdown entirely.

    ๐Ÿ‘ท For comparison, a local benchmark showed that in v2.3.4 inserting a unique job ๐Ÿ‘ท into a table of 1 million jobs runs around 4.81 IPS. In v2.4.0 it runs at 925.34 IPS, nearly 200x faster.

    The V10 migration is optional. If you decide to migrate, first create a new migration:

    mix ecto.gen.migration upgrade_oban_jobs_to_v10
    

    Next, call Oban.Migrations in the generated migration:

    defmodule MyApp.Repo.Migrations.UpdateObanJobsToV10 do
      use Ecto.Migration
    
      def up do
        Oban.Migrations.up(version: 10)
      end
    
      def down do
        Oban.Migrations.down(version: 10)
      end
    end
    

    โž• Added

    • [Oban] Add [:oban, :supervisor, :init] event emitted when an Oban supervisor starts.

    • ๐Ÿ”Œ [Oban.Telemetry] Wrap built-in plugins with telemetry spans and consistently include conf in all telemetry events.

    • [Oban.Config] Improve the error messages raised during initial validation. Also, the Config module is now public with light documentation.

    • โœ… [Oban.Testing] Support specifying a default prefix for all test assertions. This comes with improved assertion messages that now include the prefix.

    • [Oban.Repo] Add delete/3 as a convenient wrapper around c:Ecto.Repo.delete/2.

    • ๐Ÿ‘ท [Oban.Job] New check constraints prevent inserting jobs with an invalid priority, negative max_attempts or an attempt number beyond the maximum.

    ๐Ÿ”„ Changed

    • ๐Ÿ—„ [Oban.Telemetry] Deprecate and replace span/3 usage in favor of the official :telemetry.span/3, which wasn't available when span/3 was implemented.

    ๐Ÿ›  Fixed

    • [Oban] Inspect the error reason for a failed insert!/2 call before it is raised as an error. When insert!/2 was called in a transaction the error could be :rollback, which wasn't a valid error.