All Versions
55
Latest Version
Avg Release Cycle
25 days
Latest Release
33 days ago

Changelog History
Page 4

  • v2.0.0-rc.2 Changes

    June 23, 2020

    πŸ’₯ Breaking Changes

    • [Oban] The interface for pause_queue/2, resume_queue/2 and scale_queue/3 now matches the recently changed start_queue/2 and stop_queue/2. All queue manipulation functions now have a consistent interface, including the ability to work in :local_only mode. See the notes around start_queue/2 and stop_queue/2 in [2.0.0-rc.1][] for help upgrading.

    • [Oban] Replace drain_queue/3 with drain_queue/2, which now has an interface consistent with the other *_queue/2 operations.

    Where you previously called drain_queue/2,3 like this:

    ```elixir
    Oban.drain_queue(:myqueue, with_safety: false)
    ```
    

    You'll now it with options, like this:

    ```elixir
    Oban.drain_queue(queue: :myqueue, with_safety: false)
    ```
    

    πŸ›  Fixed

    • [Oban.Breaker] Prevent connection bomb when the Notifier experiences repeated disconnections.

    • πŸ‘· [Oban.Queue.Executor] Fix error reporting when a worker fails to resolve.

    • 🌲 [Oban.Telemetry] Stop logging the :error value for circuit trip events. The error is a struct that isn't JSON encodable. We include the normalized Postgrex / DBConnection message already, so the error is redundant.

    βž• Added

    • ⏱ [Oban.Crontab] Add @reboot special case to crontab scheduling

    • ⏱ [Oban.Telemetry] Add new :producer events for descheduling and dispatching jobs from queue producers.

    βœ‚ Removed

    • 🚚 [Oban] Removed Oban.kill_job/2, which wasn't as flexible as Oban.cancel_job/2. Use Oban.cancel_job/2 instead to safely discard scheduled jobs or killing executing jobs.
  • v2.0.0-rc.1 Changes

    June 12, 2020

    πŸ’₯ Breaking Changes

    • 🌲 [Oban.Config] The :verbose setting is renamed to :log. The setting started off as a simple boolean, but it has morphed to align with the log values accepted by calls to Ecto.Repo.

    To migrate, replace any :verbose declarations:

    ```elixir
    config :my_app, Oban,
      verbose: false,
      ...
    ```
    

    With use of :log instead:

    ```elixir
    config :my_app, Oban,
      log: false,
      ...
    ```
    
    • [Oban] The interface for start_queue/3 is replaced with start_queue/2 and stop_queue/2 no longer accepts a queue name as the second argument. Instead, both functions now accept a keyword list of options. This enables the new local_only flag, which allows you to dynamically start and stop queues only for the local node.

    Where you previously called start_queue/2,3 or stop_queue/2 like this:

    ```elixir
    :ok = Oban.start_queue(:myqueue, 10)
    :ok = Oban.stop_queue(:myqueue)
    ```
    

    You'll now call them with options, like this:

    ```elixir
    :ok = Oban.start_queue(queue: :myqueue, limit: 10)
    :ok = Oban.stop_queue(queue: :myqueue)
    ```
    

    Or, to only control the queue locally:

    ```elixir
    :ok = Oban.start_queue(queue: :myqueue, limit: 10, local_only: true)
    :ok = Oban.stop_queue(queue: :myqueue, local_only: true)
    ```
    
  • v2.0.0-rc.0 Changes

    June 03, 2020

    πŸ’₯ Breaking Changes

    • πŸ‘· [Oban.Worker] The perform/2 callback is replaced with perform/1, where the only argument is an Oban.Job struct. This unifies the interface for all Oban.Worker callbacks and helps to eliminate confusion around pattern matching on arguments.

    To migrate change all worker definitions from accepting an args map and a job struct:

    ```elixir
    def perform(%{"id" => id}, _job), do: IO.inspect(id)
    ```
    

    To accept a single job struct and match on the args key directly:

    ```elixir
    def perform(%Job{args: %{"id" => id}}), do: IO.inspect(id)
    ```
    
    • πŸ‘· [Oban.Worker] The backoff/1 callback now expects a job struct instead of an integer. That allows applications to finely control backoff based on more than just the current attempt number. Use of backoff/1 with an integer is no longer supported.

    To migrate change any worker definitions that used a raw attempt like this:

    ```elixir
    def backoff(attempt), do: attempt * 60
    ```
    

    To match on a job struct instead, like this:

    ```elixir
    def backoff(%Job{attempt: attempt}), do: attempt * 60
    ```
    
    • [Oban.Telemetry] The format for telemetry events has changed to match the new telemetry span convention. This listing maps the old event to the new one:

      • [:oban, :started] -> [:oban, :job, :start]
      • [:oban, :success] -> [:oban, :job, :stop]
      • [:oban, :failure] -> [:oban, :job, :exception]
      • [:oban, :trip_circuit] -> [:oban, :circuit, :trip]
      • [:oban, :open_circuit] -> [:oban, :circuit, :open]

    In addition, for exceptions the stacktrace meta key has changed from :stack to the standardized :stacktrace.

    • πŸ”§ [Oban.Prune] Configurable pruning is no longer available. Instead, pruning is handled by the new plugin system. A fixed period pruning module is enabled as a default plugin. The plugin always retains "prunable" (discarded or complete) jobs for 60 seconds.

    Remove any :prune, :prune_interval or prune_limit settings from your config. To disable the pruning plugin in test mode set plugins: false instead.

    • πŸ‘· [Oban.Beat] Pulse tracking and periodic job rescue are no longer available. Pulse tracking and rescuing will be handled by an external plugin. This is primarily an implementation detail, but it means that jobs may be left in the executing state after a crash or forced shutdown.

    Remove any :beats_maxage, :rescue_after or :rescue_interval settings from your config.

    πŸ›  Fixed

    • ⏱ [Oban.Scheduler] Ensure isolation between transaction locks in different prefixes. A node with multiple prefix-isolated instances (i.e. "public" and "private") would always attempt to schedule cron jobs at the same moment. The first scheduler would acquire a lock and block out the second, preventing the second scheduler from ever scheduling jobs.

    • [Oban.Query] Correctly prefix unprepared unique queries. Unique queries always targeted the "public" prefix, which either caused incorrect results when there were both "public" and an alternate prefix. In situations where there wasn't a public oban_jobs table at all it would cause cryptic transaction errors.

    • πŸ‘· [Oban.Query] Wrap all job fetching in an explicit transaction to enforce FOR UPDATE SKIP LOCKED semantics. Prior to this it was possible to run the same job at the same time on multiple nodes.

    • [Oban.Crontab] Fix weekday matching for Sunday, which is represented as 0 in crontabs.

    βž• Added

    • [Oban] Bubble up errors and exits when draining queues by passing with_safety: false as an option to Oban.drain_queue/2.

    • ⏱ [Oban] Add Oban.cancel_job/2 for safely discarding scheduled jobs or killing executing jobs. This deprecates kill_job/2, which isn't as flexible.

    • πŸ‘· [Oban.Worker] Support returning {:snooze, seconds} from perform/1 to re-schedule a job some number of seconds in the future. This is useful for recycling jobs that aren't ready to run yet, e.g. because of rate limiting.

    • πŸ‘· [Oban.Worker] Support returning :discard from perform/1 to immediately discard a job. This is useful when a job encounters an error that won't resolve with time, e.g. invalid arguments or a missing record.

    • πŸ‘· [Oban.Job] Introduce a virtual unsaved_error field, which is populated with an error map after failed execution. The unsaved_error field is set before any calls to the worker's backoff/1 callback, allowing workers to calculate a custom backoff depending on the error that failed the job.

    • πŸ‘· [Oban.Worker] Add :infinity option for unique period.

    • [Oban.Telemetry] Add span/3 for reporting normalized :start, :stop and :exception events with timing information.

    • πŸ“‡ [Oban.Telemetry] Include the configured prefix in all event metadata. This makes it possible to identify which schema prefix a job ran with, which is useful for differentiating errors in a multi-tenant system.

    • [Oban.Telemetry] Include queue_time as a measurement with stop and exception events. This is a measurement in milliseconds of the amount of time between when a job was scheduled to run and when it was last attempted.

    • βœ… [Oban.Testing] Add perform_job/2,3 helper to automate validating, normalizing and performing jobs while unit testing. This is now the preferred way to unit test workers.

    To update your tests replace any calls to perform/1,2 with the new Oban.Testing.perform_job/2,3 helper:

      defmodule MyApp.WorkerTest do
        use MyApp.DataCase, async: true
    
        use Oban.Testing, repo: MyApp.Repo
    
        alias MyApp.Worker
    
        test "doing business in my worker" do
          assert :ok = perform_job(Worker, %{id: 1})
        end
      end
    

    The perform_job/2,3 helper will verify the worker, the arguments and any provided options. It will then verify that your worker returns a valid result and return the value for you to assert on.

    • πŸ‘ [Oban.Crontab] Add support for non-standard expressions such as @daily, @hourly, @midnight, etc.

    • πŸ‘ [Oban.Crontab] Add support for using step values in conjunction with ranges, enabling expressions like 10-30/2, 15-45/3, etc.

    πŸ”„ Changed

    • [Oban.Notifier] Make the module public and clean up the primary function interfaces. Listening for and delivering notifications is simplified and no longer requires macros for pattern matching.

    Notifier dispatching performance is slightly improved as well. It is now a no-op if no processes are listening to a notification's channel.

    • πŸ‘· [Oban.Query] The completed_at timestamp is no longer set for failed jobs, whether they are put in the discarded or retryable state. However, the information is still available and is recorded in the errors array as the at value with the error for that attempt.

    This corrects a long standing inconsistency between discarding a job manually or automatically when it exhausts retries.

    • πŸ‘· [Oban.Producer] Stop dispatching jobs immediately on queue startup. Instead, only dispatch on the first poll. This makes it possible to send the producer a message or allow sandboxed connection access before the initial dispatch.

    • 0️⃣ [Oban.Worker] Limit default backoff calculations to 20 attempts, or roughly 24 days. The change addresses an issue with snoozing, which can increase a job's attempts into the hundreds or thousands. In this situation the algorithm calculates the backoff using a ratio of attempts to max attempts, but is still limited to roughly 24 days.

    πŸ‘€ For changes prior to 2.0 see the 1.2 branch

    πŸš€ [Unreleased]: https://github.com/sorentwo/oban/compare/v2.10.1...HEAD

  • v1.2.0 Changes

    March 05, 2020

    πŸ›  Fixed

    • πŸ‘· [Oban] Handle the :shutdown message when a job is killed purposefully. Previously the message was ignored, which caused the producer to keep a reference to the discarded job and prevented dispatching additional jobs.

    βž• Added

    • [Oban.Testing] Add assert_enqueued/2 and refute_enqueued/2 to allow asserting with a timeout, like assert_received.

    • [Oban.Telemetry] Add [:oban, :started] event to report the system time as start_time when jobs start. This enables span tracking for jobs, improving integration with monitoring tools like NewRelic.

  • v1.1.0 Changes

    February 17, 2020

    πŸ›  Fixed

    • [Oban.Crontab] Allow any number of spaces and/or tabs in cron expressions.

    • [Oban.Pruner] Prevent deadlocks while pruning by ensuring that only a single node can prune at any given time.

    • [Oban.Queue.Executor] Improve handling of nested/linked process failures. Previously if a job spawned a process that crashed (i.e. through Task.async) then the job was left stuck in an executing state.

    βž• Added

    • πŸ‘· [Oban.Worker] Add timeout/1 callback for limiting how long a job may execute. The default value is :infinity, which allows a job to run indefinitely and mirrors the previous behavior.

    • πŸ“‡ [Oban.Telemetry] Add :error and :stack to trip_circuit event metadata.

    • ⚑️ [Oban.Queue.Executor] Retry success/failure database updates after a job finishes.

    On occasion something may happen to the database connection and updating a job's state would fail. Now we retry writing with a linear backoff to prevent the job from getting stuck in an executing state.

    πŸ”„ Changed

    • πŸ‘· [Oban.Worker] Tighten the spec for perform/2. Now workers are expected to return only :ok, {:ok, result} or {:error, reason}. A warning is logged if any other value is returnedβ€”for high throughput systems this may cause performance issues and you should update your worker's return values.

    Returning a success tuple is supported for testing purposes and backward compatibility, while returning :ok on success if preferred.

  • v1.0.0 Changes

    January 29, 2020

    No changes from [1.0.0-rc.2][].

  • v1.0.0-rc.2 Changes

    January 22, 2020

    πŸ›  Fixed

    • [Oban.Migration] Separate adding and modifying new columns in the V8 migration. The columns can't be modified without a flush.

    πŸ”„ Changed

    • βœ… [Oban.Testing] Accept a custom prefix when making test assertions.
  • v1.0.0-rc.1 Changes

    January 21, 2020

    Migration Required (V8)

    πŸš€ This is the first required migration since 0.8.0, released in 09/2019. It brings πŸ‘· with it a new column, discarded_at, a streamlined notifications trigger, job πŸ‘· prioritiy and job tags.

    ⬆️ Upgrading only requires running the new migration.

    First, generate a new migration:

    mix ecto.gen.migration upgrade_oban_jobs_to_v8
    

    Next, call Oban.Migrations in the generated migration:

    defmodule MyApp.Repo.Migrations.UpdateObanJobsToV8 do
      use Ecto.Migration
    
      def up, do: Oban.Migrations.up(version: 8)
      def down, do: Oban.Migrations.down()
    end
    

    ⬆️ Oban will manage upgrading to V8 regardless of the version your application is currently using, and it will roll back a single version.

    βž• Added

    • ⏱ [Oban] Add timezone support for scheduling cronjobs using timezones other than "Etc/UTC". Using a custom timezone requires a timezone database such as tzdata.

    • πŸ”§ [Oban] Add dispatch_cooldown option to configure the minimum time between a producer fetching more jobs to execute.

    • πŸ”§ [Oban] Add beats_maxage option to configure how long heartbeat rows are retained in the oban_beats table. Each queue generates one row per second, so rows may accumulate quickly. The default value is now five minutes, down from one hour previously.

    • πŸ‘· [Oban.Job] Add discarded_at timestamp to help identify jobs that were discarded and not completed. The timestamp is added by the V8 migration and it is also included in the original create table from V1 as a minor space saving optimization (packing datetime columns together because they use a predictable 4bytes of space).

    • πŸ‘· [Oban.Job] Add numerical priority value to control the order of execution for jobs within a queue. The priority can be set between 0 and 3, with 0 being the default and the highest priority.

    • πŸ‘· [Oban.Job] Add tags field for arbitrarily organizing associated tags. Tags are a list of strings stored as an array in the database, making them easy to search and filter by.

    πŸ”„ Changed

    • 0️⃣ [Oban] Change the default prune value from :disabled to {:maxlen, 1_000}. Many people don't change the default until they realize that a lot of jobs are lingering in the database. It is rare that anybody would want to keep all of their jobs forever, so a conservative default is better than :disabled.

    • [Oban] Change oban_beats retention from one hour to five minutes. The value is now configurable, with a default of 300s. The lower bound is 60s because we require one minute of heartbeat activity to rescue orphaned jobs.

    • [Oban.Queue.Producer] Introduce "dispatch cooldown" as a way to debounce repeatedly fetching new jobs. Repeated fetching floods the producer's message queue and forces the producer to repeatedly fetch one job at a time, which is not especially efficient. Debounced fetching is much more efficient for the producer and the database, increasing maximum jobs/sec throughput so that it scales linearly with a queue's concurrency settings (up to what the database can handle).

    • πŸ‘· [Oban.Query] Discard jobs that have exhausted the maximum attempts rather than rescuing them. This prevents repeatedly attempting a job that has consistently crashed the BEAM.

    • [Oban.Query] Use transactional locks to prevent duplicate inserts without relying on unique constraints in the database. This provides strong unique guarantees without requiring migrations.

    βœ‚ Removed

    • [Oban.Notifications] An overhauled and simplified insert trigger no longer emits update notifications. This was largely an internal implementation detail and wasn't publicly documented, but it does effect the UI.
  • v0.12.1 Changes

    December 13, 2019

    πŸ›  Fixed

    • πŸ”€ [Oban.Worker] Deep merge unique options between the worker and custom params. Previously the unique options passed to Worker.new/2 would completely override the options stored by use Oban.Worker. This was primarily an issue for crontab jobs, where the period is always passed by the scheduler.

    πŸ”„ Changed

    • ⏱ [Oban] Allow setting crontab: false or crontab: nil to disable scheduling. This matches the queues behavior and makes it possible to override the default configuration within each environment, i.e. when testing.

    • βœ… [Oban.Testing] Better error message for Oban.Testing.assert_enqueued/2

  • v0.12.0 Changes

    November 26, 2019

    Migration Optional (V7)

    The queries used to prune by limit and age are written to utilize a single 🐎 partial index for a huge performance boost on large tables. The new V7 migration will create the index for youβ€”but that may not be ideal for tables with millions πŸ‘· of completed or discarded jobs because it can't be done concurrently.

    πŸ‘· If you have an extremely large jobs table you can add the index concurrently in a dedicated migration:

    create index(
             :oban_jobs,
             ["attempted_at desc", :id],
             where: "state in ('completed', 'discarded')",
             name: :oban_jobs_attempted_at_id_index,
             concurrently: true
           )
    

    βž• Added

    • [Oban] Add start_queue/3 and stop_queue/2 for dynamically starting and stopping supervised queues across nodes.

    • [Oban] Add drain_queue/3 to accept drain options. with_scheduled: true allows draining scheduled jobs.

    • [Oban] Expose circuit_backoff as a "twiddly" option that controls how long tripped circuit breakers wait until re-opening.

    • βœ… [Oban.Testing] Accept a value/delta tuple for testing timestamp fields. This allows more robust testing of timestamps such as scheduled_at.

    • [Oban.Telemetry] Emit [:oban, :trip_circuit] and [:oban, :open_circuit] events for circuit breaker activity. Previously an error was logged when the circuit was tripped, but there wasn't any way to monitor circuit breakers.

    Circuit breaker activity is logged by the default telemetry logger (both :trip_circuit and :open_circuit events).

    πŸ›  Fixed

    • [Oban.Query] Avoid using prepared statements for all unique queries. This forces Postgres to use a custom plan (which utilizes the compound index) rather than falling back to a generic plan.

    • πŸ‘· [Oban.Job] Include all permitted fields when converting a Job to a map, preserving any optional values that were either specified by the user or came via Worker defaults.

    • [Oban.Migrations] Guard against missing migration modules in federated environments.

    πŸ”„ Changed

    • [Oban] Allow the multi name provided to Oban.insert/3,4 to be any term, not just an atom.

    • [Oban.Query] Use a consistent and more performant set of queries for pruning. Both pruning methods are optimized to utilize a single partial index.