Que v2.0.0.beta1 Release Notes

Release Date: 2022-03-24 // 5 months ago
  • ๐Ÿš€ Preliminary release of Ruby 3 support

    Notable changes:

    • ๐Ÿ‘Œ Support for Ruby 3 introduced
    • ๐Ÿ‘ท Database schema has changed to split the job arguments args column into args and kwargs columns, for reliable args and kwargs splitting for Ruby 3.
      • The job schema version is now 2. Note that job schema version is distinct from database schema version and Que version. The job_schema_version column of the que_jobs table no longer defaults and has a not null constraint, so when manually inserting jobs into the table, this must be specified as 2. If you have a gem that needs to support multiple Que versions, best not to blindly use the value of Que.job_schema_version; instead have different code paths depending on the value of Que.job_schema_version. You could also use this to know whether keyword arguments are in args or kwargs.
    • ๐Ÿ‘ท Passing a hash literal as the last job argument to be splatted into job keyword arguments is no longer supported.
    • ๐Ÿ‘ท Dropped support for providing job options as top-level keyword arguments to Job.enqueue, i.e. queue, priority, run_at, job_class, and tags. Job options now need to be nested under the job_options keyword argument instead. See #336
    • โฌ‡๏ธ Dropped support for Ruby < 2.7
    • โฌ‡๏ธ Dropped support for Rails < 6.0
    • โฑ The #by_args method on the Job model (for both Sequel and ActiveRecord) now searches based on both args and kwargs, but it performs a subset match instead of an exact match. For instance, if your job was scheduled with 'a', 'b', 'c', foo: 'bar', baz: 1, by_args('a', 'b', baz: 1) would find and return the job.
    • ๐Ÿš€ This release contains a database migration. You will need to migrate Que to the latest database schema version (6). For example, on ActiveRecord and Rails 6:
    class UpdateQueTablesToVersion6 < ActiveRecord::Migration[6.0]
      def up
        Que.migrate!(version: 6)
      end
    
      def down
        Que.migrate!(version: 5)
      end
    end
    

    โฌ†๏ธ Recommended upgrade process:

    ๐Ÿ‘ท When using Que 2.x, a job enqueued with Ruby 2.7 will run as expected on Ruby 3. We recommend:

    โฌ†๏ธ 1. Upgrade your project to the latest 1.x version of Que (1.3.1+) - IMPORTANT: adds support for zero downtime upgrade to Que 2.x, see changelog below โฌ†๏ธ 2. Upgrade your project to Ruby 2.7 and Rails 6.x if it is not already โฌ†๏ธ 3. Upgrade your project to Que 2.x but stay on Ruby 2.7 - IMPORTANT: You will need to continue to run Que 1.x workers until all jobs enqueued using Que 1.x (i.e. with a job_schema_version of 1) have been finished. See below โฌ†๏ธ 4. Upgrade your project to Ruby 3

    NOTES:

    • If you were already running Ruby 2.7 and were not passing a hash literal as the last job argument, you may be able to upgrade a running system without draining the queue, though this is not recommended.
    • โฌ†๏ธ For all other cases, you will need to follow the recommended process above or first completely drain the queue (stop enqueuing new jobs and finish processing any jobs in the database, including cleaning out any expired jobs) before upgrading.

    ๐Ÿš€ Deploying Que 1.x and 2.x workers simultaneously:

    ๐Ÿ‘ท To run workers with two different versions of Que, you'll probably need to temporarily duplicate your gem bundle, with the Que version being the only difference. e.g.:

    • ๐Ÿ”’ Copy your Gemfile and Gemfile.lock into a directory called que-1-gemfile
    • Set a suitable Que version in each Gemfile
    • โšก๏ธ Update the bundle at que-1-gemfile/Gemfile.lock using BUNDLE_GEMFILE=que-1-gemfile/Gemfile bundle
    • ๐Ÿš€ Create a second deployment of Que, but with your que command prefixed with BUNDLE_GEMFILE=que-1-gemfile/Gemfile

    โฌ†๏ธ We'd appreciate feedback on your experience upgrading to and running Que 2. Feel free to post on our Discord, or if you run into trouble, open an issue on GitHub.


Previous changes from v1.4.0

    • ๐Ÿ›  Fixed
      • The poller will no longer sleep when jobs exist for only some of its priorities. It now skips sleeping when a poll returns more jobs than the lowest priority requested. #349.
        • An alternative was considered which skipped polling when only some of the waiting worker priorities would be fully utilised (diagram explanation); but this was decided against for code complexity reasons. #348
    • ๐Ÿ—„ Deprecated:
      • Deprecated --minimum-buffer-size option. It was not actually used, and will be removed in v2.0.0. #346
        • It became used in 1.0.0.beta4, and that changelog entry has been updated to reflect this.
    • ๐Ÿ“š Documentation:
      • Reformatted the changelog to be more consistent, including adding links to all issue/PR numbers. #347