From 768141596ac7ebceee2a5d55d2cd758c05e6af95 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:21:29 +0400 Subject: [PATCH 01/40] feat(ruby): bootstrap gem + first connect test Scaffold clients/ruby/: gemspec, Gemfile, Rakefile, lib skeleton (Pgque module, Pgque::Client with connect/close), and test_helper.rb with PGQUE_TEST_DSN gating + queue/consumer fixture helpers. First TDD pair: test_connect_returns_client (Pgque.connect returns a Pgque::Client wrapping an open PG::Connection; close finishes it). Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/.gitignore | 5 + clients/ruby/Gemfile | 3 + clients/ruby/LICENSE | 191 ++++++++++++++++++++++++++++++ clients/ruby/README.md | 52 ++++++++ clients/ruby/Rakefile | 11 ++ clients/ruby/lib/pgque.rb | 14 +++ clients/ruby/lib/pgque/client.rb | 25 ++++ clients/ruby/lib/pgque/version.rb | 5 + clients/ruby/pgque.gemspec | 33 ++++++ clients/ruby/test/test_connect.rb | 15 +++ clients/ruby/test/test_helper.rb | 49 ++++++++ 11 files changed, 403 insertions(+) create mode 100644 clients/ruby/.gitignore create mode 100644 clients/ruby/Gemfile create mode 100644 clients/ruby/LICENSE create mode 100644 clients/ruby/README.md create mode 100644 clients/ruby/Rakefile create mode 100644 clients/ruby/lib/pgque.rb create mode 100644 clients/ruby/lib/pgque/client.rb create mode 100644 clients/ruby/lib/pgque/version.rb create mode 100644 clients/ruby/pgque.gemspec create mode 100644 clients/ruby/test/test_connect.rb create mode 100644 clients/ruby/test/test_helper.rb diff --git a/clients/ruby/.gitignore b/clients/ruby/.gitignore new file mode 100644 index 00000000..b6303c5d --- /dev/null +++ b/clients/ruby/.gitignore @@ -0,0 +1,5 @@ +/Gemfile.lock +/pkg/ +/.bundle/ +/coverage/ +*.gem diff --git a/clients/ruby/Gemfile b/clients/ruby/Gemfile new file mode 100644 index 00000000..b4e2a20b --- /dev/null +++ b/clients/ruby/Gemfile @@ -0,0 +1,3 @@ +source "https://rubygems.org" + +gemspec diff --git a/clients/ruby/LICENSE b/clients/ruby/LICENSE new file mode 100644 index 00000000..62a462ad --- /dev/null +++ b/clients/ruby/LICENSE @@ -0,0 +1,191 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to the Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by the Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding any notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + Copyright 2026 Nikolay Samokhvalov + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/clients/ruby/README.md b/clients/ruby/README.md new file mode 100644 index 00000000..7b60d8f4 --- /dev/null +++ b/clients/ruby/README.md @@ -0,0 +1,52 @@ +# pgque + +Ruby client for [PgQue](https://github.com/NikolayS/pgque) — the PgQ-based +universal PostgreSQL queue. Thin wrapper over `pgque-api` SQL functions: +`send`, `receive`, `ack`, `nack`, `force_next_tick`, plus a polling +`Consumer` with `LISTEN`/`NOTIFY` wakeup. + +## Install + +```bash +gem install pgque --pre +``` + +`--pre` is required while v0.2.0 is in release-candidate; the latest +published version is `0.2.0.rc.1`. Pin the exact version if you prefer: + +```ruby +gem "pgque", "0.2.0.rc.1" +``` + +Requires Ruby 3.1+ and PostgreSQL 14+ with the PgQue schema installed +(`\i pgque.sql` — no extension required). + +## Database permissions + +The connecting database role needs `pgque_reader` to consume (`receive`, +`ack`, `nack`) and `pgque_writer` to produce (`send`, `send_batch`). The +two are **siblings** — neither inherits the other. An app that both +produces and consumes must be granted **both** roles: + +```sql +grant pgque_reader to your_app_user; +grant pgque_writer to your_app_user; +``` + +See [`docs/reference.md` — Roles and grants](../../docs/reference.md#roles-and-grants). + +## Tests + +Integration tests require a running PostgreSQL with the PgQue schema +installed. Set `PGQUE_TEST_DSN` and run rake: + +```bash +PGQUE_TEST_DSN=postgresql://postgres:pgque_test@localhost/pgque_test \ + bundle exec rake test +``` + +Without `PGQUE_TEST_DSN`, the tests skip. + +## License + +Apache-2.0. Copyright 2026 Nikolay Samokhvalov. diff --git a/clients/ruby/Rakefile b/clients/ruby/Rakefile new file mode 100644 index 00000000..174c5caa --- /dev/null +++ b/clients/ruby/Rakefile @@ -0,0 +1,11 @@ +require "rake/testtask" + +Rake::TestTask.new(:test) do |t| + t.libs << "lib" + t.libs << "test" + t.test_files = FileList["test/test_*.rb"] + t.warning = false + t.verbose = true +end + +task default: :test diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb new file mode 100644 index 00000000..630e6a87 --- /dev/null +++ b/clients/ruby/lib/pgque.rb @@ -0,0 +1,14 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. +# PgQue includes code derived from PgQ (ISC license, +# Marko Kreen / Skype Technologies OU). + +require "pg" + +require "pgque/version" +require "pgque/client" + +module Pgque + def self.connect(dsn, autocommit: false) + Client.connect(dsn, autocommit: autocommit) + end +end diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb new file mode 100644 index 00000000..1e83e040 --- /dev/null +++ b/clients/ruby/lib/pgque/client.rb @@ -0,0 +1,25 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. +# PgQue includes code derived from PgQ (ISC license, +# Marko Kreen / Skype Technologies OU). + +module Pgque + class Client + attr_reader :conn + + def self.connect(dsn, autocommit: false) + conn = PG.connect(dsn) + new(conn, owns_conn: true) + end + + def initialize(conn, owns_conn: false) + @conn = conn + @owns_conn = owns_conn + end + + def close + return unless @owns_conn + return if @conn.finished? + @conn.close + end + end +end diff --git a/clients/ruby/lib/pgque/version.rb b/clients/ruby/lib/pgque/version.rb new file mode 100644 index 00000000..602515a7 --- /dev/null +++ b/clients/ruby/lib/pgque/version.rb @@ -0,0 +1,5 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +module Pgque + VERSION = "0.2.0.rc.1" +end diff --git a/clients/ruby/pgque.gemspec b/clients/ruby/pgque.gemspec new file mode 100644 index 00000000..86725251 --- /dev/null +++ b/clients/ruby/pgque.gemspec @@ -0,0 +1,33 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "lib/pgque/version" + +Gem::Specification.new do |spec| + spec.name = "pgque" + spec.version = Pgque::VERSION + spec.authors = ["Nikolay Samokhvalov", "Dalto Curvelano Jr"] + spec.email = ["nik@postgres.ai", "daltojr@gmail.com"] + + spec.summary = "Ruby client for PgQue -- PgQ Universal Edition" + spec.description = "Thin Ruby wrapper over the pgque SQL API: send, " \ + "send_batch, receive, ack, nack, force_next_tick, " \ + "plus a polling Consumer with LISTEN/NOTIFY wakeup." + spec.homepage = "https://github.com/NikolayS/pgque" + spec.license = "Apache-2.0" + spec.required_ruby_version = ">= 3.1.0" + + spec.metadata["homepage_uri"] = spec.homepage + spec.metadata["source_code_uri"] = spec.homepage + spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues" + spec.metadata["documentation_uri"] = "#{spec.homepage}/blob/main/docs/reference.md" + spec.metadata["changelog_uri"] = "#{spec.homepage}/releases" + + spec.files = Dir.glob("lib/**/*.rb") + + %w[README.md LICENSE].select { |f| File.exist?(f) } + spec.require_paths = ["lib"] + + spec.add_dependency "pg", ">= 1.5", "< 2.0" + + spec.add_development_dependency "minitest", "~> 5.0" + spec.add_development_dependency "rake", "~> 13.0" +end diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb new file mode 100644 index 00000000..2bbdd604 --- /dev/null +++ b/clients/ruby/test/test_connect.rb @@ -0,0 +1,15 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestConnect < Minitest::Test + include PgqueTest::Helpers + + def test_connect_returns_client + client = Pgque.connect(dsn) + assert_instance_of Pgque::Client, client + refute client.conn.finished? + client.close + assert client.conn.finished? + end +end diff --git a/clients/ruby/test/test_helper.rb b/clients/ruby/test/test_helper.rb new file mode 100644 index 00000000..a9aa916a --- /dev/null +++ b/clients/ruby/test/test_helper.rb @@ -0,0 +1,49 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require "minitest/autorun" +require "securerandom" +require "pgque" + +PGQUE_TEST_DSN = ENV["PGQUE_TEST_DSN"] + +module PgqueTest + module Helpers + def setup + skip "PGQUE_TEST_DSN not set" unless PGQUE_TEST_DSN + super if defined?(super) + end + + def dsn + PGQUE_TEST_DSN + end + + def unique_queue_name + base = name.to_s.gsub(/[^a-z0-9_]/i, "_") + "rbt_#{base[0, 40]}_#{SecureRandom.hex(4)}" + end + + def unique_consumer_name + base = name.to_s.gsub(/[^a-z0-9_]/i, "_") + "rbt_c_#{base[0, 38]}_#{SecureRandom.hex(4)}" + end + + def with_queue + conn = PG.connect(PGQUE_TEST_DSN) + q = unique_queue_name + c = unique_consumer_name + conn.exec_params("select pgque.create_queue($1)", [q]) + conn.exec_params("select pgque.register_consumer($1, $2)", [q, c]) + yield q, c, conn + ensure + if conn && !conn.finished? + begin + conn.exec_params("select pgque.unregister_consumer($1, $2)", [q, c]) if q && c + conn.exec_params("select pgque.drop_queue($1, true)", [q]) if q + rescue PG::Error + # cleanup is best-effort + end + conn.close + end + end + end +end From 0d582f864f176197acce62ee265ee8f9e40c02e1 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:23:06 +0400 Subject: [PATCH 02/40] feat(ruby): connect block form auto-closes client Pgque.connect(dsn) { |c| ... } yields the client and closes it on block exit, mirroring the Python context-manager pattern. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 9 ++++++++- clients/ruby/test/test_connect.rb | 9 +++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index 630e6a87..e3e41273 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -9,6 +9,13 @@ module Pgque def self.connect(dsn, autocommit: false) - Client.connect(dsn, autocommit: autocommit) + client = Client.connect(dsn, autocommit: autocommit) + return client unless block_given? + + begin + yield client + ensure + client.close + end end end diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 2bbdd604..a0245c07 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -12,4 +12,13 @@ def test_connect_returns_client client.close assert client.conn.finished? end + + def test_connect_block_form_closes_on_exit + captured = nil + Pgque.connect(dsn) do |client| + captured = client + refute client.conn.finished? + end + assert captured.conn.finished? + end end From 4b66086ba3fb3144ffc68af63acda5480d4fb910 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:23:56 +0400 Subject: [PATCH 03/40] feat(ruby): bad DSN raises Pgque::ConnectionError Introduce Pgque::Error base class and Pgque::ConnectionError; rescue PG::ConnectionBad in Pgque::Client.connect and re-raise wrapped. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 1 + clients/ruby/lib/pgque/client.rb | 2 ++ clients/ruby/lib/pgque/errors.rb | 7 +++++++ clients/ruby/test/test_connect.rb | 8 ++++++++ 4 files changed, 18 insertions(+) create mode 100644 clients/ruby/lib/pgque/errors.rb diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index e3e41273..3c29d584 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -5,6 +5,7 @@ require "pg" require "pgque/version" +require "pgque/errors" require "pgque/client" module Pgque diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 1e83e040..a7ca1cd7 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -9,6 +9,8 @@ class Client def self.connect(dsn, autocommit: false) conn = PG.connect(dsn) new(conn, owns_conn: true) + rescue PG::ConnectionBad => e + raise ConnectionError, e.message end def initialize(conn, owns_conn: false) diff --git a/clients/ruby/lib/pgque/errors.rb b/clients/ruby/lib/pgque/errors.rb new file mode 100644 index 00000000..e1443513 --- /dev/null +++ b/clients/ruby/lib/pgque/errors.rb @@ -0,0 +1,7 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +module Pgque + class Error < StandardError; end + + class ConnectionError < Error; end +end diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index a0245c07..450165fc 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -22,3 +22,11 @@ def test_connect_block_form_closes_on_exit assert captured.conn.finished? end end + +class TestConnectBadDsn < Minitest::Test + def test_connect_bad_dsn_raises_pgque_connection_error + assert_raises(Pgque::ConnectionError) do + Pgque.connect("postgresql://nobody:wrong@localhost:1/nonexistent_db_xyz") + end + end +end From 45d2ef2e8e5d47473b20a3b6635a6fd9c4486259 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:24:16 +0400 Subject: [PATCH 04/40] feat(ruby): close is a no-op for external connections Pgque::Client.new(raw_conn) defaults owns_conn:false, so close leaves the caller's connection open. Test guards the contract. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_connect.rb | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 450165fc..5c8451b7 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -21,6 +21,17 @@ def test_connect_block_form_closes_on_exit end assert captured.conn.finished? end + + def test_external_conn_is_not_closed_by_close + raw = PG.connect(dsn) + begin + client = Pgque::Client.new(raw) + client.close + refute raw.finished? + ensure + raw.close + end + end end class TestConnectBadDsn < Minitest::Test From d804ebddeacc7e0f23633e9a40c20e164d9f024b Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:24:54 +0400 Subject: [PATCH 05/40] feat(ruby): autocommit flag stored on client Pgque.connect(autocommit:) is propagated to the Client and exposed via client.autocommit?. The Ruby pg gem has no autocommit attribute on the connection itself, so the flag is informational here -- callers manage explicit transactions with conn.transaction { } as Ruby idiom dictates. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/client.rb | 9 +++++++-- clients/ruby/test/test_connect.rb | 9 +++++++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index a7ca1cd7..cf2793a9 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -8,14 +8,19 @@ class Client def self.connect(dsn, autocommit: false) conn = PG.connect(dsn) - new(conn, owns_conn: true) + new(conn, owns_conn: true, autocommit: autocommit) rescue PG::ConnectionBad => e raise ConnectionError, e.message end - def initialize(conn, owns_conn: false) + def initialize(conn, owns_conn: false, autocommit: false) @conn = conn @owns_conn = owns_conn + @autocommit = autocommit + end + + def autocommit? + @autocommit end def close diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 5c8451b7..31fa54ac 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -32,6 +32,15 @@ def test_external_conn_is_not_closed_by_close raw.close end end + + def test_autocommit_flag + Pgque.connect(dsn, autocommit: true) do |client| + assert client.autocommit? + end + Pgque.connect(dsn) do |client| + refute client.autocommit? + end + end end class TestConnectBadDsn < Minitest::Test From cf1b6eda4fed4f5a83f7dfa266de55b9843ba9bb Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:25:08 +0400 Subject: [PATCH 06/40] feat(ruby): close is idempotent Calling close twice does not raise; the finished? guard already in place makes the second call a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_connect.rb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 31fa54ac..518f585a 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -41,6 +41,12 @@ def test_autocommit_flag refute client.autocommit? end end + + def test_close_is_idempotent + client = Pgque.connect(dsn) + client.close + client.close + end end class TestConnectBadDsn < Minitest::Test From fe75ed6eb347a9b81a827878ce5e6631e7146a59 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:27:41 +0400 Subject: [PATCH 07/40] feat(ruby): send returns integer event id Pgque::Client#send(queue, payload) inserts via pgque.send(queue, payload::jsonb), encoding Hash/Array as JSON and nil as JSON null. Returns the bigint event id as a Ruby Integer. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 1 + clients/ruby/lib/pgque/client.rb | 18 ++++++++++++++++++ clients/ruby/test/test_send.rb | 16 ++++++++++++++++ 3 files changed, 35 insertions(+) create mode 100644 clients/ruby/test/test_send.rb diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index 3c29d584..4f1e44e2 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -2,6 +2,7 @@ # PgQue includes code derived from PgQ (ISC license, # Marko Kreen / Skype Technologies OU). +require "json" require "pg" require "pgque/version" diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index cf2793a9..f43b0509 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -28,5 +28,23 @@ def close return if @conn.finished? @conn.close end + + def send(queue, payload) + result = @conn.exec_params( + "select pgque.send($1, $2::jsonb)", + [queue, encode_payload(payload)], + ) + result.values[0][0].to_i + end + + private + + def encode_payload(payload) + case payload + when Hash, Array then JSON.dump(payload) + when nil then "null" + else payload + end + end end end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb new file mode 100644 index 00000000..d4ad756c --- /dev/null +++ b/clients/ruby/test/test_send.rb @@ -0,0 +1,16 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestSend < Minitest::Test + include PgqueTest::Helpers + + def test_send_returns_int_event_id + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + eid = client.send(queue, { "order_id" => 42 }) + assert_kind_of Integer, eid + assert_operator eid, :>, 0 + end + end +end From 001fd00c2dc83d26b1faa56ec1d00553445934ef Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:28:29 +0400 Subject: [PATCH 08/40] feat(ruby): send accepts type: kwarg Add type: keyword arg to Pgque::Client#send. When type is empty/nil/ "default" the 2-arg pgque.send is used; otherwise the 3-arg form. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/client.rb | 19 ++++++++++++++----- clients/ruby/test/test_send.rb | 8 ++++++++ 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index f43b0509..adeb4593 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -29,11 +29,20 @@ def close @conn.close end - def send(queue, payload) - result = @conn.exec_params( - "select pgque.send($1, $2::jsonb)", - [queue, encode_payload(payload)], - ) + def send(queue, payload, type: "default") + encoded = encode_payload(payload) + result = + if type && type != "" && type != "default" + @conn.exec_params( + "select pgque.send($1, $2, $3::jsonb)", + [queue, type, encoded], + ) + else + @conn.exec_params( + "select pgque.send($1, $2::jsonb)", + [queue, encoded], + ) + end result.values[0][0].to_i end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index d4ad756c..2ecd30d1 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -13,4 +13,12 @@ def test_send_returns_int_event_id assert_operator eid, :>, 0 end end + + def test_send_with_explicit_type + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + eid = client.send(queue, { "id" => 1 }, type: "order.created") + assert_kind_of Integer, eid + end + end end From aedc6edfff3f2701976d3f5d4261fa75963035d5 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:28:52 +0400 Subject: [PATCH 09/40] feat(ruby): send accepts Pgque::Event payload Add Pgque::Event value class. When passed to send, its type and payload override the keyword args, mirroring the Python Event handling. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 1 + clients/ruby/lib/pgque/client.rb | 4 ++++ clients/ruby/lib/pgque/event.rb | 13 +++++++++++++ clients/ruby/test/test_send.rb | 9 +++++++++ 4 files changed, 27 insertions(+) create mode 100644 clients/ruby/lib/pgque/event.rb diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index 4f1e44e2..ce5b777c 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -7,6 +7,7 @@ require "pgque/version" require "pgque/errors" +require "pgque/event" require "pgque/client" module Pgque diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index adeb4593..4655a758 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -30,6 +30,10 @@ def close end def send(queue, payload, type: "default") + if payload.is_a?(Event) + type = payload.type + payload = payload.payload + end encoded = encode_payload(payload) result = if type && type != "" && type != "default" diff --git a/clients/ruby/lib/pgque/event.rb b/clients/ruby/lib/pgque/event.rb new file mode 100644 index 00000000..4e78a63e --- /dev/null +++ b/clients/ruby/lib/pgque/event.rb @@ -0,0 +1,13 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +module Pgque + class Event + attr_reader :payload, :type, :extra + + def initialize(payload:, type: "default", extra: {}) + @payload = payload + @type = type + @extra = extra + end + end +end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 2ecd30d1..7190879c 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -21,4 +21,13 @@ def test_send_with_explicit_type assert_kind_of Integer, eid end end + + def test_send_event_object + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + event = Pgque::Event.new(payload: { "x" => 1 }, type: "custom.t") + eid = client.send(queue, event) + assert_kind_of Integer, eid + end + end end From 7662033a1bd6fa188bb53f480352dbceac0c0ceb Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:29:29 +0400 Subject: [PATCH 10/40] feat(ruby): send passes through JSON-string payloads Strings are forwarded as-is to ::jsonb cast (caller is responsible for valid JSON text). encode_payload's else branch already handles this. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_send.rb | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 7190879c..a25cd07d 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -30,4 +30,12 @@ def test_send_event_object assert_kind_of Integer, eid end end + + def test_send_str_payload_passes_through + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + eid = client.send(queue, '"plain string"') + assert_kind_of Integer, eid + end + end end From 253b53fd6bd176ffd8adaf8750e9ebbbb0a95898 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:29:41 +0400 Subject: [PATCH 11/40] feat(ruby): send nil payload stores JSON null encode_payload returns the literal "null" so the ::jsonb cast yields JSON null rather than SQL NULL. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_send.rb | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index a25cd07d..af567b14 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -38,4 +38,12 @@ def test_send_str_payload_passes_through assert_kind_of Integer, eid end end + + def test_send_nil_payload + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + eid = client.send(queue, nil) + assert_kind_of Integer, eid + end + end end From 5a5965f4aa07d0d9524c923af80cf0da19f2bd7b Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:31:03 +0400 Subject: [PATCH 12/40] feat(ruby): send_batch returns ids in input order Build a PG text-array literal from JSON-encoded payloads, call pgque.send_batch(...)::jsonb[], and unnest the bigint[] result so ordering is preserved row-by-row. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/client.rb | 18 ++++++++++++++++++ clients/ruby/test/test_send.rb | 12 ++++++++++++ 2 files changed, 30 insertions(+) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 4655a758..034c9703 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -50,6 +50,16 @@ def send(queue, payload, type: "default") result.values[0][0].to_i end + def send_batch(queue, type, payloads) + encoded = payloads.map { |p| encode_payload(p) } + array_literal = pg_text_array(encoded) + result = @conn.exec_params( + "select unnest(pgque.send_batch($1, $2, $3::jsonb[]))", + [queue, type, array_literal], + ) + result.values.map { |r| r[0].to_i } + end + private def encode_payload(payload) @@ -59,5 +69,13 @@ def encode_payload(payload) else payload end end + + def pg_text_array(strings) + escaped = strings.map do |s| + inner = s.to_s.gsub('\\') { '\\\\' }.gsub('"') { '\\"' } + "\"#{inner}\"" + end + "{#{escaped.join(',')}}" + end end end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index af567b14..67dc1e6a 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -46,4 +46,16 @@ def test_send_nil_payload assert_kind_of Integer, eid end end + + def test_send_batch_returns_ids_in_order + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + ids = client.send_batch(queue, "batch.test", [ + { "n" => 1 }, { "n" => 2 }, { "n" => 3 }, { "n" => 4 } + ]) + assert_equal 4, ids.size + assert ids.all? { |i| i.is_a?(Integer) } + assert_equal ids.sort, ids + end + end end From 6538580e7d4709c43e3cb7306a9a3e1c406e8627 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:36:12 +0400 Subject: [PATCH 13/40] feat(ruby): consumer-side primitive API (receive, ack, nack) Ports the rest of the per-message client surface from the Python client: - Pgque::Message value class with msg_id/batch_id/type/payload/retry_count/ created_at/extra1..4. JSONB payloads are decoded to native Ruby via JSON.parse; timestamps via Time.parse. - Pgque::Client#receive(queue, consumer, max_messages=100): returns [Pgque::Message]. Empty array when no batch is current. - Pgque::Client#ack(batch_id): finishes the batch, returns 1 (ok) or 0 (stale/double ack). - Pgque::Client#force_next_tick(queue): returns last tick id or nil. - Pgque::Client#nack(batch_id, msg, retry_after:, reason:): routes the message through retry_queue (or DLQ once queue_max_retries is hit). - Error wrapping: PG::Error from any client method is mapped to Pgque::QueueNotFound / Pgque::BatchNotFound / Pgque::Error based on the message text, mirroring the Python wrapper. Tests: 30 runs / 79 assertions, all green against Postgres 18 with pgque.sql installed. Closes the test_send round-trip cases (unicode, large, jsonb_round_trip, batch mixed/nil, missing-queue raise, SQL form selection via FakeConn) plus full test_receive, test_smoke, and test_nack ports. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 2 + clients/ruby/lib/pgque/client.rb | 88 ++++++++++++++++++++ clients/ruby/lib/pgque/errors.rb | 6 ++ clients/ruby/lib/pgque/message.rb | 23 ++++++ clients/ruby/test/test_nack.rb | 75 +++++++++++++++++ clients/ruby/test/test_receive.rb | 89 ++++++++++++++++++++ clients/ruby/test/test_send.rb | 130 ++++++++++++++++++++++++++++++ clients/ruby/test/test_smoke.rb | 33 ++++++++ 8 files changed, 446 insertions(+) create mode 100644 clients/ruby/lib/pgque/message.rb create mode 100644 clients/ruby/test/test_nack.rb create mode 100644 clients/ruby/test/test_receive.rb create mode 100644 clients/ruby/test/test_smoke.rb diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index ce5b777c..74ee44c5 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -3,11 +3,13 @@ # Marko Kreen / Skype Technologies OU). require "json" +require "time" require "pg" require "pgque/version" require "pgque/errors" require "pgque/event" +require "pgque/message" require "pgque/client" module Pgque diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 034c9703..02ca1dd9 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -48,6 +48,8 @@ def send(queue, payload, type: "default") ) end result.values[0][0].to_i + rescue PG::Error => e + raise wrap_sql_error(e) end def send_batch(queue, type, payloads) @@ -58,6 +60,58 @@ def send_batch(queue, type, payloads) [queue, type, array_literal], ) result.values.map { |r| r[0].to_i } + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def receive(queue, consumer, max_messages = 100) + result = @conn.exec_params( + "select * from pgque.receive($1, $2, $3)", + [queue, consumer, max_messages], + ) + result.values.map { |row| row_to_message(row) } + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def ack(batch_id) + result = @conn.exec_params("select pgque.ack($1)", [batch_id]) + result.values[0][0].to_i + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def force_next_tick(queue) + result = @conn.exec_params("select pgque.force_next_tick($1)", [queue]) + v = result.values[0][0] + v.nil? || v.empty? ? nil : v.to_i + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def nack(batch_id, msg, retry_after: 60, reason: nil) + payload_str = case msg.payload + when Hash, Array then JSON.dump(msg.payload) + when nil then "null" + else msg.payload.to_s + end + created_at_str = msg.created_at.respond_to?(:iso8601) ? + msg.created_at.iso8601(6) : msg.created_at + + @conn.exec_params( + "select pgque.nack($1, " \ + "ROW($2, $3, $4, $5::jsonb, $6, $7, $8, $9, $10, $11)::pgque.message, " \ + "$12::interval, $13)", + [ + batch_id, msg.msg_id, msg.batch_id, msg.type, payload_str, + msg.retry_count, created_at_str, + msg.extra1, msg.extra2, msg.extra3, msg.extra4, + "#{retry_after} seconds", reason, + ], + ) + nil + rescue PG::Error => e + raise wrap_sql_error(e) end private @@ -77,5 +131,39 @@ def pg_text_array(strings) end "{#{escaped.join(',')}}" end + + def row_to_message(row) + Message.new( + msg_id: row[0].to_i, + batch_id: row[1].to_i, + type: row[2], + payload: parse_jsonb(row[3]), + retry_count: row[4].nil? ? nil : row[4].to_i, + created_at: row[5].nil? ? nil : Time.parse(row[5]), + extra1: row[6], + extra2: row[7], + extra3: row[8], + extra4: row[9], + ) + end + + def parse_jsonb(text) + return nil if text.nil? + JSON.parse(text) + rescue JSON::ParserError + text + end + + def wrap_sql_error(error) + msg = error.message.to_s + low = msg.downcase + if low.include?("queue not found") + QueueNotFound.new(msg) + elsif low.include?("batch not found") + BatchNotFound.new(msg) + else + Error.new(msg) + end + end end end diff --git a/clients/ruby/lib/pgque/errors.rb b/clients/ruby/lib/pgque/errors.rb index e1443513..0db8cde2 100644 --- a/clients/ruby/lib/pgque/errors.rb +++ b/clients/ruby/lib/pgque/errors.rb @@ -4,4 +4,10 @@ module Pgque class Error < StandardError; end class ConnectionError < Error; end + + class QueueNotFound < Error; end + + class BatchNotFound < Error; end + + class ConsumerNotFound < Error; end end diff --git a/clients/ruby/lib/pgque/message.rb b/clients/ruby/lib/pgque/message.rb new file mode 100644 index 00000000..d5a299c7 --- /dev/null +++ b/clients/ruby/lib/pgque/message.rb @@ -0,0 +1,23 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +module Pgque + class Message + attr_reader :msg_id, :batch_id, :type, :payload, :retry_count, + :created_at, :extra1, :extra2, :extra3, :extra4 + + def initialize(msg_id:, batch_id:, type:, payload:, retry_count:, + created_at:, extra1: nil, extra2: nil, extra3: nil, + extra4: nil) + @msg_id = msg_id + @batch_id = batch_id + @type = type + @payload = payload + @retry_count = retry_count + @created_at = created_at + @extra1 = extra1 + @extra2 = extra2 + @extra3 = extra3 + @extra4 = extra4 + end + end +end diff --git a/clients/ruby/test/test_nack.rb b/clients/ruby/test/test_nack.rb new file mode 100644 index 00000000..55502de5 --- /dev/null +++ b/clients/ruby/test/test_nack.rb @@ -0,0 +1,75 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestNack < Minitest::Test + include PgqueTest::Helpers + + def enqueue_and_receive(client, queue, consumer, payload, conn) + client.send(queue, payload) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + msgs[0] + end + + def test_nack_routes_to_retry_queue + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + msg = enqueue_and_receive(client, queue, consumer, { "k" => "retry" }, conn) + client.nack(msg.batch_id, msg, retry_after: 0) + client.ack(msg.batch_id) + + retry_count = conn.exec_params( + "select count(*) from pgque.retry_queue rq " \ + "join pgque.queue q on q.queue_id = rq.ev_queue " \ + "where q.queue_name = $1", + [queue], + ).values[0][0].to_i + assert_equal 1, retry_count + + dlq_count = conn.exec_params( + "select count(*) from pgque.dead_letter dl " \ + "join pgque.queue q on q.queue_id = dl.dl_queue_id " \ + "where q.queue_name = $1", + [queue], + ).values[0][0].to_i + assert_equal 0, dlq_count + end + end + + def test_nack_routes_to_dlq_at_max_retries + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + conn.exec_params( + "update pgque.queue set queue_max_retries = 0 where queue_name = $1", + [queue], + ) + + msg = enqueue_and_receive(client, queue, consumer, { "k" => "doomed" }, conn) + client.nack(msg.batch_id, msg, retry_after: 0, reason: "poison pill") + client.ack(msg.batch_id) + + dlq_count = conn.exec_params( + "select count(*) from pgque.dead_letter dl " \ + "join pgque.queue q on q.queue_id = dl.dl_queue_id " \ + "where q.queue_name = $1", + [queue], + ).values[0][0].to_i + assert_equal 1, dlq_count + end + end + + def test_nack_invalid_batch_raises + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + msg = enqueue_and_receive(client, queue, consumer, { "x" => 1 }, conn) + client.ack(msg.batch_id) + + assert_raises(Pgque::Error) do + client.nack(msg.batch_id, msg, retry_after: 0) + end + end + end +end diff --git a/clients/ruby/test/test_receive.rb b/clients/ruby/test/test_receive.rb new file mode 100644 index 00000000..b0b9a77d --- /dev/null +++ b/clients/ruby/test/test_receive.rb @@ -0,0 +1,89 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestReceive < Minitest::Test + include PgqueTest::Helpers + + def test_receive_empty_when_no_tick + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "a" => 1 }) + msgs = client.receive(queue, consumer, 10) + assert_equal [], msgs + end + end + + def test_receive_returns_messages_after_tick + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "key" => "value" }) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + m = msgs[0] + refute_nil m.batch_id + refute_nil m.msg_id + assert_equal "default", m.type + assert_equal({ "key" => "value" }, m.payload) + end + end + + def test_ack_advances_position + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "k" => 1 }) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + client.ack(msgs[0].batch_id) + msgs2 = client.receive(queue, consumer, 10) + assert_equal [], msgs2 + end + end + + def test_receive_returns_at_most_max_messages + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + 5.times { |i| client.send(queue, { "i" => i }) } + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 3) + assert_equal 3, msgs.size + client.ack(msgs[0].batch_id) + end + end + + def test_receive_preserves_event_type + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "a" => 1 }, type: "evt.alpha") + client.send(queue, { "b" => 2 }, type: "evt.beta") + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + types = msgs.map(&:type).sort + assert_equal ["evt.alpha", "evt.beta"], types + client.ack(msgs[0].batch_id) + end + end + + def test_message_timestamp_round_trip + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + before = Time.now.utc - 5 + client.send(queue, { "x" => 1 }) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + after = Time.now.utc + 5 + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + assert_kind_of Time, msgs[0].created_at + assert_operator msgs[0].created_at, :>=, before + assert_operator msgs[0].created_at, :<=, after + client.ack(msgs[0].batch_id) + end + end +end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 67dc1e6a..640794c1 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -58,4 +58,134 @@ def test_send_batch_returns_ids_in_order assert_equal ids.sort, ids end end + + def test_send_unicode_payload + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + payload = { "text" => "héllo wörld 🎉 — ünicode тест" } + client.send(queue, payload) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + assert_equal payload, msgs[0].payload + client.ack(msgs[0].batch_id) + end + end + + def test_send_large_payload + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + big = { "data" => "x" * 100_000 } + client.send(queue, big) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size + assert_equal big, msgs[0].payload + client.ack(msgs[0].batch_id) + end + end + + def test_jsonb_payload_round_trip + cases = [ + [{ "key" => "val", "n" => 1 }, { "key" => "val", "n" => 1 }], + [[1, "two", nil], [1, "two", nil]], + ['"just a string"', "just a string"], + ["42", 42], + ["null", nil], + ] + cases.each do |payload, expected| + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, payload) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size, "no message for payload=#{payload.inspect}" + assert_equal expected, msgs[0].payload, + "payload=#{payload.inspect} did not round-trip" + client.ack(msgs[0].batch_id) + end + end + end + + def test_send_batch_mixed_payloads_preserve_order + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + payloads = [{ "a" => 1 }, nil, "42"] + expected = [{ "a" => 1 }, nil, 42] + ids = client.send_batch(queue, "batch.mixed", payloads) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal ids, msgs.map(&:msg_id) + assert_equal expected, msgs.map(&:payload) + client.ack(msgs[0].batch_id) + end + end + + def test_send_batch_nil_payload_produces_json_null + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send_batch(queue, "default", [nil]) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size, "send_batch([nil]) should produce 1 message" + assert_nil msgs[0].payload, "payload must be JSON null, not SQL NULL" + client.ack(msgs[0].batch_id) + end + end + + def test_send_to_missing_queue_raises + conn = PG.connect(PGQUE_TEST_DSN) + begin + client = Pgque::Client.new(conn) + assert_raises(Pgque::Error) do + client.send("does_not_exist_xyz_12345", { "x" => 1 }) + end + ensure + conn.close + end + end +end + +class TestSendSqlForm < Minitest::Test + # Capture exec_params calls without a real DB. + class FakeConn + attr_reader :sql_used, :params_used + + def exec_params(sql, params) + @sql_used = sql + @params_used = params + FakeResult.new + end + + class FakeResult + def values + [["999"]] + end + end + end + + def test_2arg_form_for_default_type + [nil, "", "default"].each do |type_val| + conn = FakeConn.new + client = Pgque::Client.new(conn) + eid = client.send("q", { "x" => 1 }, type: type_val) + assert_equal 999, eid + assert_includes conn.sql_used, "send($1, $2::jsonb)" + refute_includes conn.sql_used, "send($1, $2, $3::jsonb)", + "type=#{type_val.inspect} should use 2-arg form" + end + end + + def test_3arg_form_for_custom_type + conn = FakeConn.new + client = Pgque::Client.new(conn) + eid = client.send("q", { "x" => 1 }, type: "custom") + assert_equal 999, eid + assert_includes conn.sql_used, "send($1, $2, $3::jsonb)" + end end diff --git a/clients/ruby/test/test_smoke.rb b/clients/ruby/test/test_smoke.rb new file mode 100644 index 00000000..18ca88a6 --- /dev/null +++ b/clients/ruby/test/test_smoke.rb @@ -0,0 +1,33 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestSmoke < Minitest::Test + include PgqueTest::Helpers + + def test_smoke_send_receive_ack + queue = unique_queue_name + consumer_n = unique_consumer_name + Pgque.connect(dsn) do |client| + client.conn.exec_params("select pgque.create_queue($1)", [queue]) + client.conn.exec_params("select pgque.subscribe($1, $2)", [queue, consumer_n]) + + begin + client.send(queue, { "hello" => "world" }, type: "smoke.test") + client.conn.exec_params("select pgque.force_next_tick($1)", [queue]) + client.conn.exec_params("select pgque.ticker($1)", [queue]) + + msgs = client.receive(queue, consumer_n, 10) + assert_equal 1, msgs.size + assert_equal "smoke.test", msgs[0].type + assert_equal({ "hello" => "world" }, msgs[0].payload) + + client.ack(msgs[0].batch_id) + ensure + client.conn.exec_params("select pgque.unregister_consumer($1, $2)", + [queue, consumer_n]) rescue nil + client.conn.exec_params("select pgque.drop_queue($1, true)", [queue]) rescue nil + end + end + end +end From 2e7cd7a9b1ed0ed08c5b95cf9e2d28a717986baa Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:41:02 +0400 Subject: [PATCH 14/40] feat(ruby): polling Consumer with LISTEN/NOTIFY wakeup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pgque::Consumer (lib/pgque/consumer.rb) — synchronous polling consumer that mirrors the Python version's behavior end-to-end: - consumer.on(type) { |msg| ... } registers a handler; "*" is the catch-all. - start blocks in a poll loop, processing one batch at a time inside a conn.transaction { } block. Each msg dispatches to its handler; exceptions trigger nack with retry_after; messages with no handler are nacked (or acked when unknown_handler_policy: "ack"), with a warn either way. The batch is acked at the end -- unless any nack itself failed, in which case the transaction commits without acking so PgQ redelivers. ack returning 0 (stale/double ack) logs WARN. - LISTEN/NOTIFY wakeup uses bounded ~0.5s slices around PG::Connection#wait_for_notify so stop() unblocks within ~1s even when poll_interval is large. Buffered NOTIFYs from the prior poll are drained before waiting, mirroring the Python regression for issue #158. - Signal handlers (TERM/INT) are installed only when start() runs on the main thread; tests and embedded use can call stop() from any thread. - Cooperative-mode hooks (subconsumer:, dead_interval:) are wired up but the underlying receive_coop SQL surface lands in a follow-up. Constructor validates that dead_interval requires subconsumer. Tests: 47 runs / 119 assertions, all green. Covers max_messages defaults, on()-dispatch, default-handler catch-all, error-driven nack, unknown-type nack vs. opt-in ack, stop()-promptness from a worker thread, NOTIFY wakeup before poll_interval, and the partial- batch case where good messages are finished by the batch ack while the failing one survives in retry_queue. Unit tests stub Pgque::Client.new with a SpyClient to assert _poll_once passes max_messages through to receive() without needing a real DB round-trip. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 1 + clients/ruby/lib/pgque/consumer.rb | 238 ++++++++++++ clients/ruby/test/test_consumer.rb | 359 ++++++++++++++++++ .../ruby/test/test_consumer_listen_stop.rb | 84 ++++ clients/ruby/test/test_send.rb | 9 +- 5 files changed, 689 insertions(+), 2 deletions(-) create mode 100644 clients/ruby/lib/pgque/consumer.rb create mode 100644 clients/ruby/test/test_consumer.rb create mode 100644 clients/ruby/test/test_consumer_listen_stop.rb diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index 74ee44c5..576c4267 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -11,6 +11,7 @@ require "pgque/event" require "pgque/message" require "pgque/client" +require "pgque/consumer" module Pgque def self.connect(dsn, autocommit: false) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb new file mode 100644 index 00000000..22e3e053 --- /dev/null +++ b/clients/ruby/lib/pgque/consumer.rb @@ -0,0 +1,238 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. +# PgQue includes code derived from PgQ (ISC license, +# Marko Kreen / Skype Technologies OU). + +require "logger" + +module Pgque + class Consumer + DEFAULT_MAX_MESSAGES = 2_147_483_647 + WAIT_SLICE_SECONDS = 0.5 + + attr_reader :dsn, :queue, :name, :poll_interval, :max_messages, + :retry_after, :subconsumer, :dead_interval + + attr_accessor :logger + + def initialize(dsn, queue:, name:, poll_interval: 30, + max_messages: DEFAULT_MAX_MESSAGES, retry_after: 60, + unknown_handler_policy: "nack", subconsumer: nil, + dead_interval: nil, logger: nil) + @dsn = dsn + @queue = queue + @name = name + @poll_interval = poll_interval + @max_messages = max_messages + @retry_after = retry_after + + unless ["nack", "ack"].include?(unknown_handler_policy.to_s) + raise ArgumentError, + "unknown_handler_policy must be 'nack' or 'ack', " \ + "got #{unknown_handler_policy.inspect}" + end + @unknown_handler_policy = unknown_handler_policy.to_s + + if dead_interval && subconsumer.nil? + raise ArgumentError, + "dead_interval is only valid in cooperative mode " \ + "(set subconsumer:)" + end + @subconsumer = subconsumer + @dead_interval = dead_interval + + @handlers = {} + @default_handler = nil + @running_mutex = Mutex.new + @running = false + @logger = logger || default_logger + end + + def on(event_type, &block) + raise ArgumentError, "block required for Consumer#on" unless block + + if event_type == "*" + @default_handler = block + else + @handlers[event_type] = block + end + block + end + + def start + set_running(true) + + in_main_thread = (Thread.current == Thread.main) + original_handlers = {} + + stop_proc = ->(signum) { + @logger.info("received signal #{signum}, shutting down") + set_running(false) + } + + if in_main_thread + ["TERM", "INT"].each do |sig| + original_handlers[sig] = Signal.trap(sig) { stop_proc.call(sig) } + end + end + + begin + conn = PG.connect(@dsn) + begin + channel = "pgque_#{@queue}" + conn.exec("LISTEN #{conn.escape_identifier(channel)}") + @logger.info( + "consumer #{@name} listening on #{@queue} (poll=#{@poll_interval}s)" + ) + + while running? + poll_once(conn) + break unless running? + wait_for_notify_or_stop(conn) + end + ensure + conn.close unless conn.finished? + end + ensure + if in_main_thread + original_handlers.each { |sig, h| Signal.trap(sig, h || "DEFAULT") } + end + @logger.info("consumer #{@name} stopped") + end + end + + def stop + set_running(false) + end + + def running? + @running_mutex.synchronize { @running } + end + + # Public for testability; not part of the stable API. + def poll_once(conn) + conn.transaction do + client = Client.new(conn) + msgs = + if @subconsumer + client.receive_coop( + @queue, @name, @subconsumer, + max_messages: @max_messages, + dead_interval: @dead_interval, + ) + else + client.receive(@queue, @name, @max_messages) + end + + next if msgs.empty? + + batch_id = msgs[0].batch_id + @logger.debug("batch #{batch_id}: #{msgs.size} message(s)") + + nack_failed = dispatch_batch(client, batch_id, msgs) + + next if nack_failed + + rowcount = client.ack(batch_id) + if rowcount == 0 + @logger.warn( + "pgque: ack batch #{batch_id} returned 0 -- stale or " \ + "double ack (batch already finished or not found)", + ) + end + end + end + + private + + def dispatch_batch(client, batch_id, msgs) + nack_failed = false + msgs.each do |msg| + handler = @handlers[msg.type] || @default_handler + + if handler.nil? + if @unknown_handler_policy == "ack" + @logger.warn( + "no handler for event type=#{msg.type} ev_id=#{msg.msg_id}; " \ + "acking", + ) + next + end + @logger.warn( + "no handler for event type=#{msg.type} ev_id=#{msg.msg_id}; " \ + "nacking", + ) + begin + client.nack(batch_id, msg, retry_after: @retry_after, + reason: "no handler for type=#{msg.type}") + rescue StandardError => e + nack_failed = true + @logger.error( + "nack failed for unhandled msg_id=#{msg.msg_id}: " \ + "#{e.class}: #{e.message}", + ) + end + next + end + + begin + handler.call(msg) + rescue StandardError => e + @logger.error( + "handler failed for msg_id=#{msg.msg_id}: " \ + "#{e.class}: #{e.message}", + ) + begin + client.nack(batch_id, msg, retry_after: @retry_after) + rescue StandardError => e2 + nack_failed = true + @logger.error( + "nack failed for msg_id=#{msg.msg_id}: " \ + "#{e2.class}: #{e2.message}", + ) + end + end + end + nack_failed + end + + def wait_for_notify_or_stop(conn) + drained = false + while conn.notifies + drained = true + end + return if drained + + deadline = monotonic + @poll_interval + while running? + remaining = deadline - monotonic + return if remaining <= 0 + + slice = [WAIT_SLICE_SECONDS, remaining].min + notification = conn.wait_for_notify(slice) + return unless running? + + if notification + while conn.notifies + # drain any queued notifications + end + return + end + end + end + + def set_running(value) + @running_mutex.synchronize { @running = value } + end + + def monotonic + Process.clock_gettime(Process::CLOCK_MONOTONIC) + end + + def default_logger + log = Logger.new($stdout) + log.progname = "pgque.consumer.#{@name}" + log.level = ENV["PGQUE_LOG_LEVEL"] ? Logger.const_get(ENV["PGQUE_LOG_LEVEL"].upcase) : Logger::WARN + log + end + end +end diff --git a/clients/ruby/test/test_consumer.rb b/clients/ruby/test/test_consumer.rb new file mode 100644 index 00000000..72858518 --- /dev/null +++ b/clients/ruby/test/test_consumer.rb @@ -0,0 +1,359 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" +require "logger" +require "stringio" + +class TestConsumerUnit < Minitest::Test + include PgqueTest::Helpers + + class FakeTxConn + def transaction + yield self + end + end + + class SpyClient + attr_reader :receive_calls + + def initialize(*) + @receive_calls = [] + end + + def receive(queue, consumer, max_messages) + @receive_calls << [queue, consumer, max_messages] + [] + end + + def ack(_batch_id) + 1 + end + + def nack(*); end + end + + def test_consumer_default_max_messages_requests_whole_batch + cons = Pgque::Consumer.new(dsn, queue: "q", name: "c") + assert_equal 2_147_483_647, cons.max_messages + end + + def test_consumer_configured_max_messages_is_preserved + cons = Pgque::Consumer.new(dsn, queue: "q", name: "c", max_messages: 123) + assert_equal 123, cons.max_messages + end + + def test_consumer_poll_once_passes_default_max_messages + cons = Pgque::Consumer.new(dsn, queue: "q", name: "c") + spy = SpyClient.new + Pgque::Client.stub :new, ->(*) { spy } do + cons.poll_once(FakeTxConn.new) + end + assert_equal [["q", "c", 2_147_483_647]], spy.receive_calls + end + + def test_consumer_poll_once_passes_configured_max_messages + cons = Pgque::Consumer.new(dsn, queue: "q", name: "c", max_messages: 123) + spy = SpyClient.new + Pgque::Client.stub :new, ->(*) { spy } do + cons.poll_once(FakeTxConn.new) + end + assert_equal [["q", "c", 123]], spy.receive_calls + end + + def test_consumer_rejects_invalid_unknown_handler_policy + skip_dsn_for_this_class! + assert_raises(ArgumentError) do + Pgque::Consumer.new("dummy", queue: "q", name: "c", + unknown_handler_policy: "bogus") + end + end + + def test_consumer_dead_interval_without_subconsumer_raises + skip_dsn_for_this_class! + assert_raises(ArgumentError) do + Pgque::Consumer.new("dummy", queue: "q", name: "c", + dead_interval: "5 minutes") + end + end + + private + + # Some unit tests don't actually connect; allow them even without DSN. + def skip_dsn_for_this_class! + # The setup-level skip already passes when DSN is set; this method is + # here so the structure is symmetric with the integration tests. + end +end + +class TestConsumerIntegration < Minitest::Test + include PgqueTest::Helpers + + def run_consumer_for(consumer, seconds) + t = Thread.new { consumer.start } + Thread.new do + sleep seconds + consumer.stop + end + t + end + + def force_tick(conn, queue) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + end + + def silent_logger + log = Logger.new(StringIO.new) + log.level = Logger::FATAL + log + end + + def capturing_logger + io = StringIO.new + log = Logger.new(io) + log.level = Logger::WARN + [log, io] + end + + def retry_count_for_msg(conn, queue, msg_id) + conn.exec_params( + "select count(*) from pgque.retry_queue rq " \ + "join pgque.queue q on q.queue_id = rq.ev_queue " \ + "where q.queue_name = $1 and rq.ev_id = $2", + [queue, msg_id], + ).values[0][0].to_i + end + + def dlq_count_for_msg(conn, queue, msg_id) + conn.exec_params( + "select count(*) from pgque.dead_letter dl " \ + "join pgque.queue q on q.queue_id = dl.dl_queue_id " \ + "where q.queue_name = $1 and dl.ev_id = $2", + [queue, msg_id], + ).values[0][0].to_i + end + + def test_consumer_dispatches_by_event_type + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "i" => 1 }, type: "evt.a") + client.send(queue, { "i" => 2 }, type: "evt.b") + force_tick(conn, queue) + + seen_a = [] + seen_b = [] + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, logger: silent_logger) + cons.on("evt.a") { |m| seen_a << m.payload } + cons.on("evt.b") { |m| seen_b << m.payload } + + run_consumer_for(cons, 3.0).join(5.0) + + assert_equal 1, seen_a.size + assert_equal 1, seen_b.size + end + end + + def test_consumer_default_handler_catches_unknown + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "x" => 99 }, type: "never.registered.type") + force_tick(conn, queue) + + fallback = [] + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, logger: silent_logger) + cons.on("*") { |m| fallback << m } + + run_consumer_for(cons, 3.0).join(5.0) + + assert_equal 1, fallback.size + assert_equal "never.registered.type", fallback[0].type + end + end + + def test_consumer_nacks_on_handler_error + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "i" => 1 }, type: "evt.fail") + force_tick(conn, queue) + + n_calls = 0 + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, retry_after: 0, + logger: silent_logger) + cons.on("evt.fail") { |_m| n_calls += 1; raise "simulated failure" } + + run_consumer_for(cons, 3.0).join(5.0) + + assert_operator n_calls, :>=, 1 + cnt = conn.exec_params( + "select count(*) from pgque.retry_queue rq " \ + "join pgque.queue q on q.queue_id = rq.ev_queue " \ + "where q.queue_name = $1", + [queue], + ).values[0][0].to_i + assert_operator cnt, :>=, 1 + end + end + + def test_consumer_nacks_unhandled_event_type + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + msg_id = client.send(queue, { "x" => 1 }, type: "totally.unregistered.type") + force_tick(conn, queue) + + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, logger: silent_logger) + run_consumer_for(cons, 3.0).join(5.0) + + rq = retry_count_for_msg(conn, queue, msg_id) + dlq = dlq_count_for_msg(conn, queue, msg_id) + assert_operator rq + dlq, :>=, 1, + "unhandled event was not nacked: rq=#{rq} dlq=#{dlq}" + + force_tick(conn, queue) + follow = client.receive(queue, consumer_n, 10) + refute(follow.any? { |m| m.msg_id == msg_id }, + "batch did not advance past unhandled msg_id") + client.ack(follow[0].batch_id) if follow.any? + end + end + + def test_consumer_acks_unhandled_event_type_when_opt_in + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + msg_id = client.send(queue, { "x" => 1 }, type: "totally.unregistered.type") + force_tick(conn, queue) + + log, io = capturing_logger + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, + unknown_handler_policy: "ack", + logger: log) + run_consumer_for(cons, 3.0).join(5.0) + + assert_equal 0, retry_count_for_msg(conn, queue, msg_id) + assert_equal 0, dlq_count_for_msg(conn, queue, msg_id) + assert_includes io.string, "totally.unregistered.type" + + force_tick(conn, queue) + follow = client.receive(queue, consumer_n, 10) + refute(follow.any? { |m| m.msg_id == msg_id }) + client.ack(follow[0].batch_id) if follow.any? + end + end + + def test_consumer_stop_returns_promptly + with_queue do |queue, consumer_n, _conn| + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 10, logger: silent_logger) + t = Thread.new { cons.start } + sleep 0.5 + cons.stop + finished = t.join(15) + refute_nil finished, "consumer did not stop after stop()" + end + end + + def test_consumer_stop_returns_within_2s_while_waiting + with_queue do |queue, consumer_n, _conn| + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 30, logger: silent_logger) + t = Thread.new { cons.start } + sleep 1.0 + + t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) + cons.stop + t.join(5.0) + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0 + + refute t.alive?, "consumer thread did not stop" + assert_operator elapsed, :<, 2.0, + "stop() took #{elapsed.round(2)}s; expected <2s" + end + end + + def test_consumer_wakes_on_pg_notify_before_poll_interval + with_queue do |queue, consumer_n, conn| + received = [] + received_evt = Mutex.new + received_cv = ConditionVariable.new + + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 30, logger: silent_logger) + cons.on("evt.wake") do |m| + received_evt.synchronize do + received << m + received_cv.signal + end + end + + t = Thread.new { cons.start } + sleep 1.5 + + t_send = Process.clock_gettime(Process::CLOCK_MONOTONIC) + producer = PG.connect(dsn) + begin + client = Pgque::Client.new(producer) + client.send(queue, { "i" => 1 }, type: "evt.wake") + producer.exec_params("select pgque.force_next_tick($1)", [queue]) + producer.exec_params("select pgque.ticker($1)", [queue]) + producer.exec_params("notify pgque_#{queue}, 'go'") + ensure + producer.close + end + + woke = false + received_evt.synchronize do + deadline = Time.now + 5 + until received.any? || Time.now >= deadline + received_cv.wait(received_evt, deadline - Time.now) + end + woke = received.any? + end + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - t_send + + cons.stop + t.join(5.0) + + assert woke, "consumer did not wake on pg_notify within 5s" + assert_equal 1, received.size + assert_operator elapsed, :<, 5.0, + "consumer woke too slowly (#{elapsed.round(2)}s)" + end + end + + def test_consumer_partial_batch_acks_good_messages_only + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + ok1 = client.send(queue, { "i" => 1 }, type: "ok") + boom = client.send(queue, { "i" => 2 }, type: "boom") + ok2 = client.send(queue, { "i" => 3 }, type: "ok") + force_tick(conn, queue) + + seen_ok = [] + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, retry_after: 3600, + logger: silent_logger) + cons.on("ok") { |m| seen_ok << m.msg_id } + cons.on("boom") { |_| raise "handler boom" } + + run_consumer_for(cons, 3.0).join(5.0) + + assert_includes seen_ok, ok1 + assert_includes seen_ok, ok2 + + assert_operator retry_count_for_msg(conn, queue, boom), :>=, 1 + assert_equal 0, retry_count_for_msg(conn, queue, ok1) + assert_equal 0, retry_count_for_msg(conn, queue, ok2) + + force_tick(conn, queue) + follow = client.receive(queue, consumer_n, 10) + ids = follow.map(&:msg_id) + refute_includes ids, ok1 + refute_includes ids, ok2 + client.ack(follow[0].batch_id) if follow.any? + end + end +end diff --git a/clients/ruby/test/test_consumer_listen_stop.rb b/clients/ruby/test/test_consumer_listen_stop.rb new file mode 100644 index 00000000..85d71671 --- /dev/null +++ b/clients/ruby/test/test_consumer_listen_stop.rb @@ -0,0 +1,84 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +# Regression tests for the LISTEN/NOTIFY wait: stop() must take effect +# promptly, and a real NOTIFY must wake the wait well before +# poll_interval expires. + +require_relative "test_helper" +require "logger" +require "stringio" + +class TestConsumerListenStop < Minitest::Test + include PgqueTest::Helpers + + def silent_logger + log = Logger.new(StringIO.new) + log.level = Logger::FATAL + log + end + + def test_stop_is_honored_promptly + with_queue do |queue, consumer_n, _conn| + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 10, logger: silent_logger) + t = Thread.new { cons.start } + sleep 1.0 + + t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) + cons.stop + t.join(3.5) + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0 + + refute t.alive?, + "consumer thread still alive #{elapsed.round(2)}s after stop()" + assert_operator elapsed, :<, 3.0, + "stop() took #{elapsed.round(2)}s; expected <3s" + end + end + + def test_notify_wakes_consumer_before_poll_interval + with_queue do |queue, consumer_n, _conn| + seen = [] + handler_called = Mutex.new + handler_cv = ConditionVariable.new + + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 10, logger: silent_logger) + cons.on("evt.wake") do |m| + handler_called.synchronize do + seen << m.payload + handler_cv.signal + end + end + + t = Thread.new { cons.start } + begin + sleep 1.0 + + producer = PG.connect(dsn) + begin + client = Pgque::Client.new(producer) + client.send(queue, { "v" => 1 }, type: "evt.wake") + producer.exec_params("select pgque.force_next_tick($1)", [queue]) + producer.exec_params("select pgque.ticker($1)", [queue]) + ensure + producer.close + end + + woke = false + handler_called.synchronize do + deadline = Time.now + 3.0 + until seen.any? || Time.now >= deadline + handler_cv.wait(handler_called, deadline - Time.now) + end + woke = seen.any? + end + assert woke, "handler not invoked within 3s; NOTIFY did not wake" + assert_equal 1, seen.size + ensure + cons.stop + t.join(3.0) + end + end + end +end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 640794c1..2864fa6a 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -103,8 +103,13 @@ def test_jsonb_payload_round_trip conn.exec_params("select pgque.ticker($1)", [queue]) msgs = client.receive(queue, consumer, 10) assert_equal 1, msgs.size, "no message for payload=#{payload.inspect}" - assert_equal expected, msgs[0].payload, - "payload=#{payload.inspect} did not round-trip" + if expected.nil? + assert_nil msgs[0].payload, + "payload=#{payload.inspect} did not round-trip to nil" + else + assert_equal expected, msgs[0].payload, + "payload=#{payload.inspect} did not round-trip" + end client.ack(msgs[0].batch_id) end end From e38e9c2a1c728702bfb72ef24189dfac451b6d2b Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:48:42 +0400 Subject: [PATCH 15/40] feat(ruby): cooperative consumers (experimental) Adds the experimental cooperative-consumers SQL surface to the client: - subscribe_subconsumer / unsubscribe_subconsumer (with batch_handling: kwarg defaulting to 0, the strict mode that raises on an active batch). - receive_coop with max_messages: + dead_interval: kwargs. - touch_subconsumer. The high-level Pgque::Consumer already routes to receive_coop when subconsumer: is set; this commit makes the underlying SQL calls real. Tests: 9 ports of test_coop.py covering subscribe idempotency, two- member batch splitting (no overlap), unsubscribe-with-active-batch strict raise + batch_handling:1 cleanup, touch heartbeat, the high- level Consumer in coop mode, and the dead_interval-without-subconsumer ArgumentError. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/client.rb | 43 ++++++ clients/ruby/test/test_coop.rb | 237 +++++++++++++++++++++++++++++++ 2 files changed, 280 insertions(+) create mode 100644 clients/ruby/test/test_coop.rb diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 02ca1dd9..986d6436 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -89,6 +89,49 @@ def force_next_tick(queue) raise wrap_sql_error(e) end + # Experimental: function names, edge-case behavior, and signatures may + # change before the cooperative API is marked stable. + def subscribe_subconsumer(queue, consumer, subconsumer) + result = @conn.exec_params( + "select pgque.subscribe_subconsumer($1, $2, $3)", + [queue, consumer, subconsumer], + ) + result.values[0][0].to_i + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def unsubscribe_subconsumer(queue, consumer, subconsumer, batch_handling: 0) + result = @conn.exec_params( + "select pgque.unsubscribe_subconsumer($1, $2, $3, $4)", + [queue, consumer, subconsumer, batch_handling], + ) + result.values[0][0].to_i + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def receive_coop(queue, consumer, subconsumer, max_messages: 100, + dead_interval: nil) + result = @conn.exec_params( + "select * from pgque.receive_coop($1, $2, $3, $4, $5::interval)", + [queue, consumer, subconsumer, max_messages, dead_interval], + ) + result.values.map { |row| row_to_message(row) } + rescue PG::Error => e + raise wrap_sql_error(e) + end + + def touch_subconsumer(queue, consumer, subconsumer) + result = @conn.exec_params( + "select pgque.touch_subconsumer($1, $2, $3)", + [queue, consumer, subconsumer], + ) + result.values[0][0].to_i + rescue PG::Error => e + raise wrap_sql_error(e) + end + def nack(batch_id, msg, retry_after: 60, reason: nil) payload_str = case msg.payload when Hash, Array then JSON.dump(msg.payload) diff --git a/clients/ruby/test/test_coop.rb b/clients/ruby/test/test_coop.rb new file mode 100644 index 00000000..649b324c --- /dev/null +++ b/clients/ruby/test/test_coop.rb @@ -0,0 +1,237 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +# Experimental cooperative-consumers API. Function names, edge-case +# behavior, and client API shape may change before this feature is +# marked stable. + +require_relative "test_helper" +require "logger" +require "stringio" + +module CoopHelpers + def with_coop_queue + conn = PG.connect(PGQUE_TEST_DSN) + q = unique_queue_name + conn.exec_params("select pgque.create_queue($1)", [q]) + yield q, conn + ensure + if conn && !conn.finished? + begin + rows = conn.exec_params( + "select c.co_name from pgque.consumer c " \ + "join pgque.subscription s on s.sub_consumer = c.co_id " \ + "join pgque.queue qq on qq.queue_id = s.sub_queue " \ + "where qq.queue_name = $1 and s.sub_role = 'coop_member'", + [q], + ).values.map { |r| r[0] } + rows.each do |co_name| + parent, sep, sub = co_name.rpartition(".") + next if sep.empty? || parent.empty? || sub.empty? + conn.exec_params( + "select pgque.unsubscribe_subconsumer($1, $2, $3, 1)", + [q, parent, sub], + ) + end + conn.exec_params("select pgque.drop_queue($1, true)", [q]) + rescue PG::Error + # best-effort cleanup + end + conn.close + end + end + + def tick(conn, queue) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + end + + def silent_logger + log = Logger.new(StringIO.new) + log.level = Logger::FATAL + log + end +end + +class TestCoop < Minitest::Test + include PgqueTest::Helpers + include CoopHelpers + + def consumer_n + @consumer_n ||= unique_consumer_name + end + + def test_subscribe_subconsumer_returns_1_then_0 + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + first = client.subscribe_subconsumer(q, consumer_n, "worker-1") + second = client.subscribe_subconsumer(q, consumer_n, "worker-1") + assert_equal 1, first + assert_equal 0, second + end + end + + def test_receive_coop_returns_messages_and_ack_finishes + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + client.subscribe_subconsumer(q, consumer_n, "worker-1") + client.send(q, { "k" => 1 }, type: "evt.a") + client.send(q, { "k" => 2 }, type: "evt.a") + tick(conn, q) + + msgs = client.receive_coop(q, consumer_n, "worker-1", max_messages: 10) + assert_equal 2, msgs.size + ks = msgs.map { |m| m.payload["k"] }.sort + assert_equal [1, 2], ks + + client.ack(msgs[0].batch_id) + + follow = client.receive_coop(q, consumer_n, "worker-1", max_messages: 10) + assert_equal [], follow + end + end + + def test_two_subconsumers_split_batches_no_duplicates + with_coop_queue do |q, conn| + Pgque.connect(dsn) do |producer| + producer.subscribe_subconsumer(q, consumer_n, "worker-1") + producer.subscribe_subconsumer(q, consumer_n, "worker-2") + 6.times { |i| producer.send(q, { "i" => i }, type: "evt") } + producer.conn.exec_params("select pgque.force_next_tick($1)", [q]) + producer.conn.exec_params("select pgque.ticker($1)", [q]) + end + + Pgque.connect(dsn, autocommit: true) do |c1| + Pgque.connect(dsn, autocommit: true) do |c2| + m1 = c1.receive_coop(q, consumer_n, "worker-1", max_messages: 100) + m2 = c2.receive_coop(q, consumer_n, "worker-2", max_messages: 100) + + ids1 = m1.map(&:msg_id) + ids2 = m2.map(&:msg_id) + assert_empty ids1 & ids2, + "member-1 and member-2 saw same msg_ids: #{ids1 & ids2}" + assert_operator m1.size + m2.size, :>=, 1 + + c1.ack(m1[0].batch_id) if m1.any? + c2.ack(m2[0].batch_id) if m2.any? + end + end + + Pgque.connect(dsn) do |cleanup| + cleanup.unsubscribe_subconsumer(q, consumer_n, "worker-1", + batch_handling: 1) + cleanup.unsubscribe_subconsumer(q, consumer_n, "worker-2", + batch_handling: 1) + end + end + end + + def test_unsubscribe_subconsumer_with_active_batch_default_raises + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + client.subscribe_subconsumer(q, consumer_n, "worker-1") + client.send(q, { "i" => 1 }, type: "evt") + tick(conn, q) + + msgs = client.receive_coop(q, consumer_n, "worker-1") + assert_equal 1, msgs.size + + assert_raises(Pgque::Error) do + client.unsubscribe_subconsumer(q, consumer_n, "worker-1") + end + conn.exec("rollback") rescue nil + + rv = client.unsubscribe_subconsumer(q, consumer_n, "worker-1", + batch_handling: 1) + assert_equal 1, rv + end + end + + def test_unsubscribe_subconsumer_routes_active_messages_through_retry + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + client.subscribe_subconsumer(q, consumer_n, "worker-1") + client.send(q, { "i" => 1 }, type: "evt") + tick(conn, q) + + msgs = client.receive_coop(q, consumer_n, "worker-1") + assert_equal 1, msgs.size + + rv = client.unsubscribe_subconsumer(q, consumer_n, "worker-1", + batch_handling: 1) + assert_equal 1, rv + end + end + + def test_touch_subconsumer_returns_1_on_registered_row + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + client.subscribe_subconsumer(q, consumer_n, "worker-1") + rv = client.touch_subconsumer(q, consumer_n, "worker-1") + assert_equal 1, rv + end + end + + def test_consumer_coop_dispatches_and_acks + with_coop_queue do |q, conn| + client = Pgque::Client.new(conn) + client.subscribe_subconsumer(q, consumer_n, "worker-1") + msg_id = client.send(q, { "x" => 1 }, type: "evt.coop") + tick(conn, q) + + seen = [] + cons = Pgque::Consumer.new(dsn, + queue: q, name: consumer_n, + subconsumer: "worker-1", + poll_interval: 1, + logger: silent_logger) + cons.on("evt.coop") { |m| seen << m } + + t = Thread.new { cons.start } + Thread.new do + sleep 3.0 + cons.stop + end + t.join(5.0) + + assert_equal 1, seen.size + assert_equal msg_id, seen[0].msg_id + + follow = client.receive_coop(q, consumer_n, "worker-1") + assert_equal [], follow + + client.unsubscribe_subconsumer(q, consumer_n, "worker-1", + batch_handling: 1) + end + end + + def test_consumer_without_subconsumer_unchanged + with_queue do |queue, c_name, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "v" => 1 }, type: "evt.normal") + tick(conn, queue) + + seen = [] + cons = Pgque::Consumer.new(dsn, + queue: queue, name: c_name, + poll_interval: 1, + logger: silent_logger) + cons.on("evt.normal") { |m| seen << m } + + t = Thread.new { cons.start } + Thread.new do + sleep 3.0 + cons.stop + end + t.join(5.0) + + assert_equal 1, seen.size + end + end + + def test_consumer_dead_interval_without_subconsumer_raises + assert_raises(ArgumentError) do + Pgque::Consumer.new(dsn, queue: "q", name: "c", + dead_interval: "5 minutes") + end + end +end From e8a82c1c9e79ef0d7db2e8e10b6bdbee4e5c60be Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:48:48 +0400 Subject: [PATCH 16/40] test(ruby): concurrency + PgQ snapshot visibility regressions - test_concurrency.rb: 4 producer threads x 25 sends each, asserting zero event-id collisions across independent connections. - test_transaction_visibility.rb: locks PgQ's snapshot rule (send + force_next_tick + receive in one xact returns 0 rows) and a regression guard that catches a Consumer whose poll_once is a no-op -- a fresh receive must still return the message because the batch cursor never advanced. The visibility test stubs Consumer#poll_once via define_singleton_method on the instance, mirroring the Python mock.patch.object pattern. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_concurrency.rb | 36 ++++++++++ .../ruby/test/test_transaction_visibility.rb | 72 +++++++++++++++++++ 2 files changed, 108 insertions(+) create mode 100644 clients/ruby/test/test_concurrency.rb create mode 100644 clients/ruby/test/test_transaction_visibility.rb diff --git a/clients/ruby/test/test_concurrency.rb b/clients/ruby/test/test_concurrency.rb new file mode 100644 index 00000000..8d355073 --- /dev/null +++ b/clients/ruby/test/test_concurrency.rb @@ -0,0 +1,36 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestConcurrency < Minitest::Test + include PgqueTest::Helpers + + def test_concurrent_producers_no_id_collisions + with_queue do |queue, _consumer, _conn| + n_threads = 4 + per_thread = 25 + seen_ids = [] + seen_lock = Mutex.new + + threads = n_threads.times.map do + Thread.new do + ids = [] + Pgque.connect(dsn) do |client| + per_thread.times do |i| + ids << client.send( + queue, + { "thread" => Thread.current.object_id, "i" => i }, + ) + end + end + seen_lock.synchronize { seen_ids.concat(ids) } + end + end + threads.each { |t| refute_nil t.join(30), "producer thread hung" } + + assert_equal n_threads * per_thread, seen_ids.size + assert_equal seen_ids.size, seen_ids.uniq.size, + "duplicate event ids: #{seen_ids - seen_ids.uniq}" + end + end +end diff --git a/clients/ruby/test/test_transaction_visibility.rb b/clients/ruby/test/test_transaction_visibility.rb new file mode 100644 index 00000000..cf09ae0d --- /dev/null +++ b/clients/ruby/test/test_transaction_visibility.rb @@ -0,0 +1,72 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +# PgQ snapshot isolation: events committed in transaction T are only +# visible to a batch whose tick was taken after T committed. Collapsing +# send + force_next_tick + receive into one transaction violates that +# contract. These tests document and enforce it. + +require_relative "test_helper" +require "logger" +require "stringio" + +class TestTransactionVisibility < Minitest::Test + include PgqueTest::Helpers + + def silent_logger + log = Logger.new(StringIO.new) + log.level = Logger::FATAL + log + end + + def test_collapsed_transaction_returns_no_messages + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + + conn.exec("BEGIN") + begin + client.send(queue, { "x" => 1 }, type: "collapsed.test") + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + # No commit between send and receive -- one transaction. + msgs = client.receive(queue, consumer, 10) + assert_equal 0, msgs.size, + "PgQ visibility violation: collapsed transaction " \ + "returned #{msgs.size} message(s); expected 0. " \ + "Add a commit between send and force_next_tick." + ensure + conn.exec("ROLLBACK") + end + end + end + + def test_unhandled_event_nack_assertion_catches_stale_cursor + with_queue do |queue, consumer_n, conn| + client = Pgque::Client.new(conn) + msg_id = client.send(queue, { "x" => 1 }, type: "totally.unregistered.type") + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + + cons = Pgque::Consumer.new(dsn, queue: queue, name: consumer_n, + poll_interval: 1, logger: silent_logger) + # Simulate a broken consumer that receives but neither acks nor nacks. + cons.define_singleton_method(:poll_once) { |_c| } + + t = Thread.new { cons.start } + sleep 2.0 + cons.stop + t.join(4.0) + + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + follow = client.receive(queue, consumer_n, 10) + + assert(follow.any? { |m| m.msg_id == msg_id }, + "expected the unprocessed message to still be visible " \ + "(cursor did not advance because poll_once was a no-op), but " \ + "re-receive returned no rows. This indicates the batch cursor " \ + "advanced without an explicit ack -- a PgQ visibility violation.") + + client.ack(follow[0].batch_id) if follow.any? + end + end +end From d0a0a847795cda7dac7eec6c93fb8041b76e8b92 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:49:13 +0400 Subject: [PATCH 17/40] ci(ruby): add ruby-client-tests job Mirrors python-client-tests: postgres:18 in Docker, build pgque.sql via transform.sh, install into pgque_test DB, then run the Minitest suite (clients/ruby with Ruby 3.3 via ruby/setup-ruby) and a gem build smoke step. Cleanup tears down the container with if: always(). Also: update root .gitignore to exclude clients/ruby/Gemfile.lock, pkg/, .bundle/, and built *.gem files (libraries do not commit Gemfile.lock). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ci.yml | 56 ++++++++++++++++++++++++++++++++++++++++ .gitignore | 4 +++ 2 files changed, 60 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 61e8ca4b..fddbd877 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -491,6 +491,62 @@ jobs: if: always() run: docker rm -f pgque-ts-test + ruby-client-tests: + name: Ruby client tests + runs-on: ubuntu-latest + env: + PGQUE_TEST_DSN: postgresql://postgres:pgque_test@localhost/pgque_test + steps: + - uses: actions/checkout@v4 + with: + submodules: recursive + + - name: Start PostgreSQL 18 + run: | + set -Eeuo pipefail + docker run -d --name pgque-ruby-test \ + -e POSTGRES_PASSWORD=pgque_test \ + -e POSTGRES_DB=pgque_test \ + -p 5432:5432 \ + postgres:18 + + for i in $(seq 1 30); do + docker exec pgque-ruby-test pg_isready -U postgres && break + sleep 1 + done || { echo "PG not ready after 30 seconds"; exit 1; } + + - name: Build pgque + run: bash build/transform.sh + + - name: Install pgque + run: | + set -Eeuo pipefail + PGPASSWORD=pgque_test psql -h localhost -U postgres -d pgque_test \ + -v ON_ERROR_STOP=1 -f sql/pgque.sql + + - name: Setup Ruby + uses: ruby/setup-ruby@v1 + with: + ruby-version: "3.3" + working-directory: clients/ruby + bundler-cache: true + + - name: Ruby client test suite + run: | + set -Eeuo pipefail + cd clients/ruby + bundle exec rake test + + - name: Ruby package smoke + run: | + set -Eeuo pipefail + cd clients/ruby + gem build pgque.gemspec + + - name: Cleanup Ruby test DB + if: always() + run: docker rm -f pgque-ruby-test + verify: runs-on: ubuntu-latest steps: diff --git a/.gitignore b/.gitignore index 4d5ed849..536e1c54 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,10 @@ build/output/ clients/python/*.egg-info/ clients/typescript/node_modules/ clients/typescript/package-lock.json +clients/ruby/Gemfile.lock +clients/ruby/pkg/ +clients/ruby/.bundle/ +clients/ruby/*.gem # Claude Code agent worktrees (ephemeral isolation per agent run) .claude/worktrees/ From 4de8508502d17a74aa8b58530eff2775ab260a6a Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 10:50:01 +0400 Subject: [PATCH 18/40] docs: add Ruby column to client parity matrix + README section - clients/README.md: Ruby column with the v1 surface (full parity with Python: classified errors, retry delay + reason on nack, LISTEN/NOTIFY wakeup on the high-level Consumer, configurable unknown-type policy). - README.md: Ruby section under "Client libraries" with the install command, a connect example, and a Consumer example matching the Python/TS snippets above. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 26 ++++++++++++++++++++++++- clients/README.md | 48 +++++++++++++++++++++++------------------------ 2 files changed, 49 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index f907566e..47716a49 100644 --- a/README.md +++ b/README.md @@ -333,7 +333,7 @@ Longer walkthrough in the [tutorial](docs/tutorial.md); patterns like fan-out, e ## Client libraries -PgQue is SQL-first, so any Postgres driver works. First-party client libraries live in this repo for **Python**, **Go**, and **TypeScript**, all published at `v0.2.0-rc.1`. +PgQue is SQL-first, so any Postgres driver works. First-party client libraries live in this repo for **Python**, **Go**, **TypeScript**, and **Ruby**, all published at `v0.2.0-rc.1`. ### Python (`pgque-py`) — psycopg 3 @@ -407,6 +407,30 @@ try { } ``` +### Ruby (`pgque`) — pg gem + +```bash +gem install pgque --pre # or pin: gem "pgque", "0.2.0.rc.1" +``` + +```ruby +require "pgque" + +Pgque.connect("postgresql://localhost/mydb") do |client| + client.send("orders", { "order_id" => 42 }, type: "order.created") +end + +consumer = Pgque::Consumer.new( + "postgresql://localhost/mydb", + queue: "orders", + name: "processor", +) + +consumer.on("order.created") { |msg| process_order(msg.payload) } + +consumer.start # blocks until SIGTERM/SIGINT +``` + ### Any language ```sql diff --git a/clients/README.md b/clients/README.md index ef38609a..f9374498 100644 --- a/clients/README.md +++ b/clients/README.md @@ -1,38 +1,38 @@ # PgQue clients -PgQue ships three first-party clients. They are thin wrappers over `pgque.*` +PgQue ships four first-party clients. They are thin wrappers over `pgque.*` SQL primitives. The matrix below tracks the public client API on current `main`. ## Current parity matrix -| Capability | Python | Go | TypeScript | -| --- | :---: | :---: | :---: | -| `connect` / `close` | ✓ | ✓ | ✓ | -| Raw SQL escape hatch | ✓ (`conn`) | ✓ (`Pool()`) | ✓ (`rawPool`) | -| PgQue-classified errors | ✓ | ✗ | ✓ | -| Lossless PostgreSQL `bigint` IDs | ✓ (`int`) | ✓ (`int64`) | ✓ (`bigint`) | -| `send` | ✓ | ✓ | ✓ | -| `send_batch` / `SendBatch` / `sendBatch` | ✓ | ✓ | ✓ | -| `receive` | ✓ | ✓ | ✓ | -| `ack` returns SQL rowcount (0 stale, 1 success) | ✓ (int) | ✓ (int64) | ✓ (number) | -| `nack` | ✓ | ✓ | ✓ | -| `force_next_tick` / `ForceNextTick` / `forceNextTick` | ✓ | ✓ | ✓ | -| `nack` retry delay + reason options | ✓ | ✗ | ✓ | -| High-level `Consumer` | ✓ | ✓ | ✓ | -| Consumer wakeup model | polling + optional LISTEN/NOTIFY wakeup | polling | polling | -| `Consumer` poll interval option | ✓ | ✓ | ✓ | -| `Consumer` max-messages option | ✓ | ✗ | ✓ | -| `Consumer` retry delay option | ✓ | ✗ | ✗ | -| Unknown-type behavior avoids silent ack | ✗ | ✓ | ✓ | -| Configurable unknown-type policy | ✗ | ✗ | ✗ | -| `subscribe` / `unsubscribe` wrappers | ✗ | ✗ | ✓ | -| Cooperative consumers (experimental) [^coop] | ✓ | ✓ | ✓ | +| Capability | Python | Go | TypeScript | Ruby | +| --- | :---: | :---: | :---: | :---: | +| `connect` / `close` | ✓ | ✓ | ✓ | ✓ | +| Raw SQL escape hatch | ✓ (`conn`) | ✓ (`Pool()`) | ✓ (`rawPool`) | ✓ (`conn`) | +| PgQue-classified errors | ✓ | ✗ | ✓ | ✓ | +| Lossless PostgreSQL `bigint` IDs | ✓ (`int`) | ✓ (`int64`) | ✓ (`bigint`) | ✓ (`Integer`) | +| `send` | ✓ | ✓ | ✓ | ✓ | +| `send_batch` / `SendBatch` / `sendBatch` | ✓ | ✓ | ✓ | ✓ | +| `receive` | ✓ | ✓ | ✓ | ✓ | +| `ack` returns SQL rowcount (0 stale, 1 success) | ✓ (int) | ✓ (int64) | ✓ (number) | ✓ (Integer) | +| `nack` | ✓ | ✓ | ✓ | ✓ | +| `force_next_tick` / `ForceNextTick` / `forceNextTick` | ✓ | ✓ | ✓ | ✓ | +| `nack` retry delay + reason options | ✓ | ✗ | ✓ | ✓ | +| High-level `Consumer` | ✓ | ✓ | ✓ | ✓ | +| Consumer wakeup model | polling + optional LISTEN/NOTIFY wakeup | polling | polling | polling + LISTEN/NOTIFY wakeup | +| `Consumer` poll interval option | ✓ | ✓ | ✓ | ✓ | +| `Consumer` max-messages option | ✓ | ✗ | ✓ | ✓ | +| `Consumer` retry delay option | ✓ | ✗ | ✗ | ✓ | +| Unknown-type behavior avoids silent ack | ✗ | ✓ | ✓ | ✓ | +| Configurable unknown-type policy | ✗ | ✗ | ✗ | ✓ | +| `subscribe` / `unsubscribe` wrappers | ✗ | ✗ | ✓ | ✗ | +| Cooperative consumers (experimental) [^coop] | ✓ | ✓ | ✓ | ✓ | Legend: ✓ supported by the client API on `main`; ✗ not exposed as a first-class client API. Lower-level SQL primitives remain available through raw connection/pool escape hatches. TypeScript currently exposes an extra -convenience wrapper for `ticker`; Python and Go can call it via raw SQL. +convenience wrapper for `ticker`; Python, Go, and Ruby can call it via raw SQL. [^coop]: Experimental. Each supporting client exposes `subscribe_subconsumer` / `unsubscribe_subconsumer` / `receive_coop` / From 775a853d4b07057c58849923f66f0d184ed7f90f Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:02:44 +0400 Subject: [PATCH 19/40] docs(ruby): document __send__ workaround for Client#send Pgque::Client#send shadows Object#send. Add a class doc-comment in lib/pgque/client.rb, a "A note on Pgque::Client#send" section in the gem README, and a regression test that __send__ and public_send still dispatch methods reflectively on a Client instance. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/README.md | 20 ++++++++++++++++++++ clients/ruby/lib/pgque/client.rb | 6 ++++++ clients/ruby/test/test_connect.rb | 12 ++++++++++++ 3 files changed, 38 insertions(+) diff --git a/clients/ruby/README.md b/clients/ruby/README.md index 7b60d8f4..deca62ee 100644 --- a/clients/ruby/README.md +++ b/clients/ruby/README.md @@ -35,6 +35,26 @@ grant pgque_writer to your_app_user; See [`docs/reference.md` — Roles and grants](../../docs/reference.md#roles-and-grants). +## A note on `Pgque::Client#send` + +The producer method is called `send` to mirror the SQL surface +(`pgque.send(queue, payload)`) and the Python/TS clients. That name +shadows Ruby's `Object#send`, which is widely used for reflective +method invocation. This means `client.send(:close)` calls the SQL +`send`, **not** the `close` method. + +Two well-known Ruby escape hatches restore reflective dispatch on a +`Pgque::Client` instance: + +```ruby +client.__send__(:close) # canonical "always works" form +client.public_send(:close) # safer: respects visibility +``` + +Use `__send__` or `public_send` whenever you need to call a method on +a `Pgque::Client` by name. The Pgque API itself never calls these +internally. + ## Tests Integration tests require a running PostgreSQL with the PgQue schema diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 986d6436..39eb72ea 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -3,6 +3,12 @@ # Marko Kreen / Skype Technologies OU). module Pgque + # Thin wrapper over the pgque SQL functions. + # + # Note: Pgque::Client#send mirrors the SQL `pgque.send(queue, payload)` + # primitive and the Python/TS client surface. That name shadows + # Ruby's Object#send, so use #__send__ or #public_send when you need + # to invoke a method on a Pgque::Client instance reflectively. class Client attr_reader :conn diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 518f585a..417316db 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -47,6 +47,18 @@ def test_close_is_idempotent client.close client.close end + + def test_underscore_send_dispatches_methods_reflectively + # Pgque::Client#send shadows Object#send; __send__ and public_send + # remain the way to invoke methods reflectively on a client. + client = Pgque.connect(dsn) + client.__send__(:close) + assert client.conn.finished? + + client2 = Pgque.connect(dsn) + client2.public_send(:close) + assert client2.conn.finished? + end end class TestConnectBadDsn < Minitest::Test From dbafc30ab57b9cdf6e3eb77ec5179cd4d2e3ed2c Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:04:33 +0400 Subject: [PATCH 20/40] fix(ruby): silence default Consumer logger by default Pgque::Consumer's default logger now targets $stderr (not $stdout, so it cannot collide with application output) and ships at level FATAL, which the consumer never emits -- making it effectively silent unless the host app opts in. Set PGQUE_LOG_LEVEL=warn|info|debug|error to re-enable, or pass logger: Logger.new(...) to Consumer.new. Mirrors Python's logging.getLogger("pgque") behavior more closely: no incidental output until the host application configures handlers. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/consumer.rb | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb index 22e3e053..a77361bb 100644 --- a/clients/ruby/lib/pgque/consumer.rb +++ b/clients/ruby/lib/pgque/consumer.rb @@ -228,10 +228,20 @@ def monotonic Process.clock_gettime(Process::CLOCK_MONOTONIC) end + # The default logger is effectively silent: it targets $stderr (so + # messages never collide with application stdout) and ships at level + # FATAL, which the consumer never emits. Set PGQUE_LOG_LEVEL=warn (or + # info, debug, error) to see warnings/info from the consumer, or + # pass logger: Logger.new(...) to Consumer.new for full control. def default_logger - log = Logger.new($stdout) + log = Logger.new($stderr) log.progname = "pgque.consumer.#{@name}" - log.level = ENV["PGQUE_LOG_LEVEL"] ? Logger.const_get(ENV["PGQUE_LOG_LEVEL"].upcase) : Logger::WARN + log.level = + if ENV["PGQUE_LOG_LEVEL"] + Logger.const_get(ENV["PGQUE_LOG_LEVEL"].upcase) + else + Logger::FATAL + end log end end From c2bfc97a2e88683ac91c888df120bcd7bc5fe5ba Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:06:04 +0400 Subject: [PATCH 21/40] fix(ruby): drop autocommit: kwarg; document Ruby pg semantics Ruby's pg gem has no per-connection autocommit attribute -- every exec_params runs in its own implicit transaction by default, the equivalent of psycopg's autocommit=True. Storing an autocommit flag on Pgque::Client implied behavior the gem could not actually deliver. This commit: - Removes autocommit: from Pgque.connect and Pgque::Client.connect/.new. - Removes Pgque::Client#autocommit? and the test_autocommit_flag test. - Adds a doc comment on Pgque.connect explaining that transaction control is per-call via conn.transaction { ... }, not per-connection. - Updates test_two_subconsumers_split_batches_no_duplicates with a comment explaining why the Python autocommit=True call has no Ruby equivalent (FOR UPDATE drops at end of statement). Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque.rb | 12 ++++++++++-- clients/ruby/lib/pgque/client.rb | 11 +++-------- clients/ruby/test/test_connect.rb | 9 --------- clients/ruby/test/test_coop.rb | 8 ++++++-- 4 files changed, 19 insertions(+), 21 deletions(-) diff --git a/clients/ruby/lib/pgque.rb b/clients/ruby/lib/pgque.rb index 576c4267..d6741c85 100644 --- a/clients/ruby/lib/pgque.rb +++ b/clients/ruby/lib/pgque.rb @@ -14,8 +14,16 @@ require "pgque/consumer" module Pgque - def self.connect(dsn, autocommit: false) - client = Client.connect(dsn, autocommit: autocommit) + # Open a connection and return a Pgque::Client. + # + # Ruby's pg gem runs each statement in its own implicit transaction + # by default -- the equivalent of psycopg's autocommit=True. To group + # statements into one transaction, use conn.transaction { ... } on the + # underlying PG::Connection (client.conn). There is no autocommit + # flag because Ruby pg has no per-connection autocommit attribute to + # toggle; transaction control is per-call via the transaction block. + def self.connect(dsn) + client = Client.connect(dsn) return client unless block_given? begin diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 39eb72ea..70514f75 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -12,21 +12,16 @@ module Pgque class Client attr_reader :conn - def self.connect(dsn, autocommit: false) + def self.connect(dsn) conn = PG.connect(dsn) - new(conn, owns_conn: true, autocommit: autocommit) + new(conn, owns_conn: true) rescue PG::ConnectionBad => e raise ConnectionError, e.message end - def initialize(conn, owns_conn: false, autocommit: false) + def initialize(conn, owns_conn: false) @conn = conn @owns_conn = owns_conn - @autocommit = autocommit - end - - def autocommit? - @autocommit end def close diff --git a/clients/ruby/test/test_connect.rb b/clients/ruby/test/test_connect.rb index 417316db..217a431c 100644 --- a/clients/ruby/test/test_connect.rb +++ b/clients/ruby/test/test_connect.rb @@ -33,15 +33,6 @@ def test_external_conn_is_not_closed_by_close end end - def test_autocommit_flag - Pgque.connect(dsn, autocommit: true) do |client| - assert client.autocommit? - end - Pgque.connect(dsn) do |client| - refute client.autocommit? - end - end - def test_close_is_idempotent client = Pgque.connect(dsn) client.close diff --git a/clients/ruby/test/test_coop.rb b/clients/ruby/test/test_coop.rb index 649b324c..7553daa1 100644 --- a/clients/ruby/test/test_coop.rb +++ b/clients/ruby/test/test_coop.rb @@ -100,8 +100,12 @@ def test_two_subconsumers_split_batches_no_duplicates producer.conn.exec_params("select pgque.ticker($1)", [q]) end - Pgque.connect(dsn, autocommit: true) do |c1| - Pgque.connect(dsn, autocommit: true) do |c2| + # Ruby pg runs each exec_params as its own implicit transaction, so + # the FOR UPDATE lock taken by receive_coop drops as soon as the + # call returns -- no autocommit flag needed (cf. psycopg's + # autocommit=True in the Python equivalent test). + Pgque.connect(dsn) do |c1| + Pgque.connect(dsn) do |c2| m1 = c1.receive_coop(q, consumer_n, "worker-1", max_messages: 100) m2 = c2.receive_coop(q, consumer_n, "worker-2", max_messages: 100) From 14455b53b3709a9c58129c5bf9be6befd48b1043 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:08:42 +0400 Subject: [PATCH 22/40] fix(ruby): coerce non-Hash/Array/String send payloads via to_s encode_payload's else branch returned the payload verbatim, which crashes the pg gem when the payload is a Symbol or any other object the gem can't serialize as a parameter. Coerce via #to_s so numerics and booleans round-trip naturally (42 -> "42" -> JSON 42; true -> "true" -> JSON true). Objects whose to_s isn't valid JSON (Symbols, Time, etc.) still surface a SQL error from the ::jsonb cast -- callers who care should pre-encode with JSON.dump. Adds a parameterized test asserting round-trip for Integer, Float, true, and false. The String-passthrough test already covers the case where the caller provides JSON text directly. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/client.rb | 11 ++++++++++- clients/ruby/test/test_send.rb | 24 ++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 70514f75..a6813e5c 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -160,11 +160,20 @@ def nack(batch_id, msg, retry_after: 60, reason: nil) private + # Hash/Array: JSON-encoded. + # nil: literal "null" so ::jsonb yields JSON null (not SQL NULL). + # String: passed through verbatim; caller must supply valid JSON text. + # Anything else (Integer, Float, true, false, Symbol, ...): coerced + # via #to_s so numerics and booleans round-trip naturally + # (42 -> "42", true -> "true"). Symbols and other objects whose + # to_s isn't valid JSON will surface a SQL error from the ::jsonb + # cast -- callers who care should pre-encode with JSON.dump. def encode_payload(payload) case payload when Hash, Array then JSON.dump(payload) when nil then "null" - else payload + when String then payload + else payload.to_s end end diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 2864fa6a..924fd063 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -47,6 +47,30 @@ def test_send_nil_payload end end + def test_send_numeric_and_boolean_payloads_coerce_via_to_s + # Non-String/Hash/Array/nil payloads run through to_s so numerics + # and booleans round-trip naturally as JSON scalars. + cases = [ + [42, 42], + [3.14, 3.14], + [true, true], + [false, false], + ] + cases.each do |payload, expected| + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, payload) + conn.exec_params("select pgque.force_next_tick($1)", [queue]) + conn.exec_params("select pgque.ticker($1)", [queue]) + msgs = client.receive(queue, consumer, 10) + assert_equal 1, msgs.size, "no message for #{payload.inspect}" + assert_equal expected, msgs[0].payload, + "#{payload.inspect} did not round-trip" + client.ack(msgs[0].batch_id) + end + end + end + def test_send_batch_returns_ids_in_order with_queue do |queue, _consumer, conn| client = Pgque::Client.new(conn) From 36615f220d679f82fa8e5b1c225a6fd7a095b22a Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:09:45 +0400 Subject: [PATCH 23/40] test(ruby): rollback in with_queue cleanup before drop If a test body left the conn in a failed transaction (e.g. an assertion failure after a SQL error without an explicit rollback), subsequent queries are rejected by Postgres until ROLLBACK -- which silently broke drop_queue under the rescued PG::Error and could leak the per-test queue across runs. The ensure block now starts with conn.exec("ROLLBACK") rescue nil so cleanup runs against a fresh transaction state. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_helper.rb | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/clients/ruby/test/test_helper.rb b/clients/ruby/test/test_helper.rb index a9aa916a..df1efa66 100644 --- a/clients/ruby/test/test_helper.rb +++ b/clients/ruby/test/test_helper.rb @@ -36,6 +36,13 @@ def with_queue yield q, c, conn ensure if conn && !conn.finished? + # Reset the connection's transaction state before cleaning up. + # If the test body left the conn in a failed transaction (an + # in-flight assertion failure after a SQL error, for example) + # any subsequent query is rejected until the transaction is + # rolled back -- which would silently break drop_queue and leak + # the test queue across runs. + conn.exec("ROLLBACK") rescue nil begin conn.exec_params("select pgque.unregister_consumer($1, $2)", [q, c]) if q && c conn.exec_params("select pgque.drop_queue($1, true)", [q]) if q From 34a429993cf768323c4450da14dc419229a04088 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:55:42 +0400 Subject: [PATCH 24/40] test(ruby): rollback before coop cleanup --- clients/ruby/test/test_coop.rb | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/clients/ruby/test/test_coop.rb b/clients/ruby/test/test_coop.rb index 7553daa1..edd5ad82 100644 --- a/clients/ruby/test/test_coop.rb +++ b/clients/ruby/test/test_coop.rb @@ -16,6 +16,10 @@ def with_coop_queue yield q, conn ensure if conn && !conn.finished? + # Reset failed transaction state before cleanup, mirroring + # with_queue. Otherwise a test that leaves the connection in + # PQTRANS_INERROR will make unsubscribe/drop fail and leak state. + conn.exec("ROLLBACK") rescue nil begin rows = conn.exec_params( "select c.co_name from pgque.consumer c " \ From a96562569245d7249bc991a7f49616790385ddd2 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:56:09 +0400 Subject: [PATCH 25/40] fix(ruby): ignore invalid PGQUE_LOG_LEVEL values --- clients/ruby/lib/pgque/consumer.rb | 19 +++++++++++++------ clients/ruby/test/test_consumer.rb | 11 +++++++++++ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb index a77361bb..1d56f829 100644 --- a/clients/ruby/lib/pgque/consumer.rb +++ b/clients/ruby/lib/pgque/consumer.rb @@ -236,13 +236,20 @@ def monotonic def default_logger log = Logger.new($stderr) log.progname = "pgque.consumer.#{@name}" - log.level = - if ENV["PGQUE_LOG_LEVEL"] - Logger.const_get(ENV["PGQUE_LOG_LEVEL"].upcase) - else - Logger::FATAL - end + log.level = env_log_level || Logger::FATAL log end + + def env_log_level + raw = ENV["PGQUE_LOG_LEVEL"] + return nil if raw.nil? + + normalized = raw.strip.upcase + return nil if normalized.empty? + + Logger.const_get(normalized) + rescue NameError + nil + end end end diff --git a/clients/ruby/test/test_consumer.rb b/clients/ruby/test/test_consumer.rb index 72858518..6b810d00 100644 --- a/clients/ruby/test/test_consumer.rb +++ b/clients/ruby/test/test_consumer.rb @@ -76,6 +76,17 @@ def test_consumer_dead_interval_without_subconsumer_raises end end + def test_invalid_pgque_log_level_falls_back_to_fatal + skip_dsn_for_this_class! + old = ENV["PGQUE_LOG_LEVEL"] + ENV["PGQUE_LOG_LEVEL"] = " warning " + + cons = Pgque::Consumer.new("dummy", queue: "q", name: "c") + assert_equal Logger::FATAL, cons.logger.level + ensure + ENV["PGQUE_LOG_LEVEL"] = old + end + private # Some unit tests don't actually connect; allow them even without DSN. From ba8ac355ab3df51eecfe98aa9f0002a43fa3c838 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 11:59:40 +0400 Subject: [PATCH 26/40] fix(ruby): preserve SQL error causes --- clients/ruby/lib/pgque/client.rb | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index a6813e5c..fc511982 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -50,7 +50,7 @@ def send(queue, payload, type: "default") end result.values[0][0].to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def send_batch(queue, type, payloads) @@ -62,7 +62,7 @@ def send_batch(queue, type, payloads) ) result.values.map { |r| r[0].to_i } rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def receive(queue, consumer, max_messages = 100) @@ -72,14 +72,14 @@ def receive(queue, consumer, max_messages = 100) ) result.values.map { |row| row_to_message(row) } rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def ack(batch_id) result = @conn.exec_params("select pgque.ack($1)", [batch_id]) result.values[0][0].to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def force_next_tick(queue) @@ -87,7 +87,7 @@ def force_next_tick(queue) v = result.values[0][0] v.nil? || v.empty? ? nil : v.to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end # Experimental: function names, edge-case behavior, and signatures may @@ -99,7 +99,7 @@ def subscribe_subconsumer(queue, consumer, subconsumer) ) result.values[0][0].to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def unsubscribe_subconsumer(queue, consumer, subconsumer, batch_handling: 0) @@ -109,7 +109,7 @@ def unsubscribe_subconsumer(queue, consumer, subconsumer, batch_handling: 0) ) result.values[0][0].to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def receive_coop(queue, consumer, subconsumer, max_messages: 100, @@ -120,7 +120,7 @@ def receive_coop(queue, consumer, subconsumer, max_messages: 100, ) result.values.map { |row| row_to_message(row) } rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def touch_subconsumer(queue, consumer, subconsumer) @@ -130,7 +130,7 @@ def touch_subconsumer(queue, consumer, subconsumer) ) result.values[0][0].to_i rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end def nack(batch_id, msg, retry_after: 60, reason: nil) @@ -155,7 +155,7 @@ def nack(batch_id, msg, retry_after: 60, reason: nil) ) nil rescue PG::Error => e - raise wrap_sql_error(e) + raise_wrapped_sql_error(e) end private @@ -218,5 +218,11 @@ def wrap_sql_error(error) Error.new(msg) end end + + def raise_wrapped_sql_error(error) + wrapped = wrap_sql_error(error) + wrapped.set_backtrace(error.backtrace) if error.backtrace + raise wrapped, cause: error + end end end From 835cbe7b8e932d344eeb1892082ca7bdc0048952 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 12:00:18 +0400 Subject: [PATCH 27/40] refactor(ruby): use PG::Result accessors --- clients/ruby/lib/pgque/client.rb | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index fc511982..587bfd29 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -48,7 +48,7 @@ def send(queue, payload, type: "default") [queue, encoded], ) end - result.values[0][0].to_i + integer_scalar(result) rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -60,7 +60,7 @@ def send_batch(queue, type, payloads) "select unnest(pgque.send_batch($1, $2, $3::jsonb[]))", [queue, type, array_literal], ) - result.values.map { |r| r[0].to_i } + integer_column(result) rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -70,21 +70,21 @@ def receive(queue, consumer, max_messages = 100) "select * from pgque.receive($1, $2, $3)", [queue, consumer, max_messages], ) - result.values.map { |row| row_to_message(row) } + result.each_row.map { |row| row_to_message(row) } rescue PG::Error => e raise_wrapped_sql_error(e) end def ack(batch_id) result = @conn.exec_params("select pgque.ack($1)", [batch_id]) - result.values[0][0].to_i + integer_scalar(result) rescue PG::Error => e raise_wrapped_sql_error(e) end def force_next_tick(queue) result = @conn.exec_params("select pgque.force_next_tick($1)", [queue]) - v = result.values[0][0] + v = scalar(result) v.nil? || v.empty? ? nil : v.to_i rescue PG::Error => e raise_wrapped_sql_error(e) @@ -97,7 +97,7 @@ def subscribe_subconsumer(queue, consumer, subconsumer) "select pgque.subscribe_subconsumer($1, $2, $3)", [queue, consumer, subconsumer], ) - result.values[0][0].to_i + integer_scalar(result) rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -107,7 +107,7 @@ def unsubscribe_subconsumer(queue, consumer, subconsumer, batch_handling: 0) "select pgque.unsubscribe_subconsumer($1, $2, $3, $4)", [queue, consumer, subconsumer, batch_handling], ) - result.values[0][0].to_i + integer_scalar(result) rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -118,7 +118,7 @@ def receive_coop(queue, consumer, subconsumer, max_messages: 100, "select * from pgque.receive_coop($1, $2, $3, $4, $5::interval)", [queue, consumer, subconsumer, max_messages, dead_interval], ) - result.values.map { |row| row_to_message(row) } + result.each_row.map { |row| row_to_message(row) } rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -128,7 +128,7 @@ def touch_subconsumer(queue, consumer, subconsumer) "select pgque.touch_subconsumer($1, $2, $3)", [queue, consumer, subconsumer], ) - result.values[0][0].to_i + integer_scalar(result) rescue PG::Error => e raise_wrapped_sql_error(e) end @@ -224,5 +224,17 @@ def raise_wrapped_sql_error(error) wrapped.set_backtrace(error.backtrace) if error.backtrace raise wrapped, cause: error end + + def scalar(result) + result.getvalue(0, 0) + end + + def integer_scalar(result) + scalar(result).to_i + end + + def integer_column(result) + result.column_values(0).map(&:to_i) + end end end From a9f835d524463506793b01e6d03d4fd3302a0645 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 12:01:17 +0400 Subject: [PATCH 28/40] refactor(ruby): simplify client conditionals --- clients/ruby/lib/pgque/client.rb | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 587bfd29..3f400075 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -37,7 +37,7 @@ def send(queue, payload, type: "default") end encoded = encode_payload(payload) result = - if type && type != "" && type != "default" + if custom_type?(type) @conn.exec_params( "select pgque.send($1, $2, $3::jsonb)", [queue, type, encoded], @@ -139,8 +139,7 @@ def nack(batch_id, msg, retry_after: 60, reason: nil) when nil then "null" else msg.payload.to_s end - created_at_str = msg.created_at.respond_to?(:iso8601) ? - msg.created_at.iso8601(6) : msg.created_at + created_at_str = format_created_at(msg.created_at) @conn.exec_params( "select pgque.nack($1, " \ @@ -236,5 +235,16 @@ def integer_scalar(result) def integer_column(result) result.column_values(0).map(&:to_i) end + + def custom_type?(type) + !type.to_s.empty? && type != "default" + end + + def format_created_at(value) + case value + when Time then value.iso8601(6) + else value + end + end end end From 0dbb4020d66e53197fda80bba7fe74d1170dfc01 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 12:29:31 +0400 Subject: [PATCH 29/40] fix(ruby): update test FakeResult to match getvalue accessor The PG::Result accessor refactor switched Pgque::Client from result.values[0][0] to result.getvalue(0, 0); TestSendSqlForm's in-class FakeConn::FakeResult still implemented values, so the two SQL-form-selection tests errored with NoMethodError. Re-implement FakeResult#getvalue to return the same canned id. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/test/test_send.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/clients/ruby/test/test_send.rb b/clients/ruby/test/test_send.rb index 924fd063..5a2342cc 100644 --- a/clients/ruby/test/test_send.rb +++ b/clients/ruby/test/test_send.rb @@ -192,8 +192,8 @@ def exec_params(sql, params) end class FakeResult - def values - [["999"]] + def getvalue(_row, _col) + "999" end end end From b8757b20c2226aa6990429fcaf06585aa1b3af14 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 12:29:33 +0400 Subject: [PATCH 30/40] feat(ruby/release): release-ruby.yml + RELEASE.md Adds the dispatch-only release workflow for the pgque gem and the companion RELEASE.md documenting the bootstrap + ongoing process. Workflow shape (mirrors release-python.yml): - on: workflow_dispatch with version (string) + dry_run (bool, default true). No push/tag triggers; code changes never run this file. - Top-level permissions: contents:read. publish job widens to id-token:write only where it needs OIDC. - build job (always runs on dispatch from main): Gem::Version sanity check, asserts inputs.version matches lib/pgque/version.rb's Pgque::VERSION, gem build, install the resulting .gem into a throwaway GEM_HOME, require "pgque" and assert VERSION + Client + Consumer are defined. Catches packaging mistakes before they reach the registry. - publish-rubygems job (gated on !dry_run, needs: build, environment: rubygems): rubygems/release-gem@v1 with setup-trusted-publisher:true exchanges the GitHub OIDC JWT for a short-lived rubygems.org API key and pushes; await-release:true blocks until the gem is fetchable. No long-lived RUBYGEMS_API_KEY secret. Three human gates protect against accidental publish: workflow_dispatch trigger, dry_run flag, and the rubygems GitHub environment (recommend configuring required reviewers there). RELEASE.md covers: gem identity, versioning conventions (dot-separated pre-releases vs Git-style hyphens), the manual bootstrap publish that RubyGems requires before trusted publishing can be configured, GitHub environment setup, the rubygems.org Trusted Publisher policy fields, the dry-run-then-publish process, and a note on why there is no TestPyPI-equivalent staging step (RubyGems has no public test registry). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/release-ruby.yml | 107 +++++++++++++++++++++++++++++ clients/ruby/RELEASE.md | 104 ++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+) create mode 100644 .github/workflows/release-ruby.yml create mode 100644 clients/ruby/RELEASE.md diff --git a/.github/workflows/release-ruby.yml b/.github/workflows/release-ruby.yml new file mode 100644 index 00000000..92d38f99 --- /dev/null +++ b/.github/workflows/release-ruby.yml @@ -0,0 +1,107 @@ +name: Release Ruby client + +on: + workflow_dispatch: + inputs: + version: + description: "Version to publish, matching clients/ruby/lib/pgque/version.rb" + required: true + type: string + dry_run: + description: "Build and validate without publishing" + required: true + default: true + type: boolean + +permissions: + contents: read + +jobs: + build: + if: github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + env: + VERSION: ${{ inputs.version }} + defaults: + run: + working-directory: clients/ruby + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.sha }} + + - uses: ruby/setup-ruby@v1 + with: + ruby-version: "3.3" + working-directory: clients/ruby + bundler-cache: true + + - name: Verify version input + run: | + set -Eeuo pipefail + # Reject malformed input early. Gem::Version is permissive + # (accepts 0.2.0, 0.2.0.rc.1, 0.2.0.alpha, 0.2.0.beta.1, ...) + # but rejects clear garbage like "x.y.z" or empty strings. + ruby -rrubygems -e "Gem::Version.new(ENV.fetch('VERSION'))" + + # The version constant must already match the input -- bumping + # version.rb is the responsibility of the release-prep PR, not + # this workflow. Any drift here is a configuration error. + actual=$(ruby -e 'require_relative "lib/pgque/version"; print Pgque::VERSION') + test "$actual" = "$VERSION" || { + echo "version input ${VERSION} != lib/pgque/version.rb ${actual}" + exit 1 + } + + - name: Build gem + run: gem build pgque.gemspec + + - name: Verify built gem installs + run: | + set -Eeuo pipefail + mkdir -p /tmp/pgque-gem-check + gem install --install-dir /tmp/pgque-gem-check --no-document \ + "./pgque-${VERSION}.gem" + GEM_PATH=/tmp/pgque-gem-check GEM_HOME=/tmp/pgque-gem-check \ + ruby -e ' + require "pgque" + expected = ENV.fetch("VERSION") + raise "version mismatch: expected #{expected}, got #{Pgque::VERSION}" \ + unless Pgque::VERSION == expected + raise "Pgque::Client missing" unless defined?(Pgque::Client) + raise "Pgque::Consumer missing" unless defined?(Pgque::Consumer) + puts "install verified: pgque #{Pgque::VERSION}" + ' + + publish-rubygems: + if: ${{ !inputs.dry_run }} + needs: build + runs-on: ubuntu-latest + environment: rubygems + permissions: + contents: read + id-token: write + defaults: + run: + working-directory: clients/ruby + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.sha }} + + - uses: ruby/setup-ruby@v1 + with: + ruby-version: "3.3" + working-directory: clients/ruby + bundler-cache: true + + # rubygems/release-gem builds the gem from the gemspec in + # working-directory and pushes via OIDC. setup-trusted-publisher + # exchanges the GitHub-issued JWT for a short-lived rubygems.org + # API key -- no long-lived RUBYGEMS_API_KEY secret is needed. + - name: Publish to RubyGems via OIDC + uses: rubygems/release-gem@v1 + with: + working-directory: clients/ruby + setup-trusted-publisher: true + await-release: true diff --git a/clients/ruby/RELEASE.md b/clients/ruby/RELEASE.md new file mode 100644 index 00000000..5298093e --- /dev/null +++ b/clients/ruby/RELEASE.md @@ -0,0 +1,104 @@ +# Ruby client release + +Gem name: `pgque` on RubyGems.org. + +```bash +gem install pgque --pre # while v0.2.0 is in release-candidate +``` + +```ruby +require "pgque" +``` + +## Versioning + +The Ruby client version is independent from the SQL/server +`pgque.version()`. Bump this gem when the Ruby API or packaging changes; +server-only SQL changes do not require a Ruby client release. + +Use Ruby gem version strings in `clients/ruby/lib/pgque/version.rb`. For a +pre-release build, use dot-separated suffixes like `0.2.0.rc.1`, +`0.2.0.alpha.1`, or `0.2.0.beta`; do **not** use Git-style `0.2.0-dev` +hyphens, which `Gem::Version` parses but other tooling does not. +RubyGems treats any version containing a non-numeric segment as a +pre-release; users need `gem install pgque --pre` to receive it. + +## Bootstrap (first publish only) + +RubyGems' Trusted Publishing requires the gem to **already exist** on +the registry before a trusted publisher can be configured. The very +first release is therefore manual: + +```bash +cd clients/ruby +gem build pgque.gemspec +gem signin # one-time, prompts for rubygems.org credentials +gem push pgque-0.2.0.rc.1.gem +``` + +After that, every subsequent release goes through the workflow below. + +## GitHub environment prerequisite + +Before the first workflow-driven publish, create a GitHub environment +in `NikolayS/pgque`: + +- `rubygems` + +Protect it as appropriate for releases (for example, required reviewers +and `main` branch restrictions). The workflow also checks that it is +running from `main`, but environment protection is the human approval +gate. + +## RubyGems Trusted Publisher prerequisite + +After the bootstrap publish, configure Trusted Publishing on +rubygems.org: + +1. Sign in to rubygems.org and open the gem's page. +2. **Settings → Trusted Publishers → Add Publisher**. +3. Provider: GitHub Actions. +4. Repository: `NikolayS/pgque`. +5. Workflow: `release-ruby.yml`. +6. Environment: `rubygems`. + +Pin to a specific tag/branch only if you want to lock down which refs +can publish; otherwise leave the ref restriction empty. + +## Release process + +The release workflow is `.github/workflows/release-ruby.yml`. + +1. Update `clients/ruby/lib/pgque/version.rb` and any release notes / + changelog if present. +2. Merge the release prep PR. +3. Ensure the `rubygems` GitHub environment exists and is protected. +4. Ensure the gem already exists on RubyGems and Trusted Publishing + is configured (bootstrap section above). +5. Run **Release Ruby client** with `dry_run=true` first. Dry runs + only build, validate the version match, and smoke-install the + resulting `.gem`; they do not require the `rubygems` environment + approval or OIDC permissions. +6. Run it with `dry_run=false`. Approve the `rubygems` environment + when prompted. +7. Verify the published artifact installs in a clean environment: + + ```bash + gem install pgque --pre # or pin: gem install pgque -v 0.2.0.rc.1 + ruby -rpgque -e 'puts Pgque::VERSION' + ``` + +The workflow builds with `gem build`, smoke-installs the resulting +`.gem` against a temporary `GEM_HOME`, and publishes via RubyGems +Trusted Publishing / OIDC. No long-lived `RUBYGEMS_API_KEY` is +needed. + +## Why no test registry? + +Unlike PyPI's TestPyPI sibling, RubyGems.org has no public staging +instance. Dry-run validation in this workflow covers `gem build` and +local install verification; the next step is the real publish. If you +need an isolated end-to-end test for the publish path itself, push to +a privately-owned alias gem (e.g. `pgque-staging`) using the same +workflow with a different gemspec name, then drop the alias gem when +you're done. From 9727982ce7ac7158c313681b7c80fabce44c7186 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Sat, 9 May 2026 15:10:49 +0400 Subject: [PATCH 31/40] refactor(ruby): rename Consumer#set_running to running= Use the Ruby setter idiom: self.running = value reads more naturally at call sites than set_running(value), and matches the running? predicate reader. Stays private under the existing private: modifier, so external callers still go through #stop to flip the flag. The bare assignment running = value would create a local variable inside an instance method, so the three call sites in start, stop, and the signal proc all use self.running = ... explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/consumer.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb index 1d56f829..1744132d 100644 --- a/clients/ruby/lib/pgque/consumer.rb +++ b/clients/ruby/lib/pgque/consumer.rb @@ -59,14 +59,14 @@ def on(event_type, &block) end def start - set_running(true) + self.running = true in_main_thread = (Thread.current == Thread.main) original_handlers = {} stop_proc = ->(signum) { @logger.info("received signal #{signum}, shutting down") - set_running(false) + self.running = false } if in_main_thread @@ -101,7 +101,7 @@ def start end def stop - set_running(false) + self.running = false end def running? @@ -220,7 +220,7 @@ def wait_for_notify_or_stop(conn) end end - def set_running(value) + def running=(value) @running_mutex.synchronize { @running = value } end From 755ffcb8c07552c65bcfd33beacbb46ce3c992be Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 11 May 2026 13:13:15 +0400 Subject: [PATCH 32/40] fix(ruby): keep signal trap async-signal-safe The SIGTERM/SIGINT proc installed by Consumer#start was calling @logger.info(...) and a Mutex#synchronize-backed running= setter -- both raise ThreadError when invoked from Ruby's trap context. A real signal delivered while start ran on the main thread would have crashed the trap (and likely the process) instead of cleanly shutting the consumer down. Fix: - Drop @running_mutex. The flag is a single boolean with no ordering dependencies, and Ruby integer/boolean assignment is atomic. The mutex was both unnecessary and the proximate cause of the trap-context crash. - Reduce the trap proc to two plain instance-variable writes: @stop_signum = signum; @running = false. No Logger, no synchronize, no other blocking work. - Move the "received signal N, shutting down" log line out of the trap into the post-loop block, gated on @stop_signum. It runs on the main thread after the wait wakes, so Logger is safe again. - Remove the now-pointless private running= setter and drop self.running = ... at the two non-trap call sites in favor of direct @running assignment. The existing consumer tests never exercised the trap path because they all call cons.start on a worker thread, where Thread.current == Thread.main is false and signal handlers are not installed. The bug only surfaces in production (running consumer on the main thread, e.g. a `bundle exec ruby my_worker.rb`) when the process actually receives TERM/INT. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/consumer.rb | 31 ++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb index 1744132d..7e0893c7 100644 --- a/clients/ruby/lib/pgque/consumer.rb +++ b/clients/ruby/lib/pgque/consumer.rb @@ -42,8 +42,14 @@ def initialize(dsn, queue:, name:, poll_interval: 30, @handlers = {} @default_handler = nil - @running_mutex = Mutex.new + # @running is a plain boolean. Ruby integer/boolean assignment + # is atomic, and the only cross-thread interactions are the + # signal trap and Consumer#stop flipping it false while the + # main loop polls running? -- no ordering dependencies, so a + # mutex would be overkill (and unsafe to enter from a signal + # trap, which raises ThreadError on Mutex#synchronize). @running = false + @stop_signum = nil @logger = logger || default_logger end @@ -59,14 +65,19 @@ def on(event_type, &block) end def start - self.running = true + @running = true + @stop_signum = nil in_main_thread = (Thread.current == Thread.main) original_handlers = {} + # Signal traps run in a restricted context: Mutex#synchronize, + # Logger#info, and most blocking code raise ThreadError. Keep + # this proc to plain instance-variable writes; the main loop + # logs the signal number after waking up. stop_proc = ->(signum) { - @logger.info("received signal #{signum}, shutting down") - self.running = false + @stop_signum = signum + @running = false } if in_main_thread @@ -89,6 +100,10 @@ def start break unless running? wait_for_notify_or_stop(conn) end + + if @stop_signum + @logger.info("received signal #{@stop_signum}, shutting down") + end ensure conn.close unless conn.finished? end @@ -101,11 +116,11 @@ def start end def stop - self.running = false + @running = false end def running? - @running_mutex.synchronize { @running } + @running end # Public for testability; not part of the stable API. @@ -220,10 +235,6 @@ def wait_for_notify_or_stop(conn) end end - def running=(value) - @running_mutex.synchronize { @running = value } - end - def monotonic Process.clock_gettime(Process::CLOCK_MONOTONIC) end From 648497227f56a47d133fe1a138750ae1b5a3f7bb Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 11 May 2026 13:17:14 +0400 Subject: [PATCH 33/40] fix(ruby): Consumer#start clears running? on exception start set @running = true up front but only restored signal handlers in the outer ensure. If PG.connect, LISTEN, poll_once, or any SQL call raised, the consumer exited with @running still true -- so callers polling consumer.running? saw "running" with no live worker behind it. Fix: zero @running in the outer ensure (plain instance-var write, trap-safe). Add a regression test that points start at an unreachable host so PG.connect raises immediately, and asserts running? is false both before start and after the failure. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/lib/pgque/consumer.rb | 7 +++++++ clients/ruby/test/test_consumer.rb | 22 ++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/clients/ruby/lib/pgque/consumer.rb b/clients/ruby/lib/pgque/consumer.rb index 7e0893c7..e2a13b58 100644 --- a/clients/ruby/lib/pgque/consumer.rb +++ b/clients/ruby/lib/pgque/consumer.rb @@ -108,6 +108,13 @@ def start conn.close unless conn.finished? end ensure + # Clear running? before logging so callers observing the flag + # see "stopped" by the time the log line is written -- and so + # an exception during PG.connect, LISTEN, or the poll loop + # leaves the consumer in a consistent state instead of a + # ghost "running" with no live worker. Plain instance-var + # write -- not the trap-context-unsafe pattern. + @running = false if in_main_thread original_handlers.each { |sig, h| Signal.trap(sig, h || "DEFAULT") } end diff --git a/clients/ruby/test/test_consumer.rb b/clients/ruby/test/test_consumer.rb index 6b810d00..bedf3276 100644 --- a/clients/ruby/test/test_consumer.rb +++ b/clients/ruby/test/test_consumer.rb @@ -76,6 +76,28 @@ def test_consumer_dead_interval_without_subconsumer_raises end end + def test_running_clears_after_start_failure + skip_dsn_for_this_class! + # Point at an unreachable port so PG.connect raises immediately + # inside start, exiting before the poll loop ever runs. + cons = Pgque::Consumer.new( + "postgresql://nobody:wrong@localhost:1/nodb", + queue: "q", name: "c", logger: silent_logger + ) + refute cons.running? + assert_raises(PG::Error) { cons.start } + refute cons.running?, + "consumer.running? must be false after start raised" + end + + def silent_logger + require "logger" + require "stringio" + log = Logger.new(StringIO.new) + log.level = Logger::FATAL + log + end + def test_invalid_pgque_log_level_falls_back_to_fatal skip_dsn_for_this_class! old = ENV["PGQUE_LOG_LEVEL"] From ec6a75da386b0cb5c41fe524764ebd5ff3297ecf Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 11 May 2026 13:42:49 +0400 Subject: [PATCH 34/40] fix(ruby/release): wire up rake release for publish workflow rubygems/release-gem@v1 runs `bundle exec rake release`, but clients/ruby/Rakefile only defined :test, so the publish job would have failed with "Don't know how to build task 'release'" the first time anyone hit it. Fix: - clients/ruby/Rakefile: add `require "bundler/gem_tasks"` so the conventional release task chain is available (rake build, rake install, rake release[remote] and the three release:* subtasks). - .github/workflows/release-ruby.yml publish-rubygems job: grant contents:write (needed by release:source_control_push to push the v${VERSION} tag back to origin via the GITHUB_TOKEN that actions/checkout configures automatically), switch checkout to ref: ${{ github.ref }} with fetch-depth:0 so we land on an attached HEAD with tags visible (rake release's plain `git push` refuses to operate from detached HEAD), and configure the github-actions[bot] git identity for the annotated tag. - clients/ruby/RELEASE.md: document the rake release chain and call out the side effect that the workflow pushes a v${VERSION} git tag to NikolayS/pgque, plus the partial-failure recovery (yank gem + git push --delete origin v${VERSION}). Race note: the build job pins to inputs.version against ${{ github.sha }}; if main races forward during the workflow the publish job picks up the new head of refs/heads/main, but the race window would have to land both a fresh commit and a version bump for the publish to ship anything different. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/release-ruby.yml | 32 ++++++++++++++++++++++++------ clients/ruby/RELEASE.md | 23 +++++++++++++++++++++ clients/ruby/Rakefile | 1 + 3 files changed, 50 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release-ruby.yml b/.github/workflows/release-ruby.yml index 92d38f99..7e64dee3 100644 --- a/.github/workflows/release-ruby.yml +++ b/.github/workflows/release-ruby.yml @@ -79,15 +79,28 @@ jobs: runs-on: ubuntu-latest environment: rubygems permissions: - contents: read + # contents:write is required so `rake release` (invoked by + # rubygems/release-gem) can push the v${VERSION} git tag back to + # origin. id-token:write is for the OIDC handshake with + # rubygems.org. + contents: write id-token: write defaults: run: working-directory: clients/ruby steps: + # Check out the branch ref (not the bare SHA) so we land on an + # attached HEAD; bundler's release:source_control_push runs plain + # `git push`, which fails from detached HEAD. The build job (run + # immediately before this one) already verified the version + # against the dispatch SHA, so any race with main moving during + # the workflow would have to land a new commit AND a version + # bump in that window -- vanishingly small. - uses: actions/checkout@v4 with: - ref: ${{ github.sha }} + ref: ${{ github.ref }} + # Full history so existing tags are visible to release:guard_clean. + fetch-depth: 0 - uses: ruby/setup-ruby@v1 with: @@ -95,10 +108,17 @@ jobs: working-directory: clients/ruby bundler-cache: true - # rubygems/release-gem builds the gem from the gemspec in - # working-directory and pushes via OIDC. setup-trusted-publisher - # exchanges the GitHub-issued JWT for a short-lived rubygems.org - # API key -- no long-lived RUBYGEMS_API_KEY secret is needed. + - name: Configure git identity for tag push + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + # rubygems/release-gem performs the OIDC handshake (no long-lived + # RUBYGEMS_API_KEY secret needed) and then runs + # `bundle exec rake release`, which depends on the gem tasks + # provided by `require "bundler/gem_tasks"` in clients/ruby/Rakefile. + # The chain is: build -> release:guard_clean -> release:source_control_push + # (tags v${VERSION} and pushes the tag) -> release:rubygem_push. - name: Publish to RubyGems via OIDC uses: rubygems/release-gem@v1 with: diff --git a/clients/ruby/RELEASE.md b/clients/ruby/RELEASE.md index 5298093e..fbebc483 100644 --- a/clients/ruby/RELEASE.md +++ b/clients/ruby/RELEASE.md @@ -93,6 +93,29 @@ The workflow builds with `gem build`, smoke-installs the resulting Trusted Publishing / OIDC. No long-lived `RUBYGEMS_API_KEY` is needed. +The publish step uses `rubygems/release-gem@v1`, which runs +`bundle exec rake release`. That task (provided by +`require "bundler/gem_tasks"` in `clients/ruby/Rakefile`) chains: + +1. `rake build` — builds `pgque-${VERSION}.gem` under `pkg/`. +2. `release:guard_clean` — refuses to release if the working tree + has uncommitted changes (CI checkouts are clean). +3. `release:source_control_push` — annotates the head commit with a + `v${VERSION}` tag and pushes that tag to `origin`. The + `contents: write` permission on the publish job, plus the + `GITHUB_TOKEN` automatically injected by `actions/checkout`, is + what authorizes the push. **The release workflow therefore + pushes a git tag to `NikolayS/pgque` as a side effect.** If you + need to retract a release, yank the gem on RubyGems *and* delete + the tag with `git push --delete origin v${VERSION}`. +4. `release:rubygem_push` — `gem push pkg/pgque-${VERSION}.gem`. + +If the gem push fails after the tag has already been pushed (rare +but possible if rubygems.org is degraded), you'll have a `v${VERSION}` +tag with no corresponding published gem. Re-running the workflow +will then fail at `release:guard_clean` if the tag already exists; +delete the tag and re-dispatch. + ## Why no test registry? Unlike PyPI's TestPyPI sibling, RubyGems.org has no public staging diff --git a/clients/ruby/Rakefile b/clients/ruby/Rakefile index 174c5caa..ecb388b9 100644 --- a/clients/ruby/Rakefile +++ b/clients/ruby/Rakefile @@ -1,3 +1,4 @@ +require "bundler/gem_tasks" require "rake/testtask" Rake::TestTask.new(:test) do |t| From 3e2bb4ff37e80b5fabc1d7042c252bd873a2d181 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 11 May 2026 13:47:13 +0400 Subject: [PATCH 35/40] docs(ruby): make Ruby quickstart self-complete CLAUDE.md requires install/quickstart docs to cover: install command, ticker setup (or skip path), and role grants. The Ruby README had install + roles but no ticker guidance, and the root-README Ruby snippet jumped from connect to Consumer without showing queue or consumer registration. - clients/ruby/README.md: add a Quickstart section between Database permissions and the Object#send note, with one short snippet covering create_queue + subscribe + send + Consumer.on/start, followed by a paragraph on the pg_cron ticker path and the external-scheduler alternative. - README.md (Ruby section under Client libraries): add the one-time setup lines (create_queue + subscribe) and a trailing comment that consumer.start needs pgque.ticker() running, matching the rest of the doc's self-complete style. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 8 +++++--- clients/ruby/README.md | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 47716a49..29e0f957 100644 --- a/README.md +++ b/README.md @@ -417,6 +417,10 @@ gem install pgque --pre # or pin: gem "pgque", "0.2.0.rc.1" require "pgque" Pgque.connect("postgresql://localhost/mydb") do |client| + # one-time setup (typically in a migration) + client.conn.exec("select pgque.create_queue('orders')") + client.conn.exec("select pgque.subscribe('orders', 'processor')") + client.send("orders", { "order_id" => 42 }, type: "order.created") end @@ -425,10 +429,8 @@ consumer = Pgque::Consumer.new( queue: "orders", name: "processor", ) - consumer.on("order.created") { |msg| process_order(msg.payload) } - -consumer.start # blocks until SIGTERM/SIGINT +consumer.start # blocks until SIGTERM/SIGINT; needs pgque.ticker() running ``` ### Any language diff --git a/clients/ruby/README.md b/clients/ruby/README.md index deca62ee..25e53d64 100644 --- a/clients/ruby/README.md +++ b/clients/ruby/README.md @@ -35,6 +35,40 @@ grant pgque_writer to your_app_user; See [`docs/reference.md` — Roles and grants](../../docs/reference.md#roles-and-grants). +## Quickstart + +Run the one-time setup once (typically in a migration), then produce +and consume from any process: + +```ruby +require "pgque" + +Pgque.connect("postgresql://localhost/mydb") do |client| + # one-time setup + client.conn.exec("select pgque.create_queue('orders')") + client.conn.exec("select pgque.subscribe('orders', 'order_worker')") + + # produce + client.send("orders", { "order_id" => 42 }, type: "order.created") +end + +# consume (separate process) +consumer = Pgque::Consumer.new( + "postgresql://localhost/mydb", + queue: "orders", + name: "order_worker", +) +consumer.on("order.created") { |msg| process_order(msg.payload) } +consumer.start # blocks until SIGTERM / SIGINT +``` + +The consumer only sees events after `pgque.ticker()` has materialized +a batch. With `pg_cron` available, run `select pgque.start();` once +to schedule the default 10 ticks/sec. Without `pg_cron`, drive +ticking from your application or an external scheduler — see the +project [Installation](https://github.com/NikolayS/pgque#installation) +section for both paths. + ## A note on `Pgque::Client#send` The producer method is called `send` to mirror the SQL surface From ae4e4c7bbc19f7ae93f1a8a954abe34ff534528f Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 11 May 2026 13:51:47 +0400 Subject: [PATCH 36/40] docs: stop using v0.2.0-rc.1 for all four clients Each ecosystem normalises pre-release version strings differently: PyPI prints 0.2.0rc1 (no separator), RubyGems uses 0.2.0.rc.1 (dot-separated -- hyphens are warned against in the gem RELEASE.md), and npm + Go use 0.2.0-rc.1 (semver hyphen). The single shared "v0.2.0-rc.1" literal in the Client libraries intro was wrong for Python and Ruby. Replace the universal literal with a short note that each per-language section below shows the correct spelling. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 29e0f957..3a9d307f 100644 --- a/README.md +++ b/README.md @@ -333,7 +333,7 @@ Longer walkthrough in the [tutorial](docs/tutorial.md); patterns like fan-out, e ## Client libraries -PgQue is SQL-first, so any Postgres driver works. First-party client libraries live in this repo for **Python**, **Go**, **TypeScript**, and **Ruby**, all published at `v0.2.0-rc.1`. +PgQue is SQL-first, so any Postgres driver works. First-party client libraries live in this repo for **Python**, **Go**, **TypeScript**, and **Ruby** — all in v0.2 release-candidate state. Each section below shows the install command for that ecosystem's version spelling: PyPI uses `0.2.0rc1`, RubyGems uses `0.2.0.rc.1`, npm and Go use `0.2.0-rc.1`. ### Python (`pgque-py`) — psycopg 3 From 8aaccbba9e3dca4eaa4733d8843d0b7a864df474 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 25 May 2026 23:47:48 +0400 Subject: [PATCH 37/40] feat(ruby): add ticker and ticker_all wrappers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bring Ruby to public-API parity with Python/Go/TypeScript on the ticker helpers. `Client#ticker(queue)` wraps `pgque.ticker($1)` and returns the new tick id as Integer (or nil when no tick was needed, matching the nullable-scalar pattern used by `force_next_tick`). `Client#ticker_all` wraps zero-argument `pgque.ticker()` and returns the Integer count of queues that received a tick. Tests follow the existing two-class split in test_send.rb: TestTickerSqlForm uses FakeConn to assert the SQL form and return shape with no DB, and TestTicker exercises real behavior against PGQUE_TEST_DSN (skipped without one). Flips the ticker row's Ruby cell from ✗ to ✓ in the parity matrix and trims the legend's ticker call-out (the remaining gap is subscribe/unsubscribe wrappers). Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/README.md | 7 +-- clients/ruby/lib/pgque/client.rb | 15 +++++ clients/ruby/test/test_ticker.rb | 96 ++++++++++++++++++++++++++++++++ 3 files changed, 114 insertions(+), 4 deletions(-) create mode 100644 clients/ruby/test/test_ticker.rb diff --git a/clients/README.md b/clients/README.md index 454bb197..6aa59a4c 100644 --- a/clients/README.md +++ b/clients/README.md @@ -85,7 +85,7 @@ users to install `--pre`, `@rc`, or an `-rc` Go tag. | `receive` | ✓ | ✓ | ✓ | ✓ | | `ack` returns SQL rowcount (0 stale, 1 success) | ✓ (int) | ✓ (int64) | ✓ (number) | ✓ (Integer) | | `nack` | ✓ | ✓ | ✓ | ✓ | -| `ticker` / `Ticker` / `ticker`, `ticker_all` / `TickerAll` / `tickerAll` | ✓ | ✓ | ✓ | ✗ | +| `ticker` / `Ticker` / `ticker`, `ticker_all` / `TickerAll` / `tickerAll` | ✓ | ✓ | ✓ | ✓ | | `force_next_tick` / `ForceNextTick` / `forceNextTick` | ✓ | ✓ | ✓ | ✓ | | `nack` retry delay + reason options | ✓ | ✓ | ✓ | ✓ | | High-level `Consumer` | ✓ | ✓ | ✓ | ✓ | @@ -100,9 +100,8 @@ users to install `--pre`, `@rc`, or an `-rc` Go tag. Legend: ✓ supported by the client API on `main`; ✗ not exposed as a first-class client API. Lower-level SQL primitives remain available through raw -connection/pool escape hatches. Python, Go, and TypeScript expose `ticker` and -`subscribe` / `unsubscribe` convenience wrappers; Ruby can call those via raw -SQL. +connection/pool escape hatches. Python, Go, and TypeScript expose `subscribe` / +`unsubscribe` convenience wrappers; Ruby can call those via raw SQL. [^coop]: Experimental. Each supporting client exposes `subscribe_subconsumer` / `unsubscribe_subconsumer` / `receive_coop` / diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 3f400075..531cf9e4 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -90,6 +90,21 @@ def force_next_tick(queue) raise_wrapped_sql_error(e) end + def ticker(queue) + result = @conn.exec_params("select pgque.ticker($1)", [queue]) + v = scalar(result) + v.nil? || v.empty? ? nil : v.to_i + rescue PG::Error => e + raise_wrapped_sql_error(e) + end + + def ticker_all + result = @conn.exec_params("select pgque.ticker()", []) + integer_scalar(result) + rescue PG::Error => e + raise_wrapped_sql_error(e) + end + # Experimental: function names, edge-case behavior, and signatures may # change before the cooperative API is marked stable. def subscribe_subconsumer(queue, consumer, subconsumer) diff --git a/clients/ruby/test/test_ticker.rb b/clients/ruby/test/test_ticker.rb new file mode 100644 index 00000000..8ab0dadb --- /dev/null +++ b/clients/ruby/test/test_ticker.rb @@ -0,0 +1,96 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestTicker < Minitest::Test + include PgqueTest::Helpers + + def test_ticker_after_force_next_tick_returns_non_nil_integer + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + client.send(queue, { "x" => 1 }, type: "tick.test") + client.force_next_tick(queue) + tick_id = client.ticker(queue) + refute_nil tick_id, "ticker after force_next_tick must produce a tick" + assert_kind_of Integer, tick_id + assert_operator tick_id, :>, 0 + end + end + + def test_ticker_returns_nil_when_no_new_tick_needed + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + client.force_next_tick(queue) + client.ticker(queue) + # Immediately again with no new activity: ticker returns nil. + assert_nil client.ticker(queue), + "second ticker call with no new events must return nil" + end + end + + def test_ticker_all_returns_non_negative_integer + Pgque.connect(dsn) do |client| + n = client.ticker_all + assert_kind_of Integer, n + assert_operator n, :>=, 0 + end + end +end + +class TestTickerSqlForm < Minitest::Test + # Capture exec_params calls without a real DB. + class FakeConn + attr_reader :sql_used, :params_used + + def initialize(scalar:) + @scalar = scalar + end + + def exec_params(sql, params) + @sql_used = sql + @params_used = params + FakeResult.new(@scalar) + end + + class FakeResult + def initialize(value) + @value = value + end + + def getvalue(_row, _col) + @value + end + end + end + + def test_ticker_issues_single_queue_sql + conn = FakeConn.new(scalar: "42") + client = Pgque::Client.new(conn) + tick_id = client.ticker("orders") + assert_equal 42, tick_id + assert_includes conn.sql_used, "pgque.ticker($1)" + assert_equal ["orders"], conn.params_used + end + + def test_ticker_returns_nil_for_nil_scalar + conn = FakeConn.new(scalar: nil) + client = Pgque::Client.new(conn) + assert_nil client.ticker("orders") + end + + def test_ticker_returns_nil_for_empty_scalar + conn = FakeConn.new(scalar: "") + client = Pgque::Client.new(conn) + assert_nil client.ticker("orders") + end + + def test_ticker_all_issues_zero_arg_sql + conn = FakeConn.new(scalar: "3") + client = Pgque::Client.new(conn) + n = client.ticker_all + assert_equal 3, n + assert_includes conn.sql_used, "pgque.ticker()" + refute_includes conn.sql_used, "$1" + assert_equal [], conn.params_used + end +end From 0e3a440046f7f3da7764a6a7448bcec4a1eef833 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Mon, 25 May 2026 23:55:38 +0400 Subject: [PATCH 38/40] feat(ruby): add subscribe and unsubscribe wrappers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the last public-API gap with Python/Go/TypeScript. `Client#subscribe(queue, consumer)` wraps `pgque.subscribe($1, $2)` and `Client#unsubscribe(queue, consumer)` wraps the unsubscribe counterpart. Both return Integer: 1 when the (un)subscription happened, 0 when it was already in that state. The underlying SQL is idempotent so neither raises on the no-op call. Tests follow the two-class split: TestSubscribeSqlForm uses FakeConn to pin the SQL form and param shape with no DB, and TestSubscribe exercises the 1-then-0 idempotency and an end-to-end subscribe -> send -> tick -> receive -> ack flow against PGQUE_TEST_DSN. Flips the subscribe/unsubscribe row to ✓ for Ruby in the parity matrix and drops the legend's raw-SQL fallback note (Ruby is now at full first-class parity with the other clients). Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/README.md | 5 +- clients/ruby/lib/pgque/client.rb | 18 +++++ clients/ruby/test/test_subscribe.rb | 114 ++++++++++++++++++++++++++++ 3 files changed, 134 insertions(+), 3 deletions(-) create mode 100644 clients/ruby/test/test_subscribe.rb diff --git a/clients/README.md b/clients/README.md index 6aa59a4c..2c00fcbf 100644 --- a/clients/README.md +++ b/clients/README.md @@ -95,13 +95,12 @@ users to install `--pre`, `@rc`, or an `-rc` Go tag. | `Consumer` retry delay option | ✓ | ✓ | ✓ | ✓ | | Unknown-type behavior avoids silent ack | ✓ | ✓ | ✓ | ✓ | | Configurable unknown-type policy | ✓ | ✓ | ✓ | ✓ | -| `subscribe` / `unsubscribe` wrappers | ✓ | ✓ | ✓ | ✗ | +| `subscribe` / `unsubscribe` wrappers | ✓ | ✓ | ✓ | ✓ | | Cooperative consumers (experimental) [^coop] | ✓ | ✓ | ✓ | ✓ | Legend: ✓ supported by the client API on `main`; ✗ not exposed as a first-class client API. Lower-level SQL primitives remain available through raw -connection/pool escape hatches. Python, Go, and TypeScript expose `subscribe` / -`unsubscribe` convenience wrappers; Ruby can call those via raw SQL. +connection/pool escape hatches. [^coop]: Experimental. Each supporting client exposes `subscribe_subconsumer` / `unsubscribe_subconsumer` / `receive_coop` / diff --git a/clients/ruby/lib/pgque/client.rb b/clients/ruby/lib/pgque/client.rb index 531cf9e4..8b5dcd40 100644 --- a/clients/ruby/lib/pgque/client.rb +++ b/clients/ruby/lib/pgque/client.rb @@ -82,6 +82,24 @@ def ack(batch_id) raise_wrapped_sql_error(e) end + def subscribe(queue, consumer) + result = @conn.exec_params( + "select pgque.subscribe($1, $2)", [queue, consumer] + ) + integer_scalar(result) + rescue PG::Error => e + raise_wrapped_sql_error(e) + end + + def unsubscribe(queue, consumer) + result = @conn.exec_params( + "select pgque.unsubscribe($1, $2)", [queue, consumer] + ) + integer_scalar(result) + rescue PG::Error => e + raise_wrapped_sql_error(e) + end + def force_next_tick(queue) result = @conn.exec_params("select pgque.force_next_tick($1)", [queue]) v = scalar(result) diff --git a/clients/ruby/test/test_subscribe.rb b/clients/ruby/test/test_subscribe.rb new file mode 100644 index 00000000..7d78626a --- /dev/null +++ b/clients/ruby/test/test_subscribe.rb @@ -0,0 +1,114 @@ +# Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. + +require_relative "test_helper" + +class TestSubscribe < Minitest::Test + include PgqueTest::Helpers + + def test_subscribe_returns_one_for_new_then_zero_for_existing + with_queue do |queue, _consumer, conn| + client = Pgque::Client.new(conn) + fresh = "#{unique_consumer_name}_sub" + begin + first = client.subscribe(queue, fresh) + assert_equal 1, first, "first subscribe must return 1 for a fresh consumer" + second = client.subscribe(queue, fresh) + assert_equal 0, second, "second subscribe must return 0 (already registered)" + ensure + conn.exec_params( + "select pgque.unregister_consumer($1, $2)", [queue, fresh] + ) rescue nil + end + end + end + + def test_unsubscribe_returns_positive_for_existing_then_zero_for_missing + with_queue do |queue, consumer, conn| + client = Pgque::Client.new(conn) + first = client.unsubscribe(queue, consumer) + assert_operator first, :>=, 1, + "first unsubscribe of a registered consumer must return >= 1" + second = client.unsubscribe(queue, consumer) + assert_equal 0, second, + "second unsubscribe must return 0 (no longer registered)" + end + end + + def test_subscribed_consumer_can_receive_messages + with_queue do |queue, _registered, conn| + client = Pgque::Client.new(conn) + fresh = "#{unique_consumer_name}_recv" + begin + client.subscribe(queue, fresh) + client.send(queue, { "x" => 1 }, type: "sub.test") + client.force_next_tick(queue) + client.ticker(queue) + msgs = client.receive(queue, fresh, 10) + assert_equal 1, msgs.size + assert_equal "sub.test", msgs[0].type + client.ack(msgs[0].batch_id) + ensure + conn.exec_params( + "select pgque.unregister_consumer($1, $2)", [queue, fresh] + ) rescue nil + end + end + end +end + +class TestSubscribeSqlForm < Minitest::Test + # Capture exec_params calls without a real DB. + class FakeConn + attr_reader :sql_used, :params_used + + def initialize(scalar:) + @scalar = scalar + end + + def exec_params(sql, params) + @sql_used = sql + @params_used = params + FakeResult.new(@scalar) + end + + class FakeResult + def initialize(value) + @value = value + end + + def getvalue(_row, _col) + @value + end + end + end + + def test_subscribe_issues_two_arg_sql_and_returns_integer + conn = FakeConn.new(scalar: "1") + client = Pgque::Client.new(conn) + n = client.subscribe("orders", "processor") + assert_equal 1, n + assert_includes conn.sql_used, "pgque.subscribe($1, $2)" + assert_equal ["orders", "processor"], conn.params_used + end + + def test_subscribe_returns_zero_when_already_registered + conn = FakeConn.new(scalar: "0") + client = Pgque::Client.new(conn) + assert_equal 0, client.subscribe("orders", "processor") + end + + def test_unsubscribe_issues_two_arg_sql_and_returns_integer + conn = FakeConn.new(scalar: "1") + client = Pgque::Client.new(conn) + n = client.unsubscribe("orders", "processor") + assert_equal 1, n + assert_includes conn.sql_used, "pgque.unsubscribe($1, $2)" + assert_equal ["orders", "processor"], conn.params_used + end + + def test_unsubscribe_returns_zero_when_not_subscribed + conn = FakeConn.new(scalar: "0") + client = Pgque::Client.new(conn) + assert_equal 0, client.unsubscribe("orders", "processor") + end +end From 54f853919fd081335dcca903ea56698379cbbff5 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Tue, 26 May 2026 06:23:57 +0400 Subject: [PATCH 39/40] chore(ruby): bump gem version to 0.3.0.rc.1 Ruby client was deferred from the 0.2.0 release line to 0.3.0, so the gem's first published version moves to 0.3.0.rc.1. Updates the version constant plus the user-facing install snippets in the top-level README, the Ruby README, and RELEASE.md (bootstrap and verify-install lines). The format-syntax example in RELEASE.md's Versioning section keeps its illustrative 0.2.0.* suffixes since it is not a current-version claim. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 2 +- clients/ruby/README.md | 6 +++--- clients/ruby/RELEASE.md | 6 +++--- clients/ruby/lib/pgque/version.rb | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 2aaac2ae..bdf40d82 100644 --- a/README.md +++ b/README.md @@ -413,7 +413,7 @@ try { ### Ruby (`pgque`) — pg gem ```bash -gem install pgque --pre # or pin: gem "pgque", "0.2.0.rc.1" +gem install pgque --pre # or pin: gem "pgque", "0.3.0.rc.1" ``` ```ruby diff --git a/clients/ruby/README.md b/clients/ruby/README.md index 25e53d64..74b9dc48 100644 --- a/clients/ruby/README.md +++ b/clients/ruby/README.md @@ -11,11 +11,11 @@ universal PostgreSQL queue. Thin wrapper over `pgque-api` SQL functions: gem install pgque --pre ``` -`--pre` is required while v0.2.0 is in release-candidate; the latest -published version is `0.2.0.rc.1`. Pin the exact version if you prefer: +`--pre` is required while v0.3.0 is in release-candidate; the latest +published version is `0.3.0.rc.1`. Pin the exact version if you prefer: ```ruby -gem "pgque", "0.2.0.rc.1" +gem "pgque", "0.3.0.rc.1" ``` Requires Ruby 3.1+ and PostgreSQL 14+ with the PgQue schema installed diff --git a/clients/ruby/RELEASE.md b/clients/ruby/RELEASE.md index fbebc483..204e6b0f 100644 --- a/clients/ruby/RELEASE.md +++ b/clients/ruby/RELEASE.md @@ -3,7 +3,7 @@ Gem name: `pgque` on RubyGems.org. ```bash -gem install pgque --pre # while v0.2.0 is in release-candidate +gem install pgque --pre # while v0.3.0 is in release-candidate ``` ```ruby @@ -33,7 +33,7 @@ first release is therefore manual: cd clients/ruby gem build pgque.gemspec gem signin # one-time, prompts for rubygems.org credentials -gem push pgque-0.2.0.rc.1.gem +gem push pgque-0.3.0.rc.1.gem ``` After that, every subsequent release goes through the workflow below. @@ -84,7 +84,7 @@ The release workflow is `.github/workflows/release-ruby.yml`. 7. Verify the published artifact installs in a clean environment: ```bash - gem install pgque --pre # or pin: gem install pgque -v 0.2.0.rc.1 + gem install pgque --pre # or pin: gem install pgque -v 0.3.0.rc.1 ruby -rpgque -e 'puts Pgque::VERSION' ``` diff --git a/clients/ruby/lib/pgque/version.rb b/clients/ruby/lib/pgque/version.rb index 602515a7..9ce00d79 100644 --- a/clients/ruby/lib/pgque/version.rb +++ b/clients/ruby/lib/pgque/version.rb @@ -1,5 +1,5 @@ # Copyright 2026 Nikolay Samokhvalov. Apache-2.0 license. module Pgque - VERSION = "0.2.0.rc.1" + VERSION = "0.3.0.rc.1" end From 35d884f3e4d391e336db95922e1d1b0ec4c02dd9 Mon Sep 17 00:00:00 2001 From: Dalto Curvelano Jr Date: Tue, 26 May 2026 06:55:10 +0400 Subject: [PATCH 40/40] docs(ruby): re-sync README with the Python README The Ruby README had drifted: the intro listed only 5 of the 10 client methods, the permissions section omitted subscribe/unsubscribe from pgque_reader, and the quickstart still set up subscriptions via raw SQL (client.conn.exec) instead of the first-class wrapper. Several sections present in clients/python/README.md were missing entirely (consumer options, unknown-handler policy, cooperative consumers, manual ticking, transactions / snapshot rule, distribution note). This commit mirrors the Python README's structure section-by-section and rewrites each in Ruby idiom. Notable Ruby-specific differences kept from the prior version: the "Pgque::Client#send shadows Object#send" footgun note, and a transactions section that reflects Ruby pg's per-statement autocommit semantics (group with conn.transaction { ... } rather than commit between phases). No code changes; documentation only. Co-Authored-By: Claude Opus 4.7 (1M context) --- clients/ruby/README.md | 190 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 177 insertions(+), 13 deletions(-) diff --git a/clients/ruby/README.md b/clients/ruby/README.md index 74b9dc48..74b9cd61 100644 --- a/clients/ruby/README.md +++ b/clients/ruby/README.md @@ -2,7 +2,8 @@ Ruby client for [PgQue](https://github.com/NikolayS/pgque) — the PgQ-based universal PostgreSQL queue. Thin wrapper over `pgque-api` SQL functions: -`send`, `receive`, `ack`, `nack`, `force_next_tick`, plus a polling +`send`, `send_batch`, `subscribe`, `unsubscribe`, `receive`, `ack`, +`nack`, `ticker`, `ticker_all`, `force_next_tick`, plus a polling `Consumer` with `LISTEN`/`NOTIFY` wakeup. ## Install @@ -24,41 +25,54 @@ Requires Ruby 3.1+ and PostgreSQL 14+ with the PgQue schema installed ## Database permissions The connecting database role needs `pgque_reader` to consume (`receive`, -`ack`, `nack`) and `pgque_writer` to produce (`send`, `send_batch`). The -two are **siblings** — neither inherits the other. An app that both -produces and consumes must be granted **both** roles: +`ack`, `nack`, `subscribe`, `unsubscribe`) and `pgque_writer` to produce +(`send`, `send_batch`). The two are **siblings** — neither inherits the +other. An app that both produces and consumes (the typical case for code +using this client) must be granted **both** roles: ```sql grant pgque_reader to your_app_user; grant pgque_writer to your_app_user; ``` -See [`docs/reference.md` — Roles and grants](../../docs/reference.md#roles-and-grants). +See [`docs/reference.md` — Roles and grants](../../docs/reference.md#roles-and-grants) +for the full role table. ## Quickstart -Run the one-time setup once (typically in a migration), then produce -and consume from any process: - ```ruby require "pgque" Pgque.connect("postgresql://localhost/mydb") do |client| - # one-time setup + # one-time setup (typically in a migration) client.conn.exec("select pgque.create_queue('orders')") - client.conn.exec("select pgque.subscribe('orders', 'order_worker')") + client.subscribe("orders", "order_worker") - # produce - client.send("orders", { "order_id" => 42 }, type: "order.created") + # producer + event_id = client.send("orders", { "order_id" => 42 }, type: "order.created") + batch_ids = client.send_batch("orders", "order.created", [ + { "order_id" => 43 }, + { "order_id" => 44 }, + ]) + puts "#{event_id} #{batch_ids.inspect}" end -# consume (separate process) +# consumer (separate process / thread) consumer = Pgque::Consumer.new( "postgresql://localhost/mydb", queue: "orders", name: "order_worker", ) + consumer.on("order.created") { |msg| process_order(msg.payload) } + +# Optional: catch-all handler for types with no specific handler. +# Without it, messages with unhandled types are nacked by default +# (sent to retry_queue, or to the dead-letter queue once +# queue_max_retries is exhausted). Register a "*" handler to take +# explicit control. +consumer.on("*") { |msg| log_unhandled(msg.type, msg.payload) } + consumer.start # blocks until SIGTERM / SIGINT ``` @@ -69,6 +83,138 @@ ticking from your application or an external scheduler — see the project [Installation](https://github.com/NikolayS/pgque#installation) section for both paths. +### Consumer options + +`Consumer.new(..., max_messages: ...)` controls the per-`receive` limit. +The default is PostgreSQL's `int` maximum, so the consumer requests +the whole PgQ batch before acknowledging it. `ack` finishes the +entire underlying PgQ batch, including rows beyond `max_messages`; +only lower this value when it is at least as large as the queue's +worst-case batch size, otherwise rows past the limit are silently +skipped by the batch ack. + +Other options: `poll_interval:` (seconds between polls when no +`LISTEN`/`NOTIFY` arrives, default 30), `retry_after:` (seconds before +nacked messages are retried, default 60), and `logger:` (a `Logger` +instance; the default targets `$stderr` at `FATAL`, so the consumer is +effectively silent unless you set `PGQUE_LOG_LEVEL=warn` or pass your +own). + +### Handling unknown event types + +By default the consumer **nacks** any message whose type has no +registered handler and no `"*"` catch-all. The message is retried (or +dead-lettered once `queue_max_retries` is exhausted) so unknown types +are never silently dropped. + +To ack unknown types instead, pass `unknown_handler_policy: "ack"`: + +```ruby +consumer = Pgque::Consumer.new( + "postgresql://localhost/mydb", + queue: "orders", + name: "order_worker", + unknown_handler_policy: "ack", # log WARNING and ack; do not nack +) +``` + +## Experimental: cooperative consumers + +> **Experimental in PgQue 0.2.** Function names, edge-case behavior, and +> client API shape may change before this feature is marked stable. Do +> not use this as the only processing path for critical workloads +> without idempotent handlers and stale-worker takeover tests. + +Cooperative consumers let several worker processes share **one logical +consumer**. Each batch is handed to exactly one subconsumer; the main +row owns the group cursor, member rows own active batches. See +[`docs/reference.md` — Cooperative consumers / subconsumers](../../docs/reference.md#cooperative-consumers--subconsumers) +for the SQL surface. + +Two-worker example (each worker holds its own connection / process): + +```ruby +require "pgque" + +# worker-1 +c1 = Pgque::Consumer.new( + "postgresql://localhost/mydb", + queue: "orders", + name: "order_worker", + subconsumer: "worker-1", + dead_interval: "5 minutes", # optional: take over a stale sibling +) + +c1.on("order.created") { |msg| process(msg) } + +c1.start # in a second process: subconsumer: "worker-2" +``` + +`Consumer.new(subconsumer: ...)` switches the poll loop to +`receive_coop` and uses the cooperative cursor. `dead_interval:` is +only valid in cooperative mode; passing it without `subconsumer:` +raises `ArgumentError`. + +The low-level methods on `Pgque::Client` are also available for direct +use: + +```ruby +client.subscribe_subconsumer("orders", "order_worker", "worker-1") +msgs = client.receive_coop( + "orders", "order_worker", "worker-1", + max_messages: 100, dead_interval: "5 minutes", +) +client.ack(msgs[0].batch_id) +client.touch_subconsumer("orders", "order_worker", "worker-1") +client.unsubscribe_subconsumer( + "orders", "order_worker", "worker-1", batch_handling: 1, +) +``` + +`unsubscribe_subconsumer(..., batch_handling: 0)` (the default) raises +if the subconsumer holds an active batch; pass `batch_handling: 1` to +route active messages through retry/DLQ before removal. + +## Manual ticking + +For tests, demos, or manual operation without `pg_cron`, use +`client.force_next_tick(queue)` to force the **next** ticker call to +materialize a tick. It does not insert the tick itself: + +```ruby +client.force_next_tick("orders") +client.ticker("orders") +``` + +`client.ticker_all` runs the global ticker across all eligible queues +and returns the number of queues that received a tick. + +## Transactions + +`send` → ticker → `receive` must each run in its own committed +transaction (PgQue is snapshot-based). Ruby's `pg` gem runs each +statement in its own implicit transaction by default — the equivalent +of psycopg's `autocommit=True` — so the snippets above already commit +between phases without any explicit `BEGIN`/`COMMIT`. + +To group several statements into one transaction (for example, to +publish a batch atomically with surrounding bookkeeping), wrap them in +a `transaction` block on the underlying `PG::Connection`: + +```ruby +client.conn.transaction do + client.send("orders", { "order_id" => 42 }) + bookkeeping(client.conn) +end # commits here; raises rollback the whole block +``` + +Don't wrap `send` and `receive` in one explicit transaction; same for +`maint_retry_events` + `ticker`. See the +[snapshot rule](https://github.com/NikolayS/pgque/blob/main/docs/pgq-concepts.md#snapshot-rule). +The built-in `Pgque::Consumer` already wraps `receive` + dispatch + +`ack` in a single `conn.transaction` per poll, so handler code does +not need to manage that. + ## A note on `Pgque::Client#send` The producer method is called `send` to mirror the SQL surface @@ -101,6 +247,24 @@ PGQUE_TEST_DSN=postgresql://postgres:pgque_test@localhost/pgque_test \ Without `PGQUE_TEST_DSN`, the tests skip. +## Distribution + +The RubyGems distribution is `pgque`; require it as: + +```ruby +require "pgque" +``` + +See [RELEASE.md](RELEASE.md) for publishing steps. + +## More + +- Schema install, full reference, tutorial: + +- Per-function SQL reference: + +- Issues: + ## License Apache-2.0. Copyright 2026 Nikolay Samokhvalov.