From 6f36adfadfcc51f8e1b92f8cd5fceb0289d73ccc Mon Sep 17 00:00:00 2001 From: Jyothi Kiran Thammana <147131742+jyothikirant-sayukth@users.noreply.github.com> Date: Mon, 23 Dec 2024 01:39:35 +0530 Subject: [PATCH 001/113] Update pg_long_running_transactions.go (#1092) To extract time in seconds for pg_long_running_transactions_oldest_timestamp_seconds query which currently return epoch time. Signed-off-by: Jyothi Kiran Thammana <147131742+jyothikirant-sayukth@users.noreply.github.com> --- collector/pg_long_running_transactions.go | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/collector/pg_long_running_transactions.go b/collector/pg_long_running_transactions.go index 846feaeed..d7d1e6d30 100644 --- a/collector/pg_long_running_transactions.go +++ b/collector/pg_long_running_transactions.go @@ -50,11 +50,13 @@ var ( ) longRunningTransactionsQuery = ` - SELECT - COUNT(*) as transactions, - MAX(EXTRACT(EPOCH FROM clock_timestamp())) AS oldest_timestamp_seconds - FROM pg_catalog.pg_stat_activity - WHERE state is distinct from 'idle' AND query not like 'autovacuum:%' + SELECT + COUNT(*) as transactions, + MAX(EXTRACT(EPOCH FROM clock_timestamp() - pg_stat_activity.xact_start)) AS oldest_timestamp_seconds +FROM pg_catalog.pg_stat_activity +WHERE state IS DISTINCT FROM 'idle' +AND query NOT LIKE 'autovacuum:%' +AND pg_stat_activity.xact_start IS NOT NULL; ` ) From 5bb170232162b2d9232d1e1d7340eeee1cafdf41 Mon Sep 17 00:00:00 2001 From: aagarwalla-fx Date: Mon, 23 Dec 2024 02:44:19 +0530 Subject: [PATCH 002/113] Fix to replace dashes with underscore in the metric names (#1103) * Fix to replace dashes with underscore in the metric names Signed-off-by: aagarwalla-fx * Code style fix Signed-off-by: aagarwalla-fx --------- Signed-off-by: aagarwalla-fx --- cmd/postgres_exporter/pg_setting.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmd/postgres_exporter/pg_setting.go b/cmd/postgres_exporter/pg_setting.go index b02416a7c..02e65dd2a 100644 --- a/cmd/postgres_exporter/pg_setting.go +++ b/cmd/postgres_exporter/pg_setting.go @@ -67,7 +67,7 @@ type pgSetting struct { func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { var ( err error - name = strings.Replace(s.name, ".", "_", -1) + name = strings.Replace(strings.Replace(s.name, ".", "_", -1), "-", "_", -1) unit = s.unit // nolint: ineffassign shortDesc = fmt.Sprintf("Server Parameter: %s", s.name) subsystem = "settings" From 5145620988a075c8f4c08ab045145510e3ddf1df Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 1 Jan 2025 18:36:32 +0100 Subject: [PATCH 003/113] Bump github.com/prometheus/exporter-toolkit from 0.13.1 to 0.13.2 (#1108) Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.13.1 to 0.13.2. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Changelog](https://github.com/prometheus/exporter-toolkit/blob/master/CHANGELOG.md) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.13.1...v0.13.2) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 18 +++++++++--------- go.sum | 40 ++++++++++++++++++++-------------------- 2 files changed, 29 insertions(+), 29 deletions(-) diff --git a/go.mod b/go.mod index bc854c3aa..9cdf3cbec 100644 --- a/go.mod +++ b/go.mod @@ -9,8 +9,8 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.20.5 github.com/prometheus/client_model v0.6.1 - github.com/prometheus/common v0.60.1 - github.com/prometheus/exporter-toolkit v0.13.1 + github.com/prometheus/common v0.61.0 + github.com/prometheus/exporter-toolkit v0.13.2 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -36,11 +36,11 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.28.0 // indirect - golang.org/x/net v0.29.0 // indirect - golang.org/x/oauth2 v0.23.0 // indirect - golang.org/x/sync v0.8.0 // indirect - golang.org/x/sys v0.26.0 // indirect - golang.org/x/text v0.19.0 // indirect - google.golang.org/protobuf v1.34.2 // indirect + golang.org/x/crypto v0.31.0 // indirect + golang.org/x/net v0.32.0 // indirect + golang.org/x/oauth2 v0.24.0 // indirect + golang.org/x/sync v0.10.0 // indirect + golang.org/x/sys v0.28.0 // indirect + golang.org/x/text v0.21.0 // indirect + google.golang.org/protobuf v1.35.2 // indirect ) diff --git a/go.sum b/go.sum index 044805aa2..e20b36f15 100644 --- a/go.sum +++ b/go.sum @@ -54,10 +54,10 @@ github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+ github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= -github.com/prometheus/common v0.60.1 h1:FUas6GcOw66yB/73KC+BOZoFJmbo/1pojoILArPAaSc= -github.com/prometheus/common v0.60.1/go.mod h1:h0LYf1R1deLSKtD4Vdg8gy4RuOvENW2J/h19V5NADQw= -github.com/prometheus/exporter-toolkit v0.13.1 h1:Evsh0gWQo2bdOHlnz9+0Nm7/OFfIwhE2Ws4A2jIlR04= -github.com/prometheus/exporter-toolkit v0.13.1/go.mod h1:ujdv2YIOxtdFxxqtloLpbqmxd5J0Le6IITUvIRSWjj0= +github.com/prometheus/common v0.61.0 h1:3gv/GThfX0cV2lpO7gkTUwZru38mxevy90Bj8YFSRQQ= +github.com/prometheus/common v0.61.0/go.mod h1:zr29OCN/2BsJRaFwG8QOBr41D6kkchKbpeNH7pAjb/s= +github.com/prometheus/exporter-toolkit v0.13.2 h1:Z02fYtbqTMy2i/f+xZ+UK5jy/bl1Ex3ndzh06T/Q9DQ= +github.com/prometheus/exporter-toolkit v0.13.2/go.mod h1:tCqnfx21q6qN1KA4U3Bfb8uWzXfijIrJz3/kTIqMV7g= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -69,24 +69,24 @@ github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sS github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= -github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw= -golang.org/x/crypto v0.28.0/go.mod h1:rmgy+3RHxRZMyY0jjAJShp2zgEdOqj2AO7U0pYmeQ7U= -golang.org/x/net v0.29.0 h1:5ORfpBpCs4HzDYoodCDBbwHzdR5UrLBZ3sOnUJmFoHo= -golang.org/x/net v0.29.0/go.mod h1:gLkgy8jTGERgjzMic6DS9+SP0ajcu6Xu3Orq/SpETg0= -golang.org/x/oauth2 v0.23.0 h1:PbgcYx2W7i4LvjJWEbf0ngHV6qJYr86PkAV3bXdLEbs= -golang.org/x/oauth2 v0.23.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= -golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ= -golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo= -golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM= -golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= -google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg= -google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= +golang.org/x/net v0.32.0 h1:ZqPmj8Kzc+Y6e0+skZsuACbx+wzMgo5MQsJh9Qd6aYI= +golang.org/x/net v0.32.0/go.mod h1:CwU0IoeOlnQQWJ6ioyFrfRuomB8GKF6KbYXZVyeXNfs= +golang.org/x/oauth2 v0.24.0 h1:KTBBxWqUa0ykRPLtV69rRto9TLXcqYkeswu48x/gvNE= +golang.org/x/oauth2 v0.24.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= +golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= +google.golang.org/protobuf v1.35.2 h1:8Ar7bF+apOIoThw1EdZl0p1oWvMqTHmpA2fRTyZO8io= +google.golang.org/protobuf v1.35.2/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From bea260951948d8fbf5fdd4059cce9f385aaced36 Mon Sep 17 00:00:00 2001 From: Nicolas Rodriguez Date: Wed, 1 Jan 2025 22:03:43 +0100 Subject: [PATCH 004/113] Checkpoint related columns in PG 17 have been moved from pg_stat_bgwriter to pg_stat_checkpointer (#1072) * Checkpoint related columns in PG 17 have been moved from pg_stat_bgwriter to pg_stat_checkpointer Fix https://github.com/prometheus-community/postgres_exporter/issues/1060 See: https://www.dbi-services.com/blog/postgresql-17-new-catalog-view-pg_stat_checkpointer/ Signed-off-by: Nicolas Rodriguez * Add support for pg_stat_checkpointer See: https://www.dbi-services.com/blog/postgresql-17-new-catalog-view-pg_stat_checkpointer/ Signed-off-by: Nicolas Rodriguez * Run integration tests with Postgres 17 Signed-off-by: Nicolas Rodriguez * Update date in file header Signed-off-by: Nicolas Rodriguez --------- Signed-off-by: Nicolas Rodriguez --- .circleci/config.yml | 1 + collector/pg_stat_bgwriter.go | 275 +++++++++++++++---------- collector/pg_stat_bgwriter_test.go | 4 +- collector/pg_stat_checkpointer.go | 221 ++++++++++++++++++++ collector/pg_stat_checkpointer_test.go | 143 +++++++++++++ 5 files changed, 533 insertions(+), 111 deletions(-) create mode 100644 collector/pg_stat_checkpointer.go create mode 100644 collector/pg_stat_checkpointer_test.go diff --git a/.circleci/config.yml b/.circleci/config.yml index d745e30f2..6cfc880d2 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -63,6 +63,7 @@ workflows: - cimg/postgres:14.9 - cimg/postgres:15.4 - cimg/postgres:16.0 + - cimg/postgres:17.0 - prometheus/build: name: build parallelism: 3 diff --git a/collector/pg_stat_bgwriter.go b/collector/pg_stat_bgwriter.go index ec446d58c..6e3bd09cb 100644 --- a/collector/pg_stat_bgwriter.go +++ b/collector/pg_stat_bgwriter.go @@ -17,6 +17,7 @@ import ( "context" "database/sql" + "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" ) @@ -101,7 +102,7 @@ var ( prometheus.Labels{}, ) - statBGWriterQuery = `SELECT + statBGWriterQueryBefore17 = `SELECT checkpoints_timed ,checkpoints_req ,checkpoint_write_time @@ -114,121 +115,177 @@ var ( ,buffers_alloc ,stats_reset FROM pg_stat_bgwriter;` + + statBGWriterQueryAfter17 = `SELECT + buffers_clean + ,maxwritten_clean + ,buffers_alloc + ,stats_reset + FROM pg_stat_bgwriter;` ) func (PGStatBGWriterCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { - db := instance.getDB() - row := db.QueryRowContext(ctx, - statBGWriterQuery) + if instance.version.GE(semver.MustParse("17.0.0")) { + db := instance.getDB() + row := db.QueryRowContext(ctx, statBGWriterQueryAfter17) - var cpt, cpr, bcp, bc, mwc, bb, bbf, ba sql.NullInt64 - var cpwt, cpst sql.NullFloat64 - var sr sql.NullTime + var bc, mwc, ba sql.NullInt64 + var sr sql.NullTime - err := row.Scan(&cpt, &cpr, &cpwt, &cpst, &bcp, &bc, &mwc, &bb, &bbf, &ba, &sr) - if err != nil { - return err - } + err := row.Scan(&bc, &mwc, &ba, &sr) + if err != nil { + return err + } - cptMetric := 0.0 - if cpt.Valid { - cptMetric = float64(cpt.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterCheckpointsTimedDesc, - prometheus.CounterValue, - cptMetric, - ) - cprMetric := 0.0 - if cpr.Valid { - cprMetric = float64(cpr.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterCheckpointsReqDesc, - prometheus.CounterValue, - cprMetric, - ) - cpwtMetric := 0.0 - if cpwt.Valid { - cpwtMetric = float64(cpwt.Float64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterCheckpointsReqTimeDesc, - prometheus.CounterValue, - cpwtMetric, - ) - cpstMetric := 0.0 - if cpst.Valid { - cpstMetric = float64(cpst.Float64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterCheckpointsSyncTimeDesc, - prometheus.CounterValue, - cpstMetric, - ) - bcpMetric := 0.0 - if bcp.Valid { - bcpMetric = float64(bcp.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterBuffersCheckpointDesc, - prometheus.CounterValue, - bcpMetric, - ) - bcMetric := 0.0 - if bc.Valid { - bcMetric = float64(bc.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterBuffersCleanDesc, - prometheus.CounterValue, - bcMetric, - ) - mwcMetric := 0.0 - if mwc.Valid { - mwcMetric = float64(mwc.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterMaxwrittenCleanDesc, - prometheus.CounterValue, - mwcMetric, - ) - bbMetric := 0.0 - if bb.Valid { - bbMetric = float64(bb.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterBuffersBackendDesc, - prometheus.CounterValue, - bbMetric, - ) - bbfMetric := 0.0 - if bbf.Valid { - bbfMetric = float64(bbf.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterBuffersBackendFsyncDesc, - prometheus.CounterValue, - bbfMetric, - ) - baMetric := 0.0 - if ba.Valid { - baMetric = float64(ba.Int64) - } - ch <- prometheus.MustNewConstMetric( - statBGWriterBuffersAllocDesc, - prometheus.CounterValue, - baMetric, - ) - srMetric := 0.0 - if sr.Valid { - srMetric = float64(sr.Time.Unix()) + bcMetric := 0.0 + if bc.Valid { + bcMetric = float64(bc.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersCleanDesc, + prometheus.CounterValue, + bcMetric, + ) + mwcMetric := 0.0 + if mwc.Valid { + mwcMetric = float64(mwc.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterMaxwrittenCleanDesc, + prometheus.CounterValue, + mwcMetric, + ) + baMetric := 0.0 + if ba.Valid { + baMetric = float64(ba.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersAllocDesc, + prometheus.CounterValue, + baMetric, + ) + srMetric := 0.0 + if sr.Valid { + srMetric = float64(sr.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterStatsResetDesc, + prometheus.CounterValue, + srMetric, + ) + } else { + db := instance.getDB() + row := db.QueryRowContext(ctx, statBGWriterQueryBefore17) + + var cpt, cpr, bcp, bc, mwc, bb, bbf, ba sql.NullInt64 + var cpwt, cpst sql.NullFloat64 + var sr sql.NullTime + + err := row.Scan(&cpt, &cpr, &cpwt, &cpst, &bcp, &bc, &mwc, &bb, &bbf, &ba, &sr) + if err != nil { + return err + } + + cptMetric := 0.0 + if cpt.Valid { + cptMetric = float64(cpt.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterCheckpointsTimedDesc, + prometheus.CounterValue, + cptMetric, + ) + cprMetric := 0.0 + if cpr.Valid { + cprMetric = float64(cpr.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterCheckpointsReqDesc, + prometheus.CounterValue, + cprMetric, + ) + cpwtMetric := 0.0 + if cpwt.Valid { + cpwtMetric = float64(cpwt.Float64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterCheckpointsReqTimeDesc, + prometheus.CounterValue, + cpwtMetric, + ) + cpstMetric := 0.0 + if cpst.Valid { + cpstMetric = float64(cpst.Float64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterCheckpointsSyncTimeDesc, + prometheus.CounterValue, + cpstMetric, + ) + bcpMetric := 0.0 + if bcp.Valid { + bcpMetric = float64(bcp.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersCheckpointDesc, + prometheus.CounterValue, + bcpMetric, + ) + bcMetric := 0.0 + if bc.Valid { + bcMetric = float64(bc.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersCleanDesc, + prometheus.CounterValue, + bcMetric, + ) + mwcMetric := 0.0 + if mwc.Valid { + mwcMetric = float64(mwc.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterMaxwrittenCleanDesc, + prometheus.CounterValue, + mwcMetric, + ) + bbMetric := 0.0 + if bb.Valid { + bbMetric = float64(bb.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersBackendDesc, + prometheus.CounterValue, + bbMetric, + ) + bbfMetric := 0.0 + if bbf.Valid { + bbfMetric = float64(bbf.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersBackendFsyncDesc, + prometheus.CounterValue, + bbfMetric, + ) + baMetric := 0.0 + if ba.Valid { + baMetric = float64(ba.Int64) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterBuffersAllocDesc, + prometheus.CounterValue, + baMetric, + ) + srMetric := 0.0 + if sr.Valid { + srMetric = float64(sr.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statBGWriterStatsResetDesc, + prometheus.CounterValue, + srMetric, + ) } - ch <- prometheus.MustNewConstMetric( - statBGWriterStatsResetDesc, - prometheus.CounterValue, - srMetric, - ) return nil } diff --git a/collector/pg_stat_bgwriter_test.go b/collector/pg_stat_bgwriter_test.go index 1c2cf98de..6fde2fb6a 100644 --- a/collector/pg_stat_bgwriter_test.go +++ b/collector/pg_stat_bgwriter_test.go @@ -52,7 +52,7 @@ func TestPGStatBGWriterCollector(t *testing.T) { rows := sqlmock.NewRows(columns). AddRow(354, 4945, 289097744, 1242257, int64(3275602074), 89320867, 450139, 2034563757, 0, int64(2725688749), srT) - mock.ExpectQuery(sanitizeQuery(statBGWriterQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(statBGWriterQueryBefore17)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -113,7 +113,7 @@ func TestPGStatBGWriterCollectorNullValues(t *testing.T) { rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(statBGWriterQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(statBGWriterQueryBefore17)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { diff --git a/collector/pg_stat_checkpointer.go b/collector/pg_stat_checkpointer.go new file mode 100644 index 000000000..284cf650c --- /dev/null +++ b/collector/pg_stat_checkpointer.go @@ -0,0 +1,221 @@ +// Copyright 2024 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/prometheus/client_golang/prometheus" +) + +const statCheckpointerSubsystem = "stat_checkpointer" + +func init() { + // WARNING: + // Disabled by default because this set of metrics is only available from Postgres 17 + registerCollector(statCheckpointerSubsystem, defaultDisabled, NewPGStatCheckpointerCollector) +} + +type PGStatCheckpointerCollector struct { +} + +func NewPGStatCheckpointerCollector(collectorConfig) (Collector, error) { + return &PGStatCheckpointerCollector{}, nil +} + +var ( + statCheckpointerNumTimedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "num_timed_total"), + "Number of scheduled checkpoints due to timeout", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerNumRequestedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "num_requested_total"), + "Number of requested checkpoints that have been performed", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerRestartpointsTimedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "restartpoints_timed_total"), + "Number of scheduled restartpoints due to timeout or after a failed attempt to perform it", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerRestartpointsReqDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "restartpoints_req_total"), + "Number of requested restartpoints", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerRestartpointsDoneDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "restartpoints_done_total"), + "Number of restartpoints that have been performed", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerWriteTimeDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "write_time_total"), + "Total amount of time that has been spent in the portion of processing checkpoints and restartpoints where files are written to disk, in milliseconds", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerSyncTimeDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "sync_time_total"), + "Total amount of time that has been spent in the portion of processing checkpoints and restartpoints where files are synchronized to disk, in milliseconds", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerBuffersWrittenDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "buffers_written_total"), + "Number of buffers written during checkpoints and restartpoints", + []string{}, + prometheus.Labels{}, + ) + statCheckpointerStatsResetDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statCheckpointerSubsystem, "stats_reset_total"), + "Time at which these statistics were last reset", + []string{}, + prometheus.Labels{}, + ) + + statCheckpointerQuery = `SELECT + num_timed + ,num_requested + ,restartpoints_timed + ,restartpoints_req + ,restartpoints_done + ,write_time + ,sync_time + ,buffers_written + ,stats_reset + FROM pg_stat_checkpointer;` +) + +func (PGStatCheckpointerCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + row := db.QueryRowContext(ctx, statCheckpointerQuery) + + // num_timed = nt = bigint + // num_requested = nr = bigint + // restartpoints_timed = rpt = bigint + // restartpoints_req = rpr = bigint + // restartpoints_done = rpd = bigint + // write_time = wt = double precision + // sync_time = st = double precision + // buffers_written = bw = bigint + // stats_reset = sr = timestamp + + var nt, nr, rpt, rpr, rpd, bw sql.NullInt64 + var wt, st sql.NullFloat64 + var sr sql.NullTime + + err := row.Scan(&nt, &nr, &rpt, &rpr, &rpd, &wt, &st, &bw, &sr) + if err != nil { + return err + } + + ntMetric := 0.0 + if nt.Valid { + ntMetric = float64(nt.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerNumTimedDesc, + prometheus.CounterValue, + ntMetric, + ) + + nrMetric := 0.0 + if nr.Valid { + nrMetric = float64(nr.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerNumRequestedDesc, + prometheus.CounterValue, + nrMetric, + ) + + rptMetric := 0.0 + if rpt.Valid { + rptMetric = float64(rpt.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerRestartpointsTimedDesc, + prometheus.CounterValue, + rptMetric, + ) + + rprMetric := 0.0 + if rpr.Valid { + rprMetric = float64(rpr.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerRestartpointsReqDesc, + prometheus.CounterValue, + rprMetric, + ) + + rpdMetric := 0.0 + if rpd.Valid { + rpdMetric = float64(rpd.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerRestartpointsDoneDesc, + prometheus.CounterValue, + rpdMetric, + ) + + wtMetric := 0.0 + if wt.Valid { + wtMetric = float64(wt.Float64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerWriteTimeDesc, + prometheus.CounterValue, + wtMetric, + ) + + stMetric := 0.0 + if st.Valid { + stMetric = float64(st.Float64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerSyncTimeDesc, + prometheus.CounterValue, + stMetric, + ) + + bwMetric := 0.0 + if bw.Valid { + bwMetric = float64(bw.Int64) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerBuffersWrittenDesc, + prometheus.CounterValue, + bwMetric, + ) + + srMetric := 0.0 + if sr.Valid { + srMetric = float64(sr.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statCheckpointerStatsResetDesc, + prometheus.CounterValue, + srMetric, + ) + + return nil +} diff --git a/collector/pg_stat_checkpointer_test.go b/collector/pg_stat_checkpointer_test.go new file mode 100644 index 000000000..2f59343bb --- /dev/null +++ b/collector/pg_stat_checkpointer_test.go @@ -0,0 +1,143 @@ +// Copyright 2024 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + "time" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatCheckpointerCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "num_timed", + "num_requested", + "restartpoints_timed", + "restartpoints_req", + "restartpoints_done", + "write_time", + "sync_time", + "buffers_written", + "stats_reset"} + + srT, err := time.Parse("2006-01-02 15:04:05.00000-07", "2023-05-25 17:10:42.81132-07") + if err != nil { + t.Fatalf("Error parsing time: %s", err) + } + + rows := sqlmock.NewRows(columns). + AddRow(354, 4945, 289097744, 1242257, int64(3275602074), 89320867, 450139, 2034563757, srT) + mock.ExpectQuery(sanitizeQuery(statCheckpointerQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatCheckpointerCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatCheckpointerCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 354}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatCheckpointerCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "num_timed", + "num_requested", + "restartpoints_timed", + "restartpoints_req", + "restartpoints_done", + "write_time", + "sync_time", + "buffers_written", + "stats_reset"} + + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statCheckpointerQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatCheckpointerCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatCheckpointerCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From ecb5ec5dff0cf2b3eb3668a7fee30d8ad11c82fa Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 7 Jan 2025 09:28:42 +0100 Subject: [PATCH 005/113] Update common Prometheus files (#1090) Signed-off-by: prombot --- .yamllint | 2 +- Makefile.common | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.yamllint b/.yamllint index 1859cb624..8d09c375f 100644 --- a/.yamllint +++ b/.yamllint @@ -1,7 +1,7 @@ --- extends: default ignore: | - ui/react-app/node_modules + **/node_modules rules: braces: diff --git a/Makefile.common b/Makefile.common index cbb5d8638..09e5bff85 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.60.2 +GOLANGCI_LINT_VERSION ?= v1.61.0 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) From 9de4f19d43427c24947ba7db7d82127141bd5fa4 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 7 Jan 2025 21:20:19 +0100 Subject: [PATCH 006/113] Update common Prometheus files (#1110) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 4 ++-- Makefile.common | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 305146993..01b943b9b 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -26,7 +26,7 @@ jobs: - name: Checkout repository uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - name: Install Go - uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # v5.1.0 + uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: go-version: 1.23.x - name: Install snmp_exporter/generator dependencies @@ -36,4 +36,4 @@ jobs: uses: golangci/golangci-lint-action@971e284b6050e8a5849b72094c50ab08da042db8 # v6.1.1 with: args: --verbose - version: v1.61.0 + version: v1.62.0 diff --git a/Makefile.common b/Makefile.common index 09e5bff85..fc47bdbb2 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.61.0 +GOLANGCI_LINT_VERSION ?= v1.62.0 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) From 7d4c278221e95ddbc62108973e5828a3ffaa2eb8 Mon Sep 17 00:00:00 2001 From: Khiem Doan Date: Mon, 13 Jan 2025 09:24:15 +0700 Subject: [PATCH 007/113] Add Postgres 17 for CI test (#1105) Signed-off-by: Khiem Doan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4c464e210..9494acb24 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Prometheus exporter for PostgreSQL server metrics. -CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16` +CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16`, `17`. ## Quick Start This package is available for Docker: From 3acc4793fc63ffb3088668f285ba658429c8806c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 13 Feb 2025 03:35:38 +0100 Subject: [PATCH 008/113] Bump github.com/prometheus/common from 0.61.0 to 0.62.0 (#1118) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.61.0 to 0.62.0. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md) - [Commits](https://github.com/prometheus/common/compare/v0.61.0...v0.62.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 6 +++--- go.sum | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/go.mod b/go.mod index 9cdf3cbec..b2c9500d0 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.20.5 github.com/prometheus/client_model v0.6.1 - github.com/prometheus/common v0.61.0 + github.com/prometheus/common v0.62.0 github.com/prometheus/exporter-toolkit v0.13.2 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -37,10 +37,10 @@ require ( github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect golang.org/x/crypto v0.31.0 // indirect - golang.org/x/net v0.32.0 // indirect + golang.org/x/net v0.33.0 // indirect golang.org/x/oauth2 v0.24.0 // indirect golang.org/x/sync v0.10.0 // indirect golang.org/x/sys v0.28.0 // indirect golang.org/x/text v0.21.0 // indirect - google.golang.org/protobuf v1.35.2 // indirect + google.golang.org/protobuf v1.36.1 // indirect ) diff --git a/go.sum b/go.sum index e20b36f15..a6dd72407 100644 --- a/go.sum +++ b/go.sum @@ -54,8 +54,8 @@ github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+ github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= -github.com/prometheus/common v0.61.0 h1:3gv/GThfX0cV2lpO7gkTUwZru38mxevy90Bj8YFSRQQ= -github.com/prometheus/common v0.61.0/go.mod h1:zr29OCN/2BsJRaFwG8QOBr41D6kkchKbpeNH7pAjb/s= +github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= +github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I= github.com/prometheus/exporter-toolkit v0.13.2 h1:Z02fYtbqTMy2i/f+xZ+UK5jy/bl1Ex3ndzh06T/Q9DQ= github.com/prometheus/exporter-toolkit v0.13.2/go.mod h1:tCqnfx21q6qN1KA4U3Bfb8uWzXfijIrJz3/kTIqMV7g= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= @@ -75,8 +75,8 @@ github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8 github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= -golang.org/x/net v0.32.0 h1:ZqPmj8Kzc+Y6e0+skZsuACbx+wzMgo5MQsJh9Qd6aYI= -golang.org/x/net v0.32.0/go.mod h1:CwU0IoeOlnQQWJ6ioyFrfRuomB8GKF6KbYXZVyeXNfs= +golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= +golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= golang.org/x/oauth2 v0.24.0 h1:KTBBxWqUa0ykRPLtV69rRto9TLXcqYkeswu48x/gvNE= golang.org/x/oauth2 v0.24.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= @@ -85,8 +85,8 @@ golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= -google.golang.org/protobuf v1.35.2 h1:8Ar7bF+apOIoThw1EdZl0p1oWvMqTHmpA2fRTyZO8io= -google.golang.org/protobuf v1.35.2/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= +google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk= +google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From d85a7710bf084309413986c577ee03d09fb8d2ef Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 14 Feb 2025 09:36:03 +0100 Subject: [PATCH 009/113] Update common Prometheus files (#1124) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 6 +++--- Makefile.common | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 01b943b9b..def9007ac 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -26,14 +26,14 @@ jobs: - name: Checkout repository uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - name: Install Go - uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 + uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 with: go-version: 1.23.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@971e284b6050e8a5849b72094c50ab08da042db8 # v6.1.1 + uses: golangci/golangci-lint-action@ec5d18412c0aeab7936cb16880d708ba2a64e1ae # v6.2.0 with: args: --verbose - version: v1.62.0 + version: v1.63.4 diff --git a/Makefile.common b/Makefile.common index fc47bdbb2..d1576bb31 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.62.0 +GOLANGCI_LINT_VERSION ?= v1.63.4 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) From 072864d179de04327840bc32fa0b1765f250b1f3 Mon Sep 17 00:00:00 2001 From: Nevermind <79126473+NevermindZ4@users.noreply.github.com> Date: Sat, 15 Feb 2025 14:54:12 +0100 Subject: [PATCH 010/113] pg_stat_statements PG17 (#1114) Signed-off-by: Nevermind <79126473+NevermindZ4@users.noreply.github.com> --- collector/pg_stat_statements.go | 30 +++++++++++++++++-- collector/pg_stat_statements_test.go | 43 ++++++++++++++++++++++++++++ 2 files changed, 71 insertions(+), 2 deletions(-) diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index 7926f533e..cabdc3619 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -112,12 +112,38 @@ var ( ) ORDER BY seconds_total DESC LIMIT 100;` + + pgStatStatementsQuery_PG17 = `SELECT + pg_get_userbyid(userid) as user, + pg_database.datname, + pg_stat_statements.queryid, + pg_stat_statements.calls as calls_total, + pg_stat_statements.total_exec_time / 1000.0 as seconds_total, + pg_stat_statements.rows as rows_total, + pg_stat_statements.shared_blk_read_time / 1000.0 as block_read_seconds_total, + pg_stat_statements.shared_blk_write_time / 1000.0 as block_write_seconds_total + FROM pg_stat_statements + JOIN pg_database + ON pg_database.oid = pg_stat_statements.dbid + WHERE + total_exec_time > ( + SELECT percentile_cont(0.1) + WITHIN GROUP (ORDER BY total_exec_time) + FROM pg_stat_statements + ) + ORDER BY seconds_total DESC + LIMIT 100;` ) func (PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { - query := pgStatStatementsQuery - if instance.version.GE(semver.MustParse("13.0.0")) { + var query string + switch { + case instance.version.GE(semver.MustParse("17.0.0")): + query = pgStatStatementsQuery_PG17 + case instance.version.GE(semver.MustParse("13.0.0")): query = pgStatStatementsNewQuery + default: + query = pgStatStatementsQuery } db := instance.getDB() diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go index 08aba34c2..ea035f557 100644 --- a/collector/pg_stat_statements_test.go +++ b/collector/pg_stat_statements_test.go @@ -151,3 +151,46 @@ func TestPGStateStatementsCollectorNewPG(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPGStateStatementsCollector_PG17(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("17.0.0")} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(pgStatStatementsQuery_PG17)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From 9e42fc0145a9d780d824d961afb7b70867558424 Mon Sep 17 00:00:00 2001 From: Michael Todorovic Date: Sat, 15 Feb 2025 15:00:48 +0100 Subject: [PATCH 011/113] fix: handle pg_replication_slots on pg<13 (#1098) * fix: handle pg_replication_slots on pg<13 Signed-off-by: Michael Todorovic * fix: tests Signed-off-by: Michael Todorovic --------- Signed-off-by: Michael Todorovic --- collector/pg_replication_slot.go | 38 ++++++++++++++++++++++++--- collector/pg_replication_slot_test.go | 17 ++++++------ 2 files changed, 44 insertions(+), 11 deletions(-) diff --git a/collector/pg_replication_slot.go b/collector/pg_replication_slot.go index 27ccddefd..e6c9773eb 100644 --- a/collector/pg_replication_slot.go +++ b/collector/pg_replication_slot.go @@ -18,6 +18,7 @@ import ( "database/sql" "log/slog" + "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" ) @@ -81,8 +82,18 @@ var ( "availability of WAL files claimed by this slot", []string{"slot_name", "slot_type", "wal_status"}, nil, ) - pgReplicationSlotQuery = `SELECT + slot_name, + slot_type, + CASE WHEN pg_is_in_recovery() THEN + pg_last_wal_receive_lsn() - '0/0' + ELSE + pg_current_wal_lsn() - '0/0' + END AS current_wal_lsn, + COALESCE(confirmed_flush_lsn, '0/0') - '0/0' AS confirmed_flush_lsn, + active + FROM pg_replication_slots;` + pgReplicationSlotNewQuery = `SELECT slot_name, slot_type, CASE WHEN pg_is_in_recovery() THEN @@ -98,9 +109,15 @@ var ( ) func (PGReplicationSlotCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + query := pgReplicationSlotQuery + abovePG13 := instance.version.GTE(semver.MustParse("13.0.0")) + if abovePG13 { + query = pgReplicationSlotNewQuery + } + db := instance.getDB() rows, err := db.QueryContext(ctx, - pgReplicationSlotQuery) + query) if err != nil { return err } @@ -114,7 +131,22 @@ func (PGReplicationSlotCollector) Update(ctx context.Context, instance *instance var isActive sql.NullBool var safeWalSize sql.NullInt64 var walStatus sql.NullString - if err := rows.Scan(&slotName, &slotType, &walLSN, &flushLSN, &isActive, &safeWalSize, &walStatus); err != nil { + + r := []any{ + &slotName, + &slotType, + &walLSN, + &flushLSN, + &isActive, + } + + if abovePG13 { + r = append(r, &safeWalSize) + r = append(r, &walStatus) + } + + err := rows.Scan(r...) + if err != nil { return err } diff --git a/collector/pg_replication_slot_test.go b/collector/pg_replication_slot_test.go index 174743ac3..981b5db62 100644 --- a/collector/pg_replication_slot_test.go +++ b/collector/pg_replication_slot_test.go @@ -17,6 +17,7 @@ import ( "testing" "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" dto "github.com/prometheus/client_model/go" "github.com/smartystreets/goconvey/convey" @@ -29,12 +30,12 @@ func TestPgReplicationSlotCollectorActive(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("13.3.7")} columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active", "safe_wal_size", "wal_status"} rows := sqlmock.NewRows(columns). AddRow("test_slot", "physical", 5, 3, true, 323906992, "reserved") - mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotNewQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -72,12 +73,12 @@ func TestPgReplicationSlotCollectorInActive(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("13.3.7")} columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active", "safe_wal_size", "wal_status"} rows := sqlmock.NewRows(columns). AddRow("test_slot", "physical", 6, 12, false, -4000, "extended") - mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotNewQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -115,12 +116,12 @@ func TestPgReplicationSlotCollectorActiveNil(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("13.3.7")} columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active", "safe_wal_size", "wal_status"} rows := sqlmock.NewRows(columns). AddRow("test_slot", "physical", 6, 12, nil, nil, "lost") - mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotNewQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -156,12 +157,12 @@ func TestPgReplicationSlotCollectorTestNilValues(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("13.3.7")} columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active", "safe_wal_size", "wal_status"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, true, nil, nil) - mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotNewQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { From 2ee2a8fa7cbd7b07c58a5e40998d6e1f469dd1af Mon Sep 17 00:00:00 2001 From: Felipe Galindo Sanchez Date: Sat, 15 Feb 2025 06:08:24 -0800 Subject: [PATCH 012/113] feat: add wait/backend to pg_stat_activity (#1106) Signed-off-by: Felipe Galindo Sanchez --- cmd/postgres_exporter/postgres_exporter.go | 3 +++ cmd/postgres_exporter/queries.go | 9 ++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/cmd/postgres_exporter/postgres_exporter.go b/cmd/postgres_exporter/postgres_exporter.go index 90f26beb0..d76e2baed 100644 --- a/cmd/postgres_exporter/postgres_exporter.go +++ b/cmd/postgres_exporter/postgres_exporter.go @@ -251,6 +251,9 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ "state": {LABEL, "connection state", nil, semver.MustParseRange(">=9.2.0")}, "usename": {LABEL, "connection usename", nil, nil}, "application_name": {LABEL, "connection application_name", nil, nil}, + "backend_type": {LABEL, "connection backend_type", nil, nil}, + "wait_event_type": {LABEL, "connection wait_event_type", nil, nil}, + "wait_event": {LABEL, "connection wait_event", nil, nil}, "count": {GAUGE, "number of connections in this state", nil, nil}, "max_tx_duration": {GAUGE, "max duration in seconds any active transaction has been running", nil, nil}, }, diff --git a/cmd/postgres_exporter/queries.go b/cmd/postgres_exporter/queries.go index 7090606e1..80be72d54 100644 --- a/cmd/postgres_exporter/queries.go +++ b/cmd/postgres_exporter/queries.go @@ -115,6 +115,9 @@ var queryOverrides = map[string][]OverrideQuery{ tmp.state, tmp2.usename, tmp2.application_name, + tmp2.backend_type, + tmp2.wait_event_type, + tmp2.wait_event, COALESCE(count,0) as count, COALESCE(max_tx_duration,0) as max_tx_duration FROM @@ -133,9 +136,13 @@ var queryOverrides = map[string][]OverrideQuery{ state, usename, application_name, + backend_type, + wait_event_type, + wait_event, count(*) AS count, MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration - FROM pg_stat_activity GROUP BY datname,state,usename,application_name) AS tmp2 + FROM pg_stat_activity + GROUP BY datname,state,usename,application_name,backend_type,wait_event_type,wait_event) AS tmp2 ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname `, }, From c3885e840a1dd8eea2a878c044a92088c1820f0e Mon Sep 17 00:00:00 2001 From: Conrad Hoffmann <1226676+bitfehler@users.noreply.github.com> Date: Sat, 15 Feb 2025 15:15:44 +0100 Subject: [PATCH 013/113] Export last replay age in replication collector (#1085) The exported replication lag does not handle all failure modes, and can report 0 for replicas that are out of sync and incapable of recovery. A proper replacement for that metric would require a different approach (see e.g. #1007), but for a lot of folks, simply exporting the age of the last replay can provide a pretty strong signal for something being amiss. I think this solution might be preferable to #977, though the lag metric needs to be fixed or abandoned eventually. Signed-off-by: Conrad Hoffmann --- collector/pg_replication.go | 19 +++++++++++++++++-- collector/pg_replication_test.go | 5 +++-- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/collector/pg_replication.go b/collector/pg_replication.go index 6067cc9b1..7f8b2fbd7 100644 --- a/collector/pg_replication.go +++ b/collector/pg_replication.go @@ -51,6 +51,15 @@ var ( "Indicates if the server is a replica", []string{}, nil, ) + pgReplicationLastReplay = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSubsystem, + "last_replay_seconds", + ), + "Age of last replay in seconds", + []string{}, nil, + ) pgReplicationQuery = `SELECT CASE @@ -61,7 +70,8 @@ var ( CASE WHEN pg_is_in_recovery() THEN 1 ELSE 0 - END as is_replica` + END as is_replica, + GREATEST (0, EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) as last_replay` ) func (c *PGReplicationCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { @@ -72,7 +82,8 @@ func (c *PGReplicationCollector) Update(ctx context.Context, instance *instance, var lag float64 var isReplica int64 - err := row.Scan(&lag, &isReplica) + var replayAge float64 + err := row.Scan(&lag, &isReplica, &replayAge) if err != nil { return err } @@ -84,5 +95,9 @@ func (c *PGReplicationCollector) Update(ctx context.Context, instance *instance, pgReplicationIsReplica, prometheus.GaugeValue, float64(isReplica), ) + ch <- prometheus.MustNewConstMetric( + pgReplicationLastReplay, + prometheus.GaugeValue, replayAge, + ) return nil } diff --git a/collector/pg_replication_test.go b/collector/pg_replication_test.go index b6df698e3..a48e9fd69 100644 --- a/collector/pg_replication_test.go +++ b/collector/pg_replication_test.go @@ -31,9 +31,9 @@ func TestPgReplicationCollector(t *testing.T) { inst := &instance{db: db} - columns := []string{"lag", "is_replica"} + columns := []string{"lag", "is_replica", "last_replay"} rows := sqlmock.NewRows(columns). - AddRow(1000, 1) + AddRow(1000, 1, 3) mock.ExpectQuery(sanitizeQuery(pgReplicationQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) @@ -49,6 +49,7 @@ func TestPgReplicationCollector(t *testing.T) { expected := []MetricResult{ {labels: labelMap{}, value: 1000, metricType: dto.MetricType_GAUGE}, {labels: labelMap{}, value: 1, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{}, value: 3, metricType: dto.MetricType_GAUGE}, } convey.Convey("Metrics comparison", t, func() { From 99e1b5118caf5094ed69bd7e11cd601e0431bfb5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 15 Feb 2025 10:01:47 -0500 Subject: [PATCH 014/113] Bump github.com/prometheus/exporter-toolkit from 0.13.2 to 0.14.0 (#1126) Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.13.2 to 0.14.0. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Changelog](https://github.com/prometheus/exporter-toolkit/blob/master/CHANGELOG.md) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.13.2...v0.14.0) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 6 +++--- go.sum | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/go.mod b/go.mod index b2c9500d0..a7b376b4e 100644 --- a/go.mod +++ b/go.mod @@ -10,7 +10,7 @@ require ( github.com/prometheus/client_golang v1.20.5 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.62.0 - github.com/prometheus/exporter-toolkit v0.13.2 + github.com/prometheus/exporter-toolkit v0.14.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -36,11 +36,11 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.31.0 // indirect + golang.org/x/crypto v0.32.0 // indirect golang.org/x/net v0.33.0 // indirect golang.org/x/oauth2 v0.24.0 // indirect golang.org/x/sync v0.10.0 // indirect - golang.org/x/sys v0.28.0 // indirect + golang.org/x/sys v0.29.0 // indirect golang.org/x/text v0.21.0 // indirect google.golang.org/protobuf v1.36.1 // indirect ) diff --git a/go.sum b/go.sum index a6dd72407..a57fc83c1 100644 --- a/go.sum +++ b/go.sum @@ -56,8 +56,8 @@ github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I= -github.com/prometheus/exporter-toolkit v0.13.2 h1:Z02fYtbqTMy2i/f+xZ+UK5jy/bl1Ex3ndzh06T/Q9DQ= -github.com/prometheus/exporter-toolkit v0.13.2/go.mod h1:tCqnfx21q6qN1KA4U3Bfb8uWzXfijIrJz3/kTIqMV7g= +github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= +github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -73,16 +73,16 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= -golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= +golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc= +golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc= golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= golang.org/x/oauth2 v0.24.0 h1:KTBBxWqUa0ykRPLtV69rRto9TLXcqYkeswu48x/gvNE= golang.org/x/oauth2 v0.24.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= -golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= +golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk= From 4c170ed56460028f5c02d16a5576250312eecf89 Mon Sep 17 00:00:00 2001 From: Joe Adams Date: Sat, 15 Feb 2025 10:35:04 -0500 Subject: [PATCH 015/113] Fix missing dsn sanitization for logging (#1104) This log line was not sanitized previously which could result in logging sensitive information. I have scanned the rest of the files and I don't see anywhere else that DSN is used in a log line without this filter. Resolves #1042 Signed-off-by: Joe Adams --- cmd/postgres_exporter/postgres_exporter.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmd/postgres_exporter/postgres_exporter.go b/cmd/postgres_exporter/postgres_exporter.go index d76e2baed..a76479611 100644 --- a/cmd/postgres_exporter/postgres_exporter.go +++ b/cmd/postgres_exporter/postgres_exporter.go @@ -681,7 +681,7 @@ func (e *Exporter) scrape(ch chan<- prometheus.Metric) { if err := e.scrapeDSN(ch, dsn); err != nil { errorsCount++ - logger.Error("error scraping dsn", "err", err, "dsn", dsn) + logger.Error("error scraping dsn", "err", err, "dsn", loggableDSN(dsn)) if _, ok := err.(*ErrorConnectToServer); ok { connectionErrorsCount++ From 8bb1a41abf0cfc6baf1bf272a7cf8989d1958dde Mon Sep 17 00:00:00 2001 From: Nicolas Rodriguez Date: Thu, 20 Feb 2025 03:49:11 +0100 Subject: [PATCH 016/113] Skip pg_stat_checkpointer collector if pg<17 (#1112) * fix: skip collector if pg<17 Signed-off-by: Michael Todorovic * fix: better condition Signed-off-by: Michael Todorovic * fix: fix PGStatCheckpointerCollector tests Signed-off-by: Nicolas Rodriguez --------- Signed-off-by: Michael Todorovic Signed-off-by: Nicolas Rodriguez Co-authored-by: Michael Todorovic --- collector/pg_stat_checkpointer.go | 16 +++++++++++++--- collector/pg_stat_checkpointer_test.go | 5 +++-- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/collector/pg_stat_checkpointer.go b/collector/pg_stat_checkpointer.go index 284cf650c..31e9c5d62 100644 --- a/collector/pg_stat_checkpointer.go +++ b/collector/pg_stat_checkpointer.go @@ -16,7 +16,9 @@ package collector import ( "context" "database/sql" + "log/slog" + "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" ) @@ -29,10 +31,11 @@ func init() { } type PGStatCheckpointerCollector struct { + log *slog.Logger } -func NewPGStatCheckpointerCollector(collectorConfig) (Collector, error) { - return &PGStatCheckpointerCollector{}, nil +func NewPGStatCheckpointerCollector(config collectorConfig) (Collector, error) { + return &PGStatCheckpointerCollector{log: config.logger}, nil } var ( @@ -104,8 +107,15 @@ var ( FROM pg_stat_checkpointer;` ) -func (PGStatCheckpointerCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { +func (c PGStatCheckpointerCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { db := instance.getDB() + + before17 := instance.version.LT(semver.MustParse("17.0.0")) + if before17 { + c.log.Warn("pg_stat_checkpointer collector is not available on PostgreSQL < 17.0.0, skipping") + return nil + } + row := db.QueryRowContext(ctx, statCheckpointerQuery) // num_timed = nt = bigint diff --git a/collector/pg_stat_checkpointer_test.go b/collector/pg_stat_checkpointer_test.go index 2f59343bb..9a8dd7f21 100644 --- a/collector/pg_stat_checkpointer_test.go +++ b/collector/pg_stat_checkpointer_test.go @@ -18,6 +18,7 @@ import ( "time" "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" dto "github.com/prometheus/client_model/go" "github.com/smartystreets/goconvey/convey" @@ -30,7 +31,7 @@ func TestPGStatCheckpointerCollector(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("17.0.0")} columns := []string{ "num_timed", @@ -92,7 +93,7 @@ func TestPGStatCheckpointerCollectorNullValues(t *testing.T) { } defer db.Close() - inst := &instance{db: db} + inst := &instance{db: db, version: semver.MustParse("17.0.0")} columns := []string{ "num_timed", From 51006aba2f695d318d52b6f5842f18bb52e54b9f Mon Sep 17 00:00:00 2001 From: Joe Adams Date: Fri, 21 Feb 2025 17:38:46 -0500 Subject: [PATCH 017/113] Prep for v0.17 (#1127) Signed-off-by: Joe Adams --- CHANGELOG.md | 22 ++++++++++++++++++++++ VERSION | 2 +- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 790a109c5..49b6b4101 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,25 @@ +## 0.17.0 / 2025-02-16 + +## What's Changed +* [ENHANCEMENT] Add Postgres 17 for CI test by @khiemdoan in https://github.com/prometheus-community/postgres_exporter/pull/1105 +* [ENHANCEMENT] Add wait/backend to pg_stat_activity by @fgalind1 in https://github.com/prometheus-community/postgres_exporter/pull/1106 +* [ENHANCEMENT] Export last replay age in replication collector by @bitfehler in https://github.com/prometheus-community/postgres_exporter/pull/1085 +* [BUGFIX] Fix pg_long_running_transactions time by @jyothikirant-sayukth in https://github.com/prometheus-community/postgres_exporter/pull/1092 +* [BUGFIX] Fix to replace dashes with underscore in the metric names by @aagarwalla-fx in https://github.com/prometheus-community/postgres_exporter/pull/1103 +* [BIGFIX] Checkpoint related columns in PG 17 have been moved from pg_stat_bgwriter to pg_stat_checkpointer by @n-rodriguez in https://github.com/prometheus-community/postgres_exporter/pull/1072 +* [BUGFIX] Fix pg_stat_statements for PG17 by @NevermindZ4 in https://github.com/prometheus-community/postgres_exporter/pull/1114 +* [BUGFIX] Handle pg_replication_slots on pg<13 by @michael-todorovic in https://github.com/prometheus-community/postgres_exporter/pull/1098 +* [BUGFIX] Fix missing dsn sanitization for logging by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1104 + +## New Contributors +* @jyothikirant-sayukth made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1092 +* @aagarwalla-fx made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1103 +* @NevermindZ4 made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1114 +* @michael-todorovic made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1098 +* @fgalind1 made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1106 + +**Full Changelog**: https://github.com/prometheus-community/postgres_exporter/compare/v0.16.0...v0.17.0 + ## 0.16.0 / 2024-11-10 BREAKING CHANGES: diff --git a/VERSION b/VERSION index 04a373efe..c5523bd09 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.16.0 +0.17.0 From 2869087f3cec1b2096f33ed9d182b0452f9b8d11 Mon Sep 17 00:00:00 2001 From: vancwo Date: Wed, 26 Feb 2025 05:21:39 -0800 Subject: [PATCH 018/113] Fix: Handle incoming labels with invalid UTF-8 (#1131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It's possible that incoming labels will contain invalid UTF-8 characters. This results in a panic. This fix sanitizes the label's string to ensure only valid UTF-8 characters are included, by replacing invalid characters with � (REPLACEMENT CHARACTER) Signed-off-by: Cooper Worobetz --- cmd/postgres_exporter/util.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmd/postgres_exporter/util.go b/cmd/postgres_exporter/util.go index 3baa6f4be..8907e7c5f 100644 --- a/cmd/postgres_exporter/util.go +++ b/cmd/postgres_exporter/util.go @@ -159,7 +159,7 @@ func dbToString(t interface{}) (string, bool) { // Try and convert to string return string(v), true case string: - return v, true + return strings.ToValidUTF8(v, "�"), true case bool: if v { return "true", true From 1e574cf4fd2a75a8a707d424eafcaa0b88cb7af4 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Wed, 26 Feb 2025 14:41:27 +0100 Subject: [PATCH 019/113] Release v0.17.1 (#1132) * [BUGFIX] Fix: Handle incoming labels with invalid UTF-8 #1131 Signed-off-by: SuperQ --- CHANGELOG.md | 4 ++++ VERSION | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 49b6b4101..d01ceb504 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,7 @@ +## 0.17.1 / 2025-02-26 + +* [BUGFIX] Fix: Handle incoming labels with invalid UTF-8 #1131 + ## 0.17.0 / 2025-02-16 ## What's Changed diff --git a/VERSION b/VERSION index c5523bd09..7cca7711a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.17.0 +0.17.1 From 457b6fa8cd89b34417a95911d7985d93a1b5c32d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Mar 2025 18:26:29 +0100 Subject: [PATCH 020/113] Bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 (#1133) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.20.5 to 1.21.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.20.5...v1.21.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 4 ++-- go.sum | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/go.mod b/go.mod index a7b376b4e..422c3241b 100644 --- a/go.mod +++ b/go.mod @@ -7,7 +7,7 @@ require ( github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.20.5 + github.com/prometheus/client_golang v1.21.0 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.62.0 github.com/prometheus/exporter-toolkit v0.14.0 @@ -25,7 +25,7 @@ require ( github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/jtolds/gls v4.20.0+incompatible // indirect - github.com/klauspost/compress v1.17.9 // indirect + github.com/klauspost/compress v1.17.11 // indirect github.com/kr/pretty v0.3.1 // indirect github.com/kr/text v0.2.0 // indirect github.com/mdlayher/socket v0.4.1 // indirect diff --git a/go.sum b/go.sum index a57fc83c1..7d4217fa1 100644 --- a/go.sum +++ b/go.sum @@ -26,8 +26,8 @@ github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE= -github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA= -github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc= +github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= @@ -50,8 +50,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+bR9r+8l63Y= -github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE= +github.com/prometheus/client_golang v1.21.0 h1:DIsaGmiaBkSangBgMtWdNfxbMNdku5IK6iNhrEqWvdA= +github.com/prometheus/client_golang v1.21.0/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= From 602302ffe22485c4e15f3101a705e588a849d1d5 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sat, 8 Mar 2025 15:00:54 +0100 Subject: [PATCH 021/113] Update common Prometheus files (#1137) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 6 +++--- Makefile.common | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index def9007ac..e36a9f1a4 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -28,12 +28,12 @@ jobs: - name: Install Go uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 with: - go-version: 1.23.x + go-version: 1.24.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@ec5d18412c0aeab7936cb16880d708ba2a64e1ae # v6.2.0 + uses: golangci/golangci-lint-action@2226d7cb06a077cd73e56eedd38eecad18e5d837 # v6.5.0 with: args: --verbose - version: v1.63.4 + version: v1.64.6 diff --git a/Makefile.common b/Makefile.common index d1576bb31..8cb383859 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.63.4 +GOLANGCI_LINT_VERSION ?= v1.64.6 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) From b0e61bf263fffeefc5d5b84fea8b8b0714958862 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 16 Mar 2025 22:34:30 +0100 Subject: [PATCH 022/113] Bump golang.org/x/net from 0.33.0 to 0.36.0 (#1138) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.33.0 to 0.36.0. - [Commits](https://github.com/golang/net/compare/v0.33.0...v0.36.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 10 +++++----- go.sum | 20 ++++++++++---------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/go.mod b/go.mod index 422c3241b..1d585cd23 100644 --- a/go.mod +++ b/go.mod @@ -36,11 +36,11 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.32.0 // indirect - golang.org/x/net v0.33.0 // indirect + golang.org/x/crypto v0.35.0 // indirect + golang.org/x/net v0.36.0 // indirect golang.org/x/oauth2 v0.24.0 // indirect - golang.org/x/sync v0.10.0 // indirect - golang.org/x/sys v0.29.0 // indirect - golang.org/x/text v0.21.0 // indirect + golang.org/x/sync v0.11.0 // indirect + golang.org/x/sys v0.30.0 // indirect + golang.org/x/text v0.22.0 // indirect google.golang.org/protobuf v1.36.1 // indirect ) diff --git a/go.sum b/go.sum index 7d4217fa1..09262cf79 100644 --- a/go.sum +++ b/go.sum @@ -73,18 +73,18 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc= -golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc= -golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= -golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= +golang.org/x/crypto v0.35.0 h1:b15kiHdrGCHrP6LvwaQ3c03kgNhhiMgvlhxHQhmg2Xs= +golang.org/x/crypto v0.35.0/go.mod h1:dy7dXNW32cAb/6/PRuTNsix8T+vJAqvuIy5Bli/x0YQ= +golang.org/x/net v0.36.0 h1:vWF2fRbw4qslQsQzgFqZff+BItCvGFQqKzKIzx1rmoA= +golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I= golang.org/x/oauth2 v0.24.0 h1:KTBBxWqUa0ykRPLtV69rRto9TLXcqYkeswu48x/gvNE= golang.org/x/oauth2 v0.24.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= -golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= -golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= -golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= -golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= +golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= +golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= +golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM= +golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk= google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= From 2ce65c324c58dceedd832583f196987f2c13b011 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sun, 23 Mar 2025 16:54:21 +0100 Subject: [PATCH 023/113] Update common Prometheus files (#1140) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index e36a9f1a4..b404ce7c7 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -33,7 +33,7 @@ jobs: run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@2226d7cb06a077cd73e56eedd38eecad18e5d837 # v6.5.0 + uses: golangci/golangci-lint-action@55c2c1448f86e01eaae002a5a3a9624417608d84 # v6.5.2 with: args: --verbose version: v1.64.6 From fca2ad84cd79fea6b414a2ce21fe8ee5b8d5a5a4 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sun, 30 Mar 2025 20:49:06 +0200 Subject: [PATCH 024/113] Update common Prometheus files (#1142) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index b404ce7c7..5342cbe08 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -26,7 +26,7 @@ jobs: - name: Checkout repository uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - name: Install Go - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0 with: go-version: 1.24.x - name: Install snmp_exporter/generator dependencies From 9e86f1ee380a4c3e4607991cd79e54713c1f3197 Mon Sep 17 00:00:00 2001 From: Ian Bibby <470816+ianbibby@users.noreply.github.com> Date: Thu, 3 Apr 2025 07:45:29 -0700 Subject: [PATCH 025/113] Adds pg_stat_progress_vacuum collector (#1141) Signed-off-by: Ian Bibby Co-authored-by: Ben Kochie --- README.md | 3 + collector/pg_stat_progress_vacuum.go | 222 ++++++++++++++++++++++ collector/pg_stat_progress_vacuum_test.go | 135 +++++++++++++ 3 files changed, 360 insertions(+) create mode 100644 collector/pg_stat_progress_vacuum.go create mode 100644 collector/pg_stat_progress_vacuum_test.go diff --git a/README.md b/README.md index 9494acb24..dedc5aa55 100644 --- a/README.md +++ b/README.md @@ -144,6 +144,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.stat_database` Enable the `stat_database` collector (default: enabled). +* `[no-]collector.stat_progress_vacuum` + Enable the `stat_progress_vacuum` collector (default: enabled). + * `[no-]collector.stat_statements` Enable the `stat_statements` collector (default: disabled). diff --git a/collector/pg_stat_progress_vacuum.go b/collector/pg_stat_progress_vacuum.go new file mode 100644 index 000000000..f8083a49f --- /dev/null +++ b/collector/pg_stat_progress_vacuum.go @@ -0,0 +1,222 @@ +// Copyright 2025 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + "log/slog" + + "github.com/prometheus/client_golang/prometheus" +) + +const progressVacuumSubsystem = "stat_progress_vacuum" + +func init() { + registerCollector(progressVacuumSubsystem, defaultEnabled, NewPGStatProgressVacuumCollector) +} + +type PGStatProgressVacuumCollector struct { + log *slog.Logger +} + +func NewPGStatProgressVacuumCollector(config collectorConfig) (Collector, error) { + return &PGStatProgressVacuumCollector{log: config.logger}, nil +} + +var vacuumPhases = []string{ + "initializing", + "scanning heap", + "vacuuming indexes", + "vacuuming heap", + "cleaning up indexes", + "truncating heap", + "performing final cleanup", +} + +var ( + statProgressVacuumPhase = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "phase"), + "Current vacuum phase (1 = active, 0 = inactive). Label 'phase' is human-readable.", + []string{"datname", "relname", "phase"}, + nil, + ) + + statProgressVacuumHeapBlksTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "heap_blks"), + "Total number of heap blocks in the table being vacuumed.", + []string{"datname", "relname"}, + nil, + ) + + statProgressVacuumHeapBlksScanned = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "heap_blks_scanned"), + "Number of heap blocks scanned so far.", + []string{"datname", "relname"}, + nil, + ) + + statProgressVacuumHeapBlksVacuumed = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "heap_blks_vacuumed"), + "Number of heap blocks vacuumed so far.", + []string{"datname", "relname"}, + nil, + ) + + statProgressVacuumIndexVacuumCount = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "index_vacuums"), + "Number of completed index vacuum cycles.", + []string{"datname", "relname"}, + nil, + ) + + statProgressVacuumMaxDeadTuples = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "max_dead_tuples"), + "Maximum number of dead tuples that can be stored before cleanup is performed.", + []string{"datname", "relname"}, + nil, + ) + + statProgressVacuumNumDeadTuples = prometheus.NewDesc( + prometheus.BuildFQName(namespace, progressVacuumSubsystem, "num_dead_tuples"), + "Current number of dead tuples found so far.", + []string{"datname", "relname"}, + nil, + ) + + // This is the view definition of pg_stat_progress_vacuum, albeit without the conversion + // of "phase" to a human-readable string. We will prefer the numeric representation. + statProgressVacuumQuery = `SELECT + d.datname, + s.relid::regclass::text AS relname, + s.param1 AS phase, + s.param2 AS heap_blks_total, + s.param3 AS heap_blks_scanned, + s.param4 AS heap_blks_vacuumed, + s.param5 AS index_vacuum_count, + s.param6 AS max_dead_tuples, + s.param7 AS num_dead_tuples + FROM + pg_stat_get_progress_info('VACUUM'::text) + s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20) + LEFT JOIN + pg_database d ON s.datid = d.oid` +) + +func (c *PGStatProgressVacuumCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statProgressVacuumQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var ( + datname sql.NullString + relname sql.NullString + phase sql.NullInt64 + heapBlksTotal sql.NullInt64 + heapBlksScanned sql.NullInt64 + heapBlksVacuumed sql.NullInt64 + indexVacuumCount sql.NullInt64 + maxDeadTuples sql.NullInt64 + numDeadTuples sql.NullInt64 + ) + + if err := rows.Scan( + &datname, + &relname, + &phase, + &heapBlksTotal, + &heapBlksScanned, + &heapBlksVacuumed, + &indexVacuumCount, + &maxDeadTuples, + &numDeadTuples, + ); err != nil { + return err + } + + datnameLabel := "unknown" + if datname.Valid { + datnameLabel = datname.String + } + relnameLabel := "unknown" + if relname.Valid { + relnameLabel = relname.String + } + + labels := []string{datnameLabel, relnameLabel} + + var phaseMetric *float64 + if phase.Valid { + v := float64(phase.Int64) + phaseMetric = &v + } + + for i, label := range vacuumPhases { + v := 0.0 + // Only the current phase should be 1.0. + if phaseMetric != nil && float64(i) == *phaseMetric { + v = 1.0 + } + labelsCopy := append(labels, label) + ch <- prometheus.MustNewConstMetric(statProgressVacuumPhase, prometheus.GaugeValue, v, labelsCopy...) + } + + heapTotal := 0.0 + if heapBlksTotal.Valid { + heapTotal = float64(heapBlksTotal.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumHeapBlksTotal, prometheus.GaugeValue, heapTotal, labels...) + + heapScanned := 0.0 + if heapBlksScanned.Valid { + heapScanned = float64(heapBlksScanned.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumHeapBlksScanned, prometheus.GaugeValue, heapScanned, labels...) + + heapVacuumed := 0.0 + if heapBlksVacuumed.Valid { + heapVacuumed = float64(heapBlksVacuumed.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumHeapBlksVacuumed, prometheus.GaugeValue, heapVacuumed, labels...) + + indexCount := 0.0 + if indexVacuumCount.Valid { + indexCount = float64(indexVacuumCount.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumIndexVacuumCount, prometheus.GaugeValue, indexCount, labels...) + + maxDead := 0.0 + if maxDeadTuples.Valid { + maxDead = float64(maxDeadTuples.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumMaxDeadTuples, prometheus.GaugeValue, maxDead, labels...) + + numDead := 0.0 + if numDeadTuples.Valid { + numDead = float64(numDeadTuples.Int64) + } + ch <- prometheus.MustNewConstMetric(statProgressVacuumNumDeadTuples, prometheus.GaugeValue, numDead, labels...) + } + + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_stat_progress_vacuum_test.go b/collector/pg_stat_progress_vacuum_test.go new file mode 100644 index 000000000..80572feb8 --- /dev/null +++ b/collector/pg_stat_progress_vacuum_test.go @@ -0,0 +1,135 @@ +// Copyright 2025 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatProgressVacuumCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datname", "relname", "phase", "heap_blks_total", "heap_blks_scanned", + "heap_blks_vacuumed", "index_vacuum_count", "max_dead_tuples", "num_dead_tuples", + } + + rows := sqlmock.NewRows(columns).AddRow( + "postgres", "a_table", 3, 3000, 400, 200, 2, 500, 123) + + mock.ExpectQuery(sanitizeQuery(statProgressVacuumQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatProgressVacuumCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatProgressVacuumCollector.Update; %+v", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "initializing"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "scanning heap"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "vacuuming indexes"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "vacuuming heap"}, metricType: dto.MetricType_GAUGE, value: 1}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "cleaning up indexes"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "truncating heap"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table", "phase": "performing final cleanup"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 3000}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 400}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 200}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 2}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 500}, + {labels: labelMap{"datname": "postgres", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 123}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(m, convey.ShouldResemble, expect) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("There were unfulfilled exceptions: %+v", err) + } +} + +func TestPGStatProgressVacuumCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datname", "relname", "phase", "heap_blks_total", "heap_blks_scanned", + "heap_blks_vacuumed", "index_vacuum_count", "max_dead_tuples", "num_dead_tuples", + } + + rows := sqlmock.NewRows(columns).AddRow( + "postgres", nil, nil, nil, nil, nil, nil, nil, nil) + + mock.ExpectQuery(sanitizeQuery(statProgressVacuumQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatProgressVacuumCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatProgressVacuumCollector.Update; %+v", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "initializing"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "scanning heap"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "vacuuming indexes"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "vacuuming heap"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "cleaning up indexes"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "truncating heap"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown", "phase": "performing final cleanup"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("There were unfulfilled exceptions: %+v", err) + } +} From 8d5ec4b3ea77033d5d2b50fad1bb8df3f495265e Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Thu, 3 Apr 2025 16:23:40 +0100 Subject: [PATCH 026/113] Update Go (#1147) * Update Go to 1.24. * Update golangci-lint to v2. * Fixup linting issues. Signed-off-by: SuperQ --- .circleci/config.yml | 4 +- .github/workflows/golangci-lint.yml | 4 +- .golangci.yml | 53 +++++++++++++++--------- .promu.yml | 2 +- Makefile.common | 2 +- cmd/postgres_exporter/pg_setting.go | 4 +- cmd/postgres_exporter/pg_setting_test.go | 4 +- collector/collector_test.go | 20 ++++----- config/config.go | 4 +- config/config_test.go | 6 +-- 10 files changed, 58 insertions(+), 45 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 6cfc880d2..ee03ff8d2 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -8,7 +8,7 @@ executors: # This must match .promu.yml. golang: docker: - - image: cimg/go:1.23 + - image: cimg/go:1.24 jobs: test: @@ -23,7 +23,7 @@ jobs: integration: docker: - - image: cimg/go:1.23 + - image: cimg/go:1.24 - image: << parameters.postgres_image >> environment: POSTGRES_DB: circle_test diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 5342cbe08..3893ef86b 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -33,7 +33,7 @@ jobs: run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@55c2c1448f86e01eaae002a5a3a9624417608d84 # v6.5.2 + uses: golangci/golangci-lint-action@1481404843c368bc19ca9406f87d6e0fc97bdcfd # v7.0.0 with: args: --verbose - version: v1.64.6 + version: v2.0.2 diff --git a/.golangci.yml b/.golangci.yml index 96487c898..4b58b08b6 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,23 +1,36 @@ ---- +version: "2" linters: enable: - - misspell - - revive - -issues: - exclude-rules: - - path: _test.go - linters: - - errcheck - -linters-settings: - errcheck: - exclude-functions: - # Never check for logger errors. - - (github.com/go-kit/log.Logger).Log - revive: + - misspell + - revive + settings: + errcheck: + exclude-functions: + - (github.com/go-kit/log.Logger).Log + revive: + rules: + - name: unused-parameter + severity: warning + disabled: true + exclusions: + generated: lax + presets: + - comments + - common-false-positives + - legacy + - std-error-handling rules: - # https://github.com/mgechev/revive/blob/master/RULES_DESCRIPTIONS.md#unused-parameter - - name: unused-parameter - severity: warning - disabled: true + - linters: + - errcheck + path: _test.go + paths: + - third_party$ + - builtin$ + - examples$ +formatters: + exclusions: + generated: lax + paths: + - third_party$ + - builtin$ + - examples$ diff --git a/.promu.yml b/.promu.yml index c2b361240..5f915289f 100644 --- a/.promu.yml +++ b/.promu.yml @@ -1,6 +1,6 @@ go: # This must match .circle/config.yml. - version: 1.23 + version: 1.24 repository: path: github.com/prometheus-community/postgres_exporter build: diff --git a/Makefile.common b/Makefile.common index 8cb383859..81bad5f42 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.64.6 +GOLANGCI_LINT_VERSION ?= v2.0.2 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) diff --git a/cmd/postgres_exporter/pg_setting.go b/cmd/postgres_exporter/pg_setting.go index 02e65dd2a..5b13e160f 100644 --- a/cmd/postgres_exporter/pg_setting.go +++ b/cmd/postgres_exporter/pg_setting.go @@ -67,7 +67,7 @@ type pgSetting struct { func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { var ( err error - name = strings.Replace(strings.Replace(s.name, ".", "_", -1), "-", "_", -1) + name = strings.ReplaceAll(strings.ReplaceAll(s.name, ".", "_"), "-", "_") unit = s.unit // nolint: ineffassign shortDesc = fmt.Sprintf("Server Parameter: %s", s.name) subsystem = "settings" @@ -131,7 +131,7 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) { case "B", "kB", "MB", "GB", "TB", "1kB", "2kB", "4kB", "8kB", "16kB", "32kB", "64kB", "16MB", "32MB", "64MB": unit = "bytes" default: - err = fmt.Errorf("Unknown unit for runtime variable: %q", s.unit) + err = fmt.Errorf("unknown unit for runtime variable: %q", s.unit) return } diff --git a/cmd/postgres_exporter/pg_setting_test.go b/cmd/postgres_exporter/pg_setting_test.go index 0e010444d..6923da630 100644 --- a/cmd/postgres_exporter/pg_setting_test.go +++ b/cmd/postgres_exporter/pg_setting_test.go @@ -214,7 +214,7 @@ var fixtures = []fixture{ n: normalised{ val: 10, unit: "", - err: `Unknown unit for runtime variable: "nonexistent"`, + err: `unknown unit for runtime variable: "nonexistent"`, }, }, } @@ -240,7 +240,7 @@ func (s *PgSettingSuite) TestNormaliseUnit(c *C) { func (s *PgSettingSuite) TestMetric(c *C) { defer func() { if r := recover(); r != nil { - if r.(error).Error() != `Unknown unit for runtime variable: "nonexistent"` { + if r.(error).Error() != `unknown unit for runtime variable: "nonexistent"` { panic(r) } } diff --git a/collector/collector_test.go b/collector/collector_test.go index 18101f00e..d3b473b43 100644 --- a/collector/collector_test.go +++ b/collector/collector_test.go @@ -48,15 +48,15 @@ func readMetric(m prometheus.Metric) MetricResult { func sanitizeQuery(q string) string { q = strings.Join(strings.Fields(q), " ") - q = strings.Replace(q, "(", "\\(", -1) - q = strings.Replace(q, "?", "\\?", -1) - q = strings.Replace(q, ")", "\\)", -1) - q = strings.Replace(q, "[", "\\[", -1) - q = strings.Replace(q, "]", "\\]", -1) - q = strings.Replace(q, "{", "\\{", -1) - q = strings.Replace(q, "}", "\\}", -1) - q = strings.Replace(q, "*", "\\*", -1) - q = strings.Replace(q, "^", "\\^", -1) - q = strings.Replace(q, "$", "\\$", -1) + q = strings.ReplaceAll(q, "(", "\\(") + q = strings.ReplaceAll(q, "?", "\\?") + q = strings.ReplaceAll(q, ")", "\\)") + q = strings.ReplaceAll(q, "[", "\\[") + q = strings.ReplaceAll(q, "]", "\\]") + q = strings.ReplaceAll(q, "{", "\\{") + q = strings.ReplaceAll(q, "}", "\\}") + q = strings.ReplaceAll(q, "*", "\\*") + q = strings.ReplaceAll(q, "^", "\\^") + q = strings.ReplaceAll(q, "$", "\\$") return q } diff --git a/config/config.go b/config/config.go index 7cdc08f7b..52c66513a 100644 --- a/config/config.go +++ b/config/config.go @@ -79,14 +79,14 @@ func (ch *Handler) ReloadConfig(f string, logger *slog.Logger) error { yamlReader, err := os.Open(f) if err != nil { - return fmt.Errorf("Error opening config file %q: %s", f, err) + return fmt.Errorf("error opening config file %q: %s", f, err) } defer yamlReader.Close() decoder := yaml.NewDecoder(yamlReader) decoder.KnownFields(true) if err = decoder.Decode(config); err != nil { - return fmt.Errorf("Error parsing config file %q: %s", f, err) + return fmt.Errorf("error parsing config file %q: %s", f, err) } ch.Lock() diff --git a/config/config_test.go b/config/config_test.go index d5d23d3ba..fa59c9b40 100644 --- a/config/config_test.go +++ b/config/config_test.go @@ -24,7 +24,7 @@ func TestLoadConfig(t *testing.T) { err := ch.ReloadConfig("testdata/config-good.yaml", nil) if err != nil { - t.Errorf("Error loading config: %s", err) + t.Errorf("error loading config: %s", err) } } @@ -39,11 +39,11 @@ func TestLoadBadConfigs(t *testing.T) { }{ { input: "testdata/config-bad-auth-module.yaml", - want: "Error parsing config file \"testdata/config-bad-auth-module.yaml\": yaml: unmarshal errors:\n line 3: field pretendauth not found in type config.AuthModule", + want: "error parsing config file \"testdata/config-bad-auth-module.yaml\": yaml: unmarshal errors:\n line 3: field pretendauth not found in type config.AuthModule", }, { input: "testdata/config-bad-extra-field.yaml", - want: "Error parsing config file \"testdata/config-bad-extra-field.yaml\": yaml: unmarshal errors:\n line 8: field doesNotExist not found in type config.AuthModule", + want: "error parsing config file \"testdata/config-bad-extra-field.yaml\": yaml: unmarshal errors:\n line 8: field doesNotExist not found in type config.AuthModule", }, } From 43576acc7642f5cb97d3d1f6bbe0f9e514ed9301 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 3 Apr 2025 18:17:55 +0200 Subject: [PATCH 027/113] Bump github.com/prometheus/client_golang from 1.21.0 to 1.21.1 (#1144) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.0 to 1.21.1. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.0...v1.21.1) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.21.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 4 +++- go.sum | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 1d585cd23..889893b5e 100644 --- a/go.mod +++ b/go.mod @@ -2,12 +2,14 @@ module github.com/prometheus-community/postgres_exporter go 1.23.0 +toolchain go1.24.1 + require ( github.com/DATA-DOG/go-sqlmock v1.5.2 github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.21.0 + github.com/prometheus/client_golang v1.21.1 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.62.0 github.com/prometheus/exporter-toolkit v0.14.0 diff --git a/go.sum b/go.sum index 09262cf79..5f7a25e95 100644 --- a/go.sum +++ b/go.sum @@ -50,8 +50,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.21.0 h1:DIsaGmiaBkSangBgMtWdNfxbMNdku5IK6iNhrEqWvdA= -github.com/prometheus/client_golang v1.21.0/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg= +github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGCQ8YOmu+Ibk= +github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= From f8b7139174317e3df6d756a02c99b4d8de649d0f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 3 Apr 2025 18:18:17 +0200 Subject: [PATCH 028/113] Bump github.com/prometheus/common from 0.62.0 to 0.63.0 (#1143) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.62.0 to 0.63.0. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md) - [Commits](https://github.com/prometheus/common/compare/v0.62.0...v0.63.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.63.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 6 +++--- go.sum | 16 ++++++++-------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/go.mod b/go.mod index 889893b5e..ac1ceeff4 100644 --- a/go.mod +++ b/go.mod @@ -11,7 +11,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.21.1 github.com/prometheus/client_model v0.6.1 - github.com/prometheus/common v0.62.0 + github.com/prometheus/common v0.63.0 github.com/prometheus/exporter-toolkit v0.14.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -40,9 +40,9 @@ require ( github.com/xhit/go-str2duration/v2 v2.1.0 // indirect golang.org/x/crypto v0.35.0 // indirect golang.org/x/net v0.36.0 // indirect - golang.org/x/oauth2 v0.24.0 // indirect + golang.org/x/oauth2 v0.25.0 // indirect golang.org/x/sync v0.11.0 // indirect golang.org/x/sys v0.30.0 // indirect golang.org/x/text v0.22.0 // indirect - google.golang.org/protobuf v1.36.1 // indirect + google.golang.org/protobuf v1.36.5 // indirect ) diff --git a/go.sum b/go.sum index 5f7a25e95..939e1d083 100644 --- a/go.sum +++ b/go.sum @@ -17,8 +17,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= -github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= -github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= @@ -54,8 +54,8 @@ github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGC github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= -github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= -github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I= +github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k= +github.com/prometheus/common v0.63.0/go.mod h1:VVFF/fBIoToEnWRVkYoXEkq3R3paCoxG9PXP74SnV18= github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= @@ -77,16 +77,16 @@ golang.org/x/crypto v0.35.0 h1:b15kiHdrGCHrP6LvwaQ3c03kgNhhiMgvlhxHQhmg2Xs= golang.org/x/crypto v0.35.0/go.mod h1:dy7dXNW32cAb/6/PRuTNsix8T+vJAqvuIy5Bli/x0YQ= golang.org/x/net v0.36.0 h1:vWF2fRbw4qslQsQzgFqZff+BItCvGFQqKzKIzx1rmoA= golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I= -golang.org/x/oauth2 v0.24.0 h1:KTBBxWqUa0ykRPLtV69rRto9TLXcqYkeswu48x/gvNE= -golang.org/x/oauth2 v0.24.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= +golang.org/x/oauth2 v0.25.0 h1:CY4y7XT9v0cRI9oupztF8AgiIu99L/ksR/Xp/6jrZ70= +golang.org/x/oauth2 v0.25.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM= golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= -google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk= -google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= +google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM= +google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From 5edc331876c3f94bd5ca26e502fee4e4df77e2d1 Mon Sep 17 00:00:00 2001 From: Felix Yuan Date: Fri, 18 Apr 2025 07:14:25 -0700 Subject: [PATCH 029/113] Record table only size bytes as well in addition to the total size bytes (#1149) * Record table only size bytes as well in addition to the total size bytes Signed-off-by: Felix Yuan * Update collector/pg_stat_user_tables.go Co-authored-by: Ben Kochie Signed-off-by: Felix Yuan * Update collector/pg_stat_user_tables.go Co-authored-by: Ben Kochie Signed-off-by: Felix Yuan * Finish renaming elements to index and table size Signed-off-by: Felix Yuan --------- Signed-off-by: Felix Yuan Co-authored-by: Ben Kochie --- collector/pg_stat_user_tables.go | 40 +++++++++++++++++++-------- collector/pg_stat_user_tables_test.go | 14 ++++++++-- 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/collector/pg_stat_user_tables.go b/collector/pg_stat_user_tables.go index 254d76a66..ad8bcace7 100644 --- a/collector/pg_stat_user_tables.go +++ b/collector/pg_stat_user_tables.go @@ -150,9 +150,15 @@ var ( []string{"datname", "schemaname", "relname"}, prometheus.Labels{}, ) - statUserTablesTotalSize = prometheus.NewDesc( - prometheus.BuildFQName(namespace, userTableSubsystem, "size_bytes"), - "Total disk space used by this table, in bytes, including all indexes and TOAST data", + statUserIndexSize = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "index_size_bytes"), + "Total disk space used by this index, in bytes", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTableSize = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "table_size_bytes"), + "Total disk space used by this table, in bytes", []string{"datname", "schemaname", "relname"}, prometheus.Labels{}, ) @@ -180,7 +186,8 @@ var ( autovacuum_count, analyze_count, autoanalyze_count, - pg_total_relation_size(relid) as total_size + pg_indexes_size(relid) as indexes_size, + pg_table_size(relid) as table_size FROM pg_stat_user_tables` ) @@ -198,10 +205,10 @@ func (c *PGStatUserTablesCollector) Update(ctx context.Context, instance *instan for rows.Next() { var datname, schemaname, relname sql.NullString var seqScan, seqTupRead, idxScan, idxTupFetch, nTupIns, nTupUpd, nTupDel, nTupHotUpd, nLiveTup, nDeadTup, - nModSinceAnalyze, vacuumCount, autovacuumCount, analyzeCount, autoanalyzeCount, totalSize sql.NullInt64 + nModSinceAnalyze, vacuumCount, autovacuumCount, analyzeCount, autoanalyzeCount, indexSize, tableSize sql.NullInt64 var lastVacuum, lastAutovacuum, lastAnalyze, lastAutoanalyze sql.NullTime - if err := rows.Scan(&datname, &schemaname, &relname, &seqScan, &seqTupRead, &idxScan, &idxTupFetch, &nTupIns, &nTupUpd, &nTupDel, &nTupHotUpd, &nLiveTup, &nDeadTup, &nModSinceAnalyze, &lastVacuum, &lastAutovacuum, &lastAnalyze, &lastAutoanalyze, &vacuumCount, &autovacuumCount, &analyzeCount, &autoanalyzeCount, &totalSize); err != nil { + if err := rows.Scan(&datname, &schemaname, &relname, &seqScan, &seqTupRead, &idxScan, &idxTupFetch, &nTupIns, &nTupUpd, &nTupDel, &nTupHotUpd, &nLiveTup, &nDeadTup, &nModSinceAnalyze, &lastVacuum, &lastAutovacuum, &lastAnalyze, &lastAutoanalyze, &vacuumCount, &autovacuumCount, &analyzeCount, &autoanalyzeCount, &indexSize, &tableSize); err != nil { return err } @@ -427,14 +434,25 @@ func (c *PGStatUserTablesCollector) Update(ctx context.Context, instance *instan datnameLabel, schemanameLabel, relnameLabel, ) - totalSizeMetric := 0.0 - if totalSize.Valid { - totalSizeMetric = float64(totalSize.Int64) + indexSizeMetric := 0.0 + if indexSize.Valid { + indexSizeMetric = float64(indexSize.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserIndexSize, + prometheus.GaugeValue, + indexSizeMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + tableSizeMetric := 0.0 + if tableSize.Valid { + tableSizeMetric = float64(tableSize.Int64) } ch <- prometheus.MustNewConstMetric( - statUserTablesTotalSize, + statUserTableSize, prometheus.GaugeValue, - totalSizeMetric, + tableSizeMetric, datnameLabel, schemanameLabel, relnameLabel, ) } diff --git a/collector/pg_stat_user_tables_test.go b/collector/pg_stat_user_tables_test.go index 5e82335c3..4649bdbc5 100644 --- a/collector/pg_stat_user_tables_test.go +++ b/collector/pg_stat_user_tables_test.go @@ -72,7 +72,8 @@ func TestPGStatUserTablesCollector(t *testing.T) { "autovacuum_count", "analyze_count", "autoanalyze_count", - "total_size"} + "index_size", + "table_size"} rows := sqlmock.NewRows(columns). AddRow("postgres", "public", @@ -96,7 +97,8 @@ func TestPGStatUserTablesCollector(t *testing.T) { 12, 13, 14, - 15) + 15, + 16) mock.ExpectQuery(sanitizeQuery(statUserTablesQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -128,6 +130,8 @@ func TestPGStatUserTablesCollector(t *testing.T) { {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 12}, {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 13}, {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 14}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 15}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 16}, } convey.Convey("Metrics comparison", t, func() { @@ -173,7 +177,8 @@ func TestPGStatUserTablesCollectorNullValues(t *testing.T) { "autovacuum_count", "analyze_count", "autoanalyze_count", - "total_size"} + "index_size", + "table_size"} rows := sqlmock.NewRows(columns). AddRow("postgres", nil, @@ -197,6 +202,7 @@ func TestPGStatUserTablesCollectorNullValues(t *testing.T) { nil, nil, nil, + nil, nil) mock.ExpectQuery(sanitizeQuery(statUserTablesQuery)).WillReturnRows(rows) ch := make(chan prometheus.Metric) @@ -229,6 +235,8 @@ func TestPGStatUserTablesCollectorNullValues(t *testing.T) { {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, } convey.Convey("Metrics comparison", t, func() { From 6526065fbcdc2fa780b1db6784b9978602f37d94 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 22 Apr 2025 21:26:10 -0400 Subject: [PATCH 030/113] Bump golang.org/x/net from 0.36.0 to 0.38.0 (#1152) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.36.0 to 0.38.0. - [Commits](https://github.com/golang/net/compare/v0.36.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 10 +++++----- go.sum | 20 ++++++++++---------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/go.mod b/go.mod index ac1ceeff4..e3eba7880 100644 --- a/go.mod +++ b/go.mod @@ -38,11 +38,11 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.35.0 // indirect - golang.org/x/net v0.36.0 // indirect + golang.org/x/crypto v0.36.0 // indirect + golang.org/x/net v0.38.0 // indirect golang.org/x/oauth2 v0.25.0 // indirect - golang.org/x/sync v0.11.0 // indirect - golang.org/x/sys v0.30.0 // indirect - golang.org/x/text v0.22.0 // indirect + golang.org/x/sync v0.12.0 // indirect + golang.org/x/sys v0.31.0 // indirect + golang.org/x/text v0.23.0 // indirect google.golang.org/protobuf v1.36.5 // indirect ) diff --git a/go.sum b/go.sum index 939e1d083..25dd5c07d 100644 --- a/go.sum +++ b/go.sum @@ -73,18 +73,18 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -golang.org/x/crypto v0.35.0 h1:b15kiHdrGCHrP6LvwaQ3c03kgNhhiMgvlhxHQhmg2Xs= -golang.org/x/crypto v0.35.0/go.mod h1:dy7dXNW32cAb/6/PRuTNsix8T+vJAqvuIy5Bli/x0YQ= -golang.org/x/net v0.36.0 h1:vWF2fRbw4qslQsQzgFqZff+BItCvGFQqKzKIzx1rmoA= -golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I= +golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34= +golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc= +golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= +golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= golang.org/x/oauth2 v0.25.0 h1:CY4y7XT9v0cRI9oupztF8AgiIu99L/ksR/Xp/6jrZ70= golang.org/x/oauth2 v0.25.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= -golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= -golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= -golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM= -golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= +golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw= +golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik= +golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY= +golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4= google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM= google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= From 5ade81cb9d391aef0606c44ee13830efa55bb3f2 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Wed, 7 May 2025 03:54:01 +0200 Subject: [PATCH 031/113] Update common Prometheus files (#1155) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- Makefile.common | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 3893ef86b..672dd424d 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -36,4 +36,4 @@ jobs: uses: golangci/golangci-lint-action@1481404843c368bc19ca9406f87d6e0fc97bdcfd # v7.0.0 with: args: --verbose - version: v2.0.2 + version: v2.1.5 diff --git a/Makefile.common b/Makefile.common index 81bad5f42..d8b798909 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.0.2 +GOLANGCI_LINT_VERSION ?= v2.1.5 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) From bd8a6135ecc149c02c429a5fa285cac228b244cc Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 6 May 2025 22:14:44 -0400 Subject: [PATCH 032/113] Bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 (#1154) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 3 +-- go.sum | 8 ++++---- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/go.mod b/go.mod index e3eba7880..9acdafc81 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.21.1 + github.com/prometheus/client_golang v1.22.0 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.63.0 github.com/prometheus/exporter-toolkit v0.14.0 @@ -27,7 +27,6 @@ require ( github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/jtolds/gls v4.20.0+incompatible // indirect - github.com/klauspost/compress v1.17.11 // indirect github.com/kr/pretty v0.3.1 // indirect github.com/kr/text v0.2.0 // indirect github.com/mdlayher/socket v0.4.1 // indirect diff --git a/go.sum b/go.sum index 25dd5c07d..cd073f325 100644 --- a/go.sum +++ b/go.sum @@ -26,8 +26,8 @@ github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE= -github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc= -github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= @@ -50,8 +50,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGCQ8YOmu+Ibk= -github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg= +github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q= +github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k= From de42ef7de8bd14f314a609ef50e4c323617258dc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20=C5=A0tiller?= Date: Fri, 16 May 2025 10:50:19 +0200 Subject: [PATCH 033/113] Export query itself together with queryId in stat_statement metrics (#940) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Export query itself together with queryId in stat_statement metrics * The feature must be enabled via flag. * Limit length of selected query. The query is not added to every metrics, but instead of new metric stat_statement_query_id is introduced that contains mapping between queryId and query. Fixes: #813 --------- Signed-off-by: Jakub Štiller Signed-off-by: Jakub Štiller --- README.md | 6 + collector/collector.go | 7 +- collector/pg_stat_statements.go | 89 +++++++++++-- collector/pg_stat_statements_test.go | 185 ++++++++++++++++++++++++++- 4 files changed, 270 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index dedc5aa55..01bd8b30d 100644 --- a/README.md +++ b/README.md @@ -150,6 +150,12 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.stat_statements` Enable the `stat_statements` collector (default: disabled). +* `[no-]collector.stat_statements.include_query` + Enable selecting statement query together with queryId. (default: disabled) + +* `--collector.stat_statements.query_length` + Maximum length of the statement text. Default is 120. + * `[no-]collector.stat_user_tables` Enable the `stat_user_tables` collector (default: enabled). diff --git a/collector/collector.go b/collector/collector.go index f2102d5ef..298bc36ee 100644 --- a/collector/collector.go +++ b/collector/collector.go @@ -37,8 +37,9 @@ const ( // Namespace for all metrics. namespace = "pg" - defaultEnabled = true - defaultDisabled = false + collectorFlagPrefix = "collector." + defaultEnabled = true + defaultDisabled = false ) var ( @@ -74,7 +75,7 @@ func registerCollector(name string, isDefaultEnabled bool, createFunc func(colle } // Create flag for this collector - flagName := fmt.Sprintf("collector.%s", name) + flagName := collectorFlagPrefix + name flagHelp := fmt.Sprintf("Enable the %s collector (default: %s).", name, helpDefaultState) defaultValue := fmt.Sprintf("%v", isDefaultEnabled) diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index cabdc3619..9160d3c16 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -16,27 +16,51 @@ package collector import ( "context" "database/sql" + "fmt" "log/slog" + "github.com/alecthomas/kingpin/v2" "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" ) const statStatementsSubsystem = "stat_statements" +var ( + includeQueryFlag *bool = nil + statementLengthFlag *uint = nil +) + func init() { // WARNING: // Disabled by default because this set of metrics can be quite expensive on a busy server // Every unique query will cause a new timeseries to be created registerCollector(statStatementsSubsystem, defaultDisabled, NewPGStatStatementsCollector) + + includeQueryFlag = kingpin.Flag( + fmt.Sprint(collectorFlagPrefix, statStatementsSubsystem, ".include_query"), + "Enable selecting statement query together with queryId. (default: disabled)"). + Default(fmt.Sprintf("%v", defaultDisabled)). + Bool() + statementLengthFlag = kingpin.Flag( + fmt.Sprint(collectorFlagPrefix, statStatementsSubsystem, ".query_length"), + "Maximum length of the statement text."). + Default("120"). + Uint() } type PGStatStatementsCollector struct { - log *slog.Logger + log *slog.Logger + includeQueryStatement bool + statementLength uint } func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { - return &PGStatStatementsCollector{log: config.logger}, nil + return &PGStatStatementsCollector{ + log: config.logger, + includeQueryStatement: *includeQueryFlag, + statementLength: *statementLengthFlag, + }, nil } var ( @@ -71,10 +95,22 @@ var ( prometheus.Labels{}, ) + statStatementsQuery = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "query_id"), + "SQL Query to queryid mapping", + []string{"queryid", "query"}, + prometheus.Labels{}, + ) +) + +const ( + pgStatStatementQuerySelect = `LEFT(pg_stat_statements.query, %d) as query,` + pgStatStatementsQuery = `SELECT pg_get_userbyid(userid) as user, pg_database.datname, pg_stat_statements.queryid, + %s pg_stat_statements.calls as calls_total, pg_stat_statements.total_time / 1000.0 as seconds_total, pg_stat_statements.rows as rows_total, @@ -96,6 +132,7 @@ var ( pg_get_userbyid(userid) as user, pg_database.datname, pg_stat_statements.queryid, + %s pg_stat_statements.calls as calls_total, pg_stat_statements.total_exec_time / 1000.0 as seconds_total, pg_stat_statements.rows as rows_total, @@ -117,6 +154,7 @@ var ( pg_get_userbyid(userid) as user, pg_database.datname, pg_stat_statements.queryid, + %s pg_stat_statements.calls as calls_total, pg_stat_statements.total_exec_time / 1000.0 as seconds_total, pg_stat_statements.rows as rows_total, @@ -135,30 +173,42 @@ var ( LIMIT 100;` ) -func (PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { - var query string +func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + var queryTemplate string switch { case instance.version.GE(semver.MustParse("17.0.0")): - query = pgStatStatementsQuery_PG17 + queryTemplate = pgStatStatementsQuery_PG17 case instance.version.GE(semver.MustParse("13.0.0")): - query = pgStatStatementsNewQuery + queryTemplate = pgStatStatementsNewQuery default: - query = pgStatStatementsQuery + queryTemplate = pgStatStatementsQuery } + var querySelect = "" + if c.includeQueryStatement { + querySelect = fmt.Sprintf(pgStatStatementQuerySelect, c.statementLength) + } + query := fmt.Sprintf(queryTemplate, querySelect) db := instance.getDB() rows, err := db.QueryContext(ctx, query) + var presentQueryIds = make(map[string]struct{}) + if err != nil { return err } defer rows.Close() for rows.Next() { - var user, datname, queryid sql.NullString + var user, datname, queryid, statement sql.NullString var callsTotal, rowsTotal sql.NullInt64 var secondsTotal, blockReadSecondsTotal, blockWriteSecondsTotal sql.NullFloat64 - - if err := rows.Scan(&user, &datname, &queryid, &callsTotal, &secondsTotal, &rowsTotal, &blockReadSecondsTotal, &blockWriteSecondsTotal); err != nil { + var columns []any + if c.includeQueryStatement { + columns = []any{&user, &datname, &queryid, &statement, &callsTotal, &secondsTotal, &rowsTotal, &blockReadSecondsTotal, &blockWriteSecondsTotal} + } else { + columns = []any{&user, &datname, &queryid, &callsTotal, &secondsTotal, &rowsTotal, &blockReadSecondsTotal, &blockWriteSecondsTotal} + } + if err := rows.Scan(columns...); err != nil { return err } @@ -229,6 +279,25 @@ func (PGStatStatementsCollector) Update(ctx context.Context, instance *instance, blockWriteSecondsTotalMetric, userLabel, datnameLabel, queryidLabel, ) + + if c.includeQueryStatement { + _, ok := presentQueryIds[queryidLabel] + if !ok { + presentQueryIds[queryidLabel] = struct{}{} + + queryLabel := "unknown" + if statement.Valid { + queryLabel = statement.String + } + + ch <- prometheus.MustNewConstMetric( + statStatementsQuery, + prometheus.CounterValue, + 1, + queryidLabel, queryLabel, + ) + } + } } if err := rows.Err(); err != nil { return err diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go index ea035f557..0497ba380 100644 --- a/collector/pg_stat_statements_test.go +++ b/collector/pg_stat_statements_test.go @@ -14,6 +14,7 @@ package collector import ( "context" + "fmt" "testing" "github.com/DATA-DOG/go-sqlmock" @@ -35,7 +36,7 @@ func TestPGStateStatementsCollector(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(pgStatStatementsQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, ""))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -66,6 +67,50 @@ func TestPGStateStatementsCollector(t *testing.T) { } } +func TestPGStateStatementsCollectorWithStatement(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("12.0.0")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 100) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100)))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 100} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + func TestPGStateStatementsCollectorNull(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { @@ -78,7 +123,7 @@ func TestPGStateStatementsCollectorNull(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(pgStatStatementsNewQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, ""))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -109,6 +154,50 @@ func TestPGStateStatementsCollectorNull(t *testing.T) { } } +func TestPGStateStatementsCollectorNullWithStatement(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, fmt.Sprintf(pgStatStatementQuerySelect, 200)))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 200} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"queryid": "unknown", "query": "unknown"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + func TestPGStateStatementsCollectorNewPG(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { @@ -121,7 +210,7 @@ func TestPGStateStatementsCollectorNewPG(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(pgStatStatementsNewQuery)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, ""))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -152,6 +241,50 @@ func TestPGStateStatementsCollectorNewPG(t *testing.T) { } } +func TestPGStateStatementsCollectorNewPGWithStatement(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, fmt.Sprintf(pgStatStatementQuerySelect, 300)))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 300} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + func TestPGStateStatementsCollector_PG17(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { @@ -164,7 +297,7 @@ func TestPGStateStatementsCollector_PG17(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(pgStatStatementsQuery_PG17)).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, ""))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -194,3 +327,47 @@ func TestPGStateStatementsCollector_PG17(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPGStateStatementsCollector_PG17_WithStatement(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("17.0.0")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300)))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 300} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From d1a957f679fd808c02cd88e6da7c11aeb7eae4d8 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sat, 17 May 2025 12:06:43 +0200 Subject: [PATCH 034/113] Update common Prometheus files (#1159) Signed-off-by: prombot --- Makefile.common | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/Makefile.common b/Makefile.common index d8b798909..4de21512f 100644 --- a/Makefile.common +++ b/Makefile.common @@ -62,6 +62,7 @@ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= GOLANGCI_LINT_VERSION ?= v2.1.5 +GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) @@ -156,9 +157,13 @@ $(GOTEST_DIR): @mkdir -p $@ .PHONY: common-format -common-format: +common-format: $(GOLANGCI_LINT) @echo ">> formatting code" $(GO) fmt $(pkgs) +ifdef GOLANGCI_LINT + @echo ">> formatting code with golangci-lint" + $(GOLANGCI_LINT) fmt $(GOLANGCI_FMT_OPTS) +endif .PHONY: common-vet common-vet: @@ -248,8 +253,8 @@ $(PROMU): cp $(PROMU_TMP)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM)/promu $(FIRST_GOPATH)/bin/promu rm -r $(PROMU_TMP) -.PHONY: proto -proto: +.PHONY: common-proto +common-proto: @echo ">> generating code from proto files" @./scripts/genproto.sh From 94e8399935e541262aaa2bf4e1723c232bea6c04 Mon Sep 17 00:00:00 2001 From: Jonathan Bowe Date: Thu, 29 May 2025 07:53:56 -0400 Subject: [PATCH 035/113] Feat: Improve Error Handling for Server.Scrape (#1158) * Update querySettings Error Return in Scrape If there are errors querying namespace mappings, the potential error from querySettings is obscured. Adding an immediate return if there are errors retreiving settings. Signed-off-by: Jonathan Bowe * Improve Verbosity of queryNamespaceMappings Errors Previously if any errors were encountered by queryNamespaceMappings, only a count of those errors was returned - making debugging those errors harder than it needs to be. I'm changing this to immediately return nil if no errors are encountered, and otherwise an error will be formatted with each of the namespaces and what the error was for that namespace. Signed-off-by: Jonathan Bowe * Simplify Error Message Co-authored-by: Ben Kochie Signed-off-by: Jonathan Bowe --------- Signed-off-by: Jonathan Bowe Signed-off-by: Jonathan Bowe Co-authored-by: Ben Kochie --- cmd/postgres_exporter/server.go | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/cmd/postgres_exporter/server.go b/cmd/postgres_exporter/server.go index bd4e76e10..3d2ecde91 100644 --- a/cmd/postgres_exporter/server.go +++ b/cmd/postgres_exporter/server.go @@ -119,12 +119,17 @@ func (s *Server) Scrape(ch chan<- prometheus.Metric, disableSettingsMetrics bool if !disableSettingsMetrics && s.master { if err = querySettings(ch, s); err != nil { err = fmt.Errorf("error retrieving settings: %s", err) + return err } } errMap := queryNamespaceMappings(ch, s) - if len(errMap) > 0 { - err = fmt.Errorf("queryNamespaceMappings returned %d errors", len(errMap)) + if len(errMap) == 0 { + return nil + } + err = fmt.Errorf("queryNamespaceMappings errors encountered") + for namespace, errStr := range errMap { + err = fmt.Errorf("%s, namespace: %s error: %s", err, namespace, errStr) } return err From d8ba628b1ecc5b45ef165c9bcfbe33c7be1b0a00 Mon Sep 17 00:00:00 2001 From: Peter Nuttall Date: Thu, 19 Jun 2025 05:33:22 +0000 Subject: [PATCH 036/113] Add a collector for `pg_buffercache_summary`. (#1165) * Add a collector for pg_buffercache_summary() * Requires PostgreSQL >= 16. --------- Signed-off-by: Peter Nuttall Co-authored-by: Ben Kochie --- collector/collector.go | 9 ++ collector/pg_buffercache_summary.go | 135 +++++++++++++++++++++++ collector/pg_buffercache_summary_test.go | 73 ++++++++++++ 3 files changed, 217 insertions(+) create mode 100644 collector/pg_buffercache_summary.go create mode 100644 collector/pg_buffercache_summary_test.go diff --git a/collector/collector.go b/collector/collector.go index 298bc36ee..84136e287 100644 --- a/collector/collector.go +++ b/collector/collector.go @@ -15,6 +15,7 @@ package collector import ( "context" + "database/sql" "errors" "fmt" "log/slog" @@ -228,3 +229,11 @@ var ErrNoData = errors.New("collector returned no data") func IsNoDataError(err error) bool { return err == ErrNoData } + +func Int32(m sql.NullInt32) float64 { + mM := 0.0 + if m.Valid { + mM = float64(m.Int32) + } + return mM +} diff --git a/collector/pg_buffercache_summary.go b/collector/pg_buffercache_summary.go new file mode 100644 index 000000000..8b0e0f007 --- /dev/null +++ b/collector/pg_buffercache_summary.go @@ -0,0 +1,135 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + "log/slog" + + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" +) + +const buffercacheSummarySubsystem = "buffercache_summary" + +func init() { + registerCollector(buffercacheSummarySubsystem, defaultDisabled, NewBuffercacheSummaryCollector) +} + +// BuffercacheSummaryCollector collects stats from pg_buffercache: https://www.postgresql.org/docs/current/pgbuffercache.html. +// +// It depends on the extension being loaded with +// +// create extension pg_buffercache; +// +// It does not take locks, see the PG docs above. +type BuffercacheSummaryCollector struct { + log *slog.Logger +} + +func NewBuffercacheSummaryCollector(config collectorConfig) (Collector, error) { + return &BuffercacheSummaryCollector{ + log: config.logger, + }, nil +} + +var ( + buffersUsedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, buffercacheSummarySubsystem, "buffers_used"), + "Number of used shared buffers", + []string{}, + prometheus.Labels{}, + ) + buffersUnusedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, buffercacheSummarySubsystem, "buffers_unused"), + "Number of unused shared buffers", + []string{}, + prometheus.Labels{}, + ) + buffersDirtyDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, buffercacheSummarySubsystem, "buffers_dirty"), + "Number of dirty shared buffers", + []string{}, + prometheus.Labels{}, + ) + buffersPinnedDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, buffercacheSummarySubsystem, "buffers_pinned"), + "Number of pinned shared buffers", + []string{}, + prometheus.Labels{}, + ) + usageCountAvgDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, buffercacheSummarySubsystem, "usagecount_avg"), + "Average usage count of used shared buffers", + []string{}, + prometheus.Labels{}, + ) + + buffercacheQuery = ` + SELECT + buffers_used, + buffers_unused, + buffers_dirty, + buffers_pinned, + usagecount_avg + FROM + pg_buffercache_summary() + ` +) + +// Update implements Collector +// It is called by the Prometheus registry when collecting metrics. +func (c BuffercacheSummaryCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + // pg_buffercache_summary is only in v16, and we don't need support for earlier currently. + if !instance.version.GE(semver.MustParse("16.0.0")) { + return nil + } + db := instance.getDB() + rows, err := db.QueryContext(ctx, buffercacheQuery) + if err != nil { + return err + } + defer rows.Close() + + var used, unused, dirty, pinned sql.NullInt32 + var usagecountAvg sql.NullFloat64 + + for rows.Next() { + if err := rows.Scan( + &used, + &unused, + &dirty, + &pinned, + &usagecountAvg, + ); err != nil { + return err + } + + usagecountAvgMetric := 0.0 + if usagecountAvg.Valid { + usagecountAvgMetric = usagecountAvg.Float64 + } + ch <- prometheus.MustNewConstMetric( + usageCountAvgDesc, + prometheus.GaugeValue, + usagecountAvgMetric) + ch <- prometheus.MustNewConstMetric(buffersUsedDesc, prometheus.GaugeValue, Int32(used)) + ch <- prometheus.MustNewConstMetric(buffersUnusedDesc, prometheus.GaugeValue, Int32(unused)) + ch <- prometheus.MustNewConstMetric(buffersDirtyDesc, prometheus.GaugeValue, Int32(dirty)) + ch <- prometheus.MustNewConstMetric(buffersPinnedDesc, prometheus.GaugeValue, Int32(pinned)) + } + + return rows.Err() +} diff --git a/collector/pg_buffercache_summary_test.go b/collector/pg_buffercache_summary_test.go new file mode 100644 index 000000000..86d91751f --- /dev/null +++ b/collector/pg_buffercache_summary_test.go @@ -0,0 +1,73 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestBuffercacheSummaryCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + columns := []string{ + "buffers_used", + "buffers_unused", + "buffers_dirty", + "buffers_pinned", + "usagecount_avg"} + + rows := sqlmock.NewRows(columns).AddRow(123, 456, 789, 234, 56.6778) + + mock.ExpectQuery(sanitizeQuery(buffercacheQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := BuffercacheSummaryCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 56.6778}, + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 123}, + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 456}, + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 789}, + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 234}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From f9a88f479b08611938dfb3b041a61e96fb582b33 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Thu, 19 Jun 2025 07:33:54 +0200 Subject: [PATCH 037/113] Update common Prometheus files (#1160) Signed-off-by: prombot --- .github/workflows/container_description.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/container_description.yml b/.github/workflows/container_description.yml index dcca16ff3..7de8bb8da 100644 --- a/.github/workflows/container_description.yml +++ b/.github/workflows/container_description.yml @@ -19,6 +19,8 @@ jobs: steps: - name: git checkout uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + persist-credentials: false - name: Set docker hub repo name run: echo "DOCKER_REPO_NAME=$(make docker-repo-name)" >> $GITHUB_ENV - name: Push README to Dockerhub @@ -41,6 +43,8 @@ jobs: steps: - name: git checkout uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + persist-credentials: false - name: Set quay.io org name run: echo "DOCKER_REPO=$(echo quay.io/${GITHUB_REPOSITORY_OWNER} | tr -d '-')" >> $GITHUB_ENV - name: Set quay.io repo name From 6d3078da3553bc1b522388e22fec01a314577cf4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 19 Jun 2025 07:34:12 +0200 Subject: [PATCH 038/113] Bump github.com/prometheus/common from 0.63.0 to 0.64.0 (#1163) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.63.0 to 0.64.0. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md) - [Commits](https://github.com/prometheus/common/compare/v0.63.0...v0.64.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.64.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 18 +++++++++--------- go.sum | 36 ++++++++++++++++++------------------ 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/go.mod b/go.mod index 9acdafc81..6f7fac66a 100644 --- a/go.mod +++ b/go.mod @@ -10,8 +10,8 @@ require ( github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.22.0 - github.com/prometheus/client_model v0.6.1 - github.com/prometheus/common v0.63.0 + github.com/prometheus/client_model v0.6.2 + github.com/prometheus/common v0.64.0 github.com/prometheus/exporter-toolkit v0.14.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -37,11 +37,11 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.36.0 // indirect - golang.org/x/net v0.38.0 // indirect - golang.org/x/oauth2 v0.25.0 // indirect - golang.org/x/sync v0.12.0 // indirect - golang.org/x/sys v0.31.0 // indirect - golang.org/x/text v0.23.0 // indirect - google.golang.org/protobuf v1.36.5 // indirect + golang.org/x/crypto v0.38.0 // indirect + golang.org/x/net v0.40.0 // indirect + golang.org/x/oauth2 v0.30.0 // indirect + golang.org/x/sync v0.14.0 // indirect + golang.org/x/sys v0.33.0 // indirect + golang.org/x/text v0.25.0 // indirect + google.golang.org/protobuf v1.36.6 // indirect ) diff --git a/go.sum b/go.sum index cd073f325..87b5b2bc6 100644 --- a/go.sum +++ b/go.sum @@ -52,10 +52,10 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q= github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= -github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= -github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= -github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k= -github.com/prometheus/common v0.63.0/go.mod h1:VVFF/fBIoToEnWRVkYoXEkq3R3paCoxG9PXP74SnV18= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.64.0 h1:pdZeA+g617P7oGv1CzdTzyeShxAGrTBsolKNOLQPGO4= +github.com/prometheus/common v0.64.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8= github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= @@ -73,20 +73,20 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34= -golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc= -golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= -golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= -golang.org/x/oauth2 v0.25.0 h1:CY4y7XT9v0cRI9oupztF8AgiIu99L/ksR/Xp/6jrZ70= -golang.org/x/oauth2 v0.25.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= -golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw= -golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= -golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik= -golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= -golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY= -golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4= -google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM= -google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= +golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= +golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= +golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY= +golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds= +golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI= +golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU= +golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ= +golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= +golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4= +golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA= +google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY= +google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From f50aac6d4593b895336a329328aee43e29b78eba Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 9 Jul 2025 11:55:58 +0200 Subject: [PATCH 039/113] Bump github.com/prometheus/common from 0.64.0 to 0.65.0 (#1173) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.64.0 to 0.65.0. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md) - [Commits](https://github.com/prometheus/common/compare/v0.64.0...v0.65.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.65.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 6f7fac66a..3ab61fb0c 100644 --- a/go.mod +++ b/go.mod @@ -11,7 +11,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.22.0 github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.64.0 + github.com/prometheus/common v0.65.0 github.com/prometheus/exporter-toolkit v0.14.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c diff --git a/go.sum b/go.sum index 87b5b2bc6..e5351b6f3 100644 --- a/go.sum +++ b/go.sum @@ -54,8 +54,8 @@ github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/ github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.64.0 h1:pdZeA+g617P7oGv1CzdTzyeShxAGrTBsolKNOLQPGO4= -github.com/prometheus/common v0.64.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8= +github.com/prometheus/common v0.65.0 h1:QDwzd+G1twt//Kwj/Ww6E9FQq1iVMmODnILtW1t2VzE= +github.com/prometheus/common v0.65.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8= github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= From 8c93ed018d886f33b355d70d1589c3cb4950f522 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Thu, 10 Jul 2025 18:46:03 +0200 Subject: [PATCH 040/113] Update common Prometheus files (#1174) Signed-off-by: prombot --- Makefile.common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index 4de21512f..6f61bec48 100644 --- a/Makefile.common +++ b/Makefile.common @@ -139,7 +139,7 @@ common-deps: update-go-deps: @echo ">> updating Go dependencies" @for m in $$($(GO) list -mod=readonly -m -f '{{ if and (not .Indirect) (not .Main)}}{{.Path}}{{end}}' all); do \ - $(GO) get -d $$m; \ + $(GO) get $$m; \ done $(GO) mod tidy From dfde566f2a306a5aeb5d82bccf34fc7f4c696b77 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Fri, 11 Jul 2025 09:31:12 +0200 Subject: [PATCH 041/113] chore: fix a typo and use `slices.Contains` (#1176) Minor fixes as I was skimming through: - fix typo in `stat_statements` - drop custom `sliceContains` in `database` Signed-off-by: Cristian Greco --- collector/pg_database.go | 12 ++---------- collector/pg_stat_statements.go | 4 ++-- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/collector/pg_database.go b/collector/pg_database.go index 4c0972080..7f98748f4 100644 --- a/collector/pg_database.go +++ b/collector/pg_database.go @@ -17,6 +17,7 @@ import ( "context" "database/sql" "log/slog" + "slices" "github.com/prometheus/client_golang/prometheus" ) @@ -102,7 +103,7 @@ func (c PGDatabaseCollector) Update(ctx context.Context, instance *instance, ch // Ignore excluded databases // Filtering is done here instead of in the query to avoid // a complicated NOT IN query with a variable number of parameters - if sliceContains(c.excludedDatabases, database) { + if slices.Contains(c.excludedDatabases, database) { continue } @@ -138,12 +139,3 @@ func (c PGDatabaseCollector) Update(ctx context.Context, instance *instance, ch } return rows.Err() } - -func sliceContains(slice []string, s string) bool { - for _, item := range slice { - if item == s { - return true - } - } - return false -} diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index 9160d3c16..d9a29ea65 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -64,7 +64,7 @@ func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { } var ( - statSTatementsCallsTotal = prometheus.NewDesc( + statStatementsCallsTotal = prometheus.NewDesc( prometheus.BuildFQName(namespace, statStatementsSubsystem, "calls_total"), "Number of times executed", []string{"user", "datname", "queryid"}, @@ -230,7 +230,7 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc callsTotalMetric = float64(callsTotal.Int64) } ch <- prometheus.MustNewConstMetric( - statSTatementsCallsTotal, + statStatementsCallsTotal, prometheus.CounterValue, callsTotalMetric, userLabel, datnameLabel, queryidLabel, From cac5d5220da23aff541679d107ea96d83e91145b Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 11 Jul 2025 20:35:36 +0200 Subject: [PATCH 042/113] Update common Prometheus files (#1178) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 8 +++++--- Makefile.common | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 672dd424d..d5d9ca2eb 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -25,15 +25,17 @@ jobs: steps: - name: Checkout repository uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + persist-credentials: false - name: Install Go - uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0 + uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 with: go-version: 1.24.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@1481404843c368bc19ca9406f87d6e0fc97bdcfd # v7.0.0 + uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0 with: args: --verbose - version: v2.1.5 + version: v2.2.1 diff --git a/Makefile.common b/Makefile.common index 6f61bec48..1f4c9025a 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.1.5 +GOLANGCI_LINT_VERSION ?= v2.2.1 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. From aa98bb30efc28f1070998d41e35ebe7fcd866072 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 16 Jul 2025 18:51:53 +0200 Subject: [PATCH 043/113] Replace another custom implementation of `slices.Contains` (#1180) Followup of https://github.com/prometheus-community/postgres_exporter/pull/1176 Signed-off-by: Cristian Greco --- cmd/postgres_exporter/datasource.go | 5 +++-- cmd/postgres_exporter/util.go | 9 --------- 2 files changed, 3 insertions(+), 11 deletions(-) diff --git a/cmd/postgres_exporter/datasource.go b/cmd/postgres_exporter/datasource.go index 7a22e177c..cebe5072e 100644 --- a/cmd/postgres_exporter/datasource.go +++ b/cmd/postgres_exporter/datasource.go @@ -18,6 +18,7 @@ import ( "net/url" "os" "regexp" + "slices" "strings" "github.com/prometheus/client_golang/prometheus" @@ -64,11 +65,11 @@ func (e *Exporter) discoverDatabaseDSNs() []string { continue } for _, databaseName := range databaseNames { - if contains(e.excludeDatabases, databaseName) { + if slices.Contains(e.excludeDatabases, databaseName) { continue } - if len(e.includeDatabases) != 0 && !contains(e.includeDatabases, databaseName) { + if len(e.includeDatabases) != 0 && !slices.Contains(e.includeDatabases, databaseName) { continue } diff --git a/cmd/postgres_exporter/util.go b/cmd/postgres_exporter/util.go index 8907e7c5f..fa692c652 100644 --- a/cmd/postgres_exporter/util.go +++ b/cmd/postgres_exporter/util.go @@ -24,15 +24,6 @@ import ( "github.com/lib/pq" ) -func contains(a []string, x string) bool { - for _, n := range a { - if x == n { - return true - } - } - return false -} - // convert a string to the corresponding ColumnUsage func stringToColumnUsage(s string) (ColumnUsage, error) { var u ColumnUsage From 06a553c8166512c9d9c5ccf257b0f9bba8751dbc Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Fri, 18 Jul 2025 20:53:45 +0200 Subject: [PATCH 044/113] Ensure database connections are always closed (#1177) Signed-off-by: Sam DeHaan Signed-off-by: Sam DeHaan Co-authored-by: Sam DeHaan Co-authored-by: Sam DeHaan --- cmd/postgres_exporter/server.go | 1 + collector/collector.go | 6 +++++- collector/probe.go | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/cmd/postgres_exporter/server.go b/cmd/postgres_exporter/server.go index 3d2ecde91..a90183cd7 100644 --- a/cmd/postgres_exporter/server.go +++ b/cmd/postgres_exporter/server.go @@ -173,6 +173,7 @@ func (s *Servers) GetServer(dsn string) (*Server, error) { s.servers[dsn] = server } if err = server.Ping(); err != nil { + server.Close() delete(s.servers, dsn) time.Sleep(time.Duration(errCount) * time.Second) continue diff --git a/collector/collector.go b/collector/collector.go index 84136e287..de7203486 100644 --- a/collector/collector.go +++ b/collector/collector.go @@ -173,11 +173,11 @@ func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) { // Set up the database connection for the collector. err := inst.setup() + defer inst.Close() if err != nil { p.logger.Error("Error opening connection to database", "err", err) return } - defer inst.Close() wg := sync.WaitGroup{} wg.Add(len(p.Collectors)) @@ -190,6 +190,10 @@ func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) { wg.Wait() } +func (p *PostgresCollector) Close() error { + return p.instance.Close() +} + func execute(ctx context.Context, name string, c Collector, instance *instance, ch chan<- prometheus.Metric, logger *slog.Logger) { begin := time.Now() err := c.Update(ctx, instance, ch) diff --git a/collector/probe.go b/collector/probe.go index 54a06261f..e40d6fee1 100644 --- a/collector/probe.go +++ b/collector/probe.go @@ -76,11 +76,11 @@ func (pc *ProbeCollector) Describe(ch chan<- *prometheus.Desc) { func (pc *ProbeCollector) Collect(ch chan<- prometheus.Metric) { // Set up the database connection for the collector. err := pc.instance.setup() + defer pc.instance.Close() if err != nil { pc.logger.Error("Error opening connection to database", "err", err) return } - defer pc.instance.Close() wg := sync.WaitGroup{} wg.Add(len(pc.collectors)) From 77e1a0d65a00bc0ec5120e27a0f372d03fec0055 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Fri, 25 Jul 2025 10:28:24 +0200 Subject: [PATCH 045/113] Update mixin to latest changes from grafana/postgres_exporter (#1179) * Update mixin to latest changes from grafana/postgres_exporter Porting a bunch of PRs accumulated over time in grafana/postgres_exporter: - https://github.com/grafana/postgres_exporter/pull/11 (@v-zhuravlev) - https://github.com/grafana/postgres_exporter/pull/12 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/13 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/14 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/15 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/16 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/17 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/20 (@gaantunes) - https://github.com/grafana/postgres_exporter/pull/21 (@mshahzeb) - https://github.com/grafana/postgres_exporter/pull/22 (@mshahzeb) Signed-off-by: Cristian Greco * rename dashboard to old name Signed-off-by: Cristian Greco * remove custom selector Signed-off-by: Cristian Greco --------- Signed-off-by: Cristian Greco Co-authored-by: Vitaly --- postgres_mixin/.lint | 15 + postgres_mixin/alerts/postgres.libsonnet | 189 ++- postgres_mixin/config.libsonnet | 4 + .../dashboards/postgres-overview.json | 1100 +++++++++-------- 4 files changed, 757 insertions(+), 551 deletions(-) create mode 100644 postgres_mixin/.lint diff --git a/postgres_mixin/.lint b/postgres_mixin/.lint new file mode 100644 index 000000000..65d621f69 --- /dev/null +++ b/postgres_mixin/.lint @@ -0,0 +1,15 @@ +--- +exclusions: + panel-units-rule: + reason: Ignoring so far, need to address this in future + panel-title-description-rule: + reason: Ignoring so far, need to address this in future + panel-datasource-rule: + reason: "Loki datasource variable is being named as loki_datasource now while linter expects 'datasource'" + template-datasource-rule: + reason: "Based on new convention we are using variable names prometheus_datasource and loki_datasource where as linter expects 'datasource'" + alert-name-camelcase: + reason: QPS is a common acronym (Queries Per Second) and should be allowed + entries: + - alert: PostgreSQLQPS + \ No newline at end of file diff --git a/postgres_mixin/alerts/postgres.libsonnet b/postgres_mixin/alerts/postgres.libsonnet index 4b0275df1..e3c8ee278 100644 --- a/postgres_mixin/alerts/postgres.libsonnet +++ b/postgres_mixin/alerts/postgres.libsonnet @@ -7,16 +7,16 @@ { alert: 'PostgreSQLMaxConnectionsReached', annotations: { - description: '{{ $labels.instance }} is exceeding the currently configured maximum Postgres connection limit (current value: {{ $value }}s). Services may be degraded - please take immediate action (you probably need to increase max_connections in the Docker image and re-deploy.', - summary: '{{ $labels.instance }} has maxed out Postgres connections.', + description: '{{ $labels.instance }} is exceeding the currently configured maximum Postgres connection limit (current value: {{ $value }}s). Services may be degraded - please take immediate action (you probably need to increase max_connections in the Docker image and re-deploy).', + summary: 'Postgres connections count is over the maximum amount.', }, expr: ||| - sum by (instance) (pg_stat_activity_count{%(postgresExporterSelector)s}) + sum by (%(agg)s) (pg_stat_activity_count{%(postgresExporterSelector)s}) >= - sum by (instance) (pg_settings_max_connections{%(postgresExporterSelector)s}) + sum by (%(agg)s) (pg_settings_max_connections{%(postgresExporterSelector)s}) - - sum by (instance) (pg_settings_superuser_reserved_connections{%(postgresExporterSelector)s}) - ||| % $._config, + sum by (%(agg)s) (pg_settings_superuser_reserved_connections{%(postgresExporterSelector)s}) + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, 'for': '1m', labels: { severity: 'warning', @@ -26,17 +26,17 @@ alert: 'PostgreSQLHighConnections', annotations: { description: '{{ $labels.instance }} is exceeding 80% of the currently configured maximum Postgres connection limit (current value: {{ $value }}s). Please check utilization graphs and confirm if this is normal service growth, abuse or an otherwise temporary condition or if new resources need to be provisioned (or the limits increased, which is mostly likely).', - summary: '{{ $labels.instance }} is over 80% of max Postgres connections.', + summary: 'Postgres connections count is over 80% of maximum amount.', }, expr: ||| - sum by (instance) (pg_stat_activity_count{%(postgresExporterSelector)s}) + sum by (%(agg)s) (pg_stat_activity_count{%(postgresExporterSelector)s}) > ( - sum by (instance) (pg_settings_max_connections{%(postgresExporterSelector)s}) + sum by (%(agg)s) (pg_settings_max_connections{%(postgresExporterSelector)s}) - - sum by (instance) (pg_settings_superuser_reserved_connections{%(postgresExporterSelector)s}) + sum by (%(agg)s) (pg_settings_superuser_reserved_connections{%(postgresExporterSelector)s}) ) * 0.8 - ||| % $._config, + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, 'for': '10m', labels: { severity: 'warning', @@ -46,7 +46,7 @@ alert: 'PostgreSQLDown', annotations: { description: '{{ $labels.instance }} is rejecting query requests from the exporter, and thus probably not allowing DNS requests to work either. User services should not be effected provided at least 1 node is still alive.', - summary: 'PostgreSQL is not processing queries: {{ $labels.instance }}', + summary: 'PostgreSQL is not processing queries.', }, expr: 'pg_up{%(postgresExporterSelector)s} != 1' % $._config, 'for': '1m', @@ -58,15 +58,15 @@ alert: 'PostgreSQLSlowQueries', annotations: { description: 'PostgreSQL high number of slow queries {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }} ', - summary: 'PostgreSQL high number of slow on {{ $labels.cluster }} for database {{ $labels.datname }} ', + summary: 'PostgreSQL high number of slow queries.', }, expr: ||| - avg by (datname) ( + avg by (datname, %(agg)s) ( rate ( - pg_stat_activity_max_tx_duration{datname!~"template.*",%(postgresExporterSelector)s}[2m] + pg_stat_activity_max_tx_duration{%(dbNameFilter)s, %(postgresExporterSelector)s}[2m] ) ) > 2 * 60 - ||| % $._config, + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, 'for': '2m', labels: { severity: 'warning', @@ -76,19 +76,19 @@ alert: 'PostgreSQLQPS', annotations: { description: 'PostgreSQL high number of queries per second on {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }}', - summary: 'PostgreSQL high number of queries per second {{ $labels.cluster }} for database {{ $labels.datname }}', + summary: 'PostgreSQL high number of queries per second.', }, expr: ||| - avg by (datname) ( + avg by (datname, %(agg)s) ( irate( - pg_stat_database_xact_commit{datname!~"template.*",%(postgresExporterSelector)s}[5m] + pg_stat_database_xact_commit{%(dbNameFilter)s, %(postgresExporterSelector)s}[5m] ) + irate( - pg_stat_database_xact_rollback{datname!~"template.*",%(postgresExporterSelector)s}[5m] + pg_stat_database_xact_rollback{%(dbNameFilter)s, %(postgresExporterSelector)s}[5m] ) ) > 10000 - ||| % $._config, + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, 'for': '5m', labels: { severity: 'warning', @@ -98,28 +98,165 @@ alert: 'PostgreSQLCacheHitRatio', annotations: { description: 'PostgreSQL low on cache hit rate on {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }}', - summary: 'PostgreSQL low cache hit rate on {{ $labels.cluster }} for database {{ $labels.datname }}', + summary: 'PostgreSQL low cache hit rate.', }, expr: ||| - avg by (datname) ( - rate(pg_stat_database_blks_hit{datname!~"template.*",%(postgresExporterSelector)s}[5m]) + avg by (datname, %(agg)s) ( + rate(pg_stat_database_blks_hit{%(dbNameFilter)s, %(postgresExporterSelector)s}[5m]) / ( rate( - pg_stat_database_blks_hit{datname!~"template.*",%(postgresExporterSelector)s}[5m] + pg_stat_database_blks_hit{%(dbNameFilter)s, %(postgresExporterSelector)s}[5m] ) + rate( - pg_stat_database_blks_read{datname!~"template.*",%(postgresExporterSelector)s}[5m] + pg_stat_database_blks_read{%(dbNameFilter)s, %(postgresExporterSelector)s}[5m] ) ) ) < 0.98 + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, + 'for': '5m', + labels: { + severity: 'warning', + }, + }, + { + alert: 'PostgresHasTooManyRollbacks', + annotations: { + description: 'PostgreSQL has too many rollbacks on {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }}', + summary: 'PostgreSQL has too many rollbacks.', + }, + expr: ||| + avg without(pod, instance) + (rate(pg_stat_database_xact_rollback{%(dbNameFilter)s}[5m]) / + (rate(pg_stat_database_xact_commit{%(dbNameFilter)s}[5m]) + rate(pg_stat_database_xact_rollback{%(dbNameFilter)s}[5m]))) > 0.10 + ||| % $._config, + 'for': '5m', + labels: { + severity: 'warning', + }, + }, + { + alert: 'PostgresHasHighDeadLocks', + annotations: { + description: 'PostgreSQL has too high deadlocks on {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }}', + summary: 'PostgreSQL has high number of deadlocks.', + }, + expr: ||| + max without(pod, instance) (rate(pg_stat_database_deadlocks{%(dbNameFilter)s}[5m]) * 60) > 5 ||| % $._config, 'for': '5m', labels: { severity: 'warning', }, }, + { + alert: 'PostgresAcquiredTooManyLocks', + annotations: { + description: 'PostgreSQL has acquired too many locks on {{ $labels.cluster }} for database {{ $labels.datname }} with a value of {{ $value }}', + summary: 'PostgreSQL has high number of acquired locks.', + }, + expr: ||| + max by(datname, %(agg)s) ( + (pg_locks_count{%(dbNameFilter)s}) + / + on(%(aggWithoutServer)s) group_left(server) ( + pg_settings_max_locks_per_transaction{} * pg_settings_max_connections{} + ) + ) > 0.20 + ||| % $._config { agg: std.join(',', $._config.groupLabels + $._config.instanceLabels), aggWithoutServer: std.join(',', std.filter(function(x) x != "server", $._config.groupLabels + $._config.instanceLabels)) }, + 'for': '5m', + labels: { + severity: 'warning', + }, + }, + { + alert: 'PostgresReplicationLaggingMore1Hour', + annotations: { + description: '{{ $labels.instance }} replication lag exceeds 1 hour. Check for network issues or load imbalances.', + summary: 'PostgreSQL replication lagging more than 1 hour.', + }, + expr: ||| + (pg_replication_lag{} > 3600) and on (%(agg)s) (pg_replication_is_replica{} == 1) + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, + 'for': '5m', + labels: { + severity: 'warning', + }, + }, + { + alert: 'PostgresHasReplicationSlotUsed', + annotations: { + description: '{{ $labels.instance }} has replication slots that are not used, which might lead to replication lag or data inconsistency.', + summary: 'PostgreSQL has unused replication slots.', + }, + expr: 'pg_replication_slots_active{} == 0', + 'for': '30m', + labels: { + severity: 'critical', + }, + }, + { + alert: 'PostgresReplicationRoleChanged', + annotations: { + description: '{{ $labels.instance }} replication role has changed. Verify if this is expected or if it indicates a failover.', + summary: 'PostgreSQL replication role change detected.', + }, + expr: 'pg_replication_is_replica{} and changes(pg_replication_is_replica{}[1m]) > 0', + labels: { + severity: 'warning', + }, + }, + { + alert: 'PostgresHasExporterErrors', + annotations: { + description: '{{ $labels.instance }} exporter is experiencing errors. Verify exporter health and configuration.', + summary: 'PostgreSQL exporter errors detected.', + }, + expr: 'pg_exporter_last_scrape_error{} > 0', + 'for': '30m', + labels: { + severity: 'critical', + }, + }, + { + alert: 'PostgresTablesNotVaccumed', + annotations: { + description: '{{ $labels.instance }} tables have not been vacuumed recently within the last hour, which may lead to performance degradation.', + summary: 'PostgreSQL tables not vacuumed.', + }, + expr: ||| + group without(pod, instance)( + timestamp( + pg_stat_user_tables_n_dead_tup{} > + pg_stat_user_tables_n_live_tup{} + * on(%(agg)s) group_left pg_settings_autovacuum_vacuum_scale_factor{} + + on(%(agg)s) group_left pg_settings_autovacuum_vacuum_threshold{} + ) + < time() - 36000 + ) + ||| % $._config { agg: std.join(', ', $._config.groupLabels + $._config.instanceLabels) }, + 'for': '30m', + labels: { + severity: 'critical', + }, + }, + { + alert: 'PostgresTooManyCheckpointsRequested', + annotations: { + description: '{{ $labels.instance }} is requesting too many checkpoints, which may lead to performance degradation.', + summary: 'PostgreSQL too many checkpoints requested.', + }, + expr: ||| + rate(pg_stat_bgwriter_checkpoints_timed_total{}[5m]) / + (rate(pg_stat_bgwriter_checkpoints_timed_total{}[5m]) + rate(pg_stat_bgwriter_checkpoints_req_total{}[5m])) + < 0.5 + |||, + 'for': '5m', + labels: { + severity: 'warning', + }, + }, ], }, ], diff --git a/postgres_mixin/config.libsonnet b/postgres_mixin/config.libsonnet index d7bd7ac1b..4ea3b7e52 100644 --- a/postgres_mixin/config.libsonnet +++ b/postgres_mixin/config.libsonnet @@ -1,5 +1,9 @@ { _config+:: { + dbNameFilter: 'datname!~"template.*"', postgresExporterSelector: '', + groupLabels: if self.enableMultiCluster then ['job', 'cluster'] else ['job'], + instanceLabels: ['instance', 'server'], + enableMultiCluster: false, }, } diff --git a/postgres_mixin/dashboards/postgres-overview.json b/postgres_mixin/dashboards/postgres-overview.json index 9bf41be6a..8baf6fbb3 100644 --- a/postgres_mixin/dashboards/postgres-overview.json +++ b/postgres_mixin/dashboards/postgres-overview.json @@ -3,81 +3,119 @@ "list": [ { "builtIn": 1, - "datasource": "-- Grafana --", + "datasource": { + "type": "datasource", + "uid": "grafana" + }, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, "type": "dashboard" } ] }, "description": "Performance metrics for Postgres", "editable": true, + "fiscalYearStartMonth": 0, "gnetId": 455, "graphTooltip": 0, - "id": 1, - "iteration": 1603191461722, + "id": 38, "links": [], + "liveNow": false, "panels": [ { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "$datasource", - "editable": true, - "error": false, + "collapse": false, + "collapsed": false, + "gridPos": { + "h": 2, + "w": 8, + "x": 0, + "y": 0 + }, + "panels": [ ], + "repeat": null, + "repeatIteration": null, + "repeatRowId": null, + "showTitle": true, + "span": 4, + "title": "PostgreSQL overview", + "titleSize": "h6", + "type": "row" + }, + { + + "datasource": { + "uid": "$datasource" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "fixedColor": "rgb(31, 120, 193)", + "mode": "fixed" + }, + "decimals": 0, + "mappings": [ + { + "options": { + "match": "null", + "result": { + "text": "N/A" + } + }, + "type": "special" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "none" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, - "grid": {}, "gridPos": { "h": 7, - "w": 20, + "w": 4, "x": 0, "y": 0 }, - "hiddenSeries": false, - "id": 1, - "isNew": true, - "legend": { - "alignAsTable": true, - "avg": true, - "current": false, - "max": true, - "min": true, - "rightSide": true, - "show": true, - "total": false, - "values": true - }, - "lines": true, - "linewidth": 1, + "id": 11, "links": [], - "nullPointMode": "connected", + "maxDataPoints": 100, "options": { - "alertThreshold": true + "colorMode": "none", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "horizontal", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "textMode": "auto" }, - "percentage": false, - "pluginVersion": "7.2.1", - "pointradius": 5, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "pluginVersion": "9.4.3", "targets": [ { - "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_fetched{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_xact_commit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum(irate(pg_stat_database_xact_rollback{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -94,7 +132,6 @@ } ], "intervalFactor": 2, - "legendFormat": "fetched", "measurement": "postgresql", "policy": "default", "refId": "A", @@ -103,7 +140,7 @@ [ { "params": [ - "tup_fetched" + "xact_commit" ], "type": "field" }, @@ -119,7 +156,7 @@ } ] ], - "step": 120, + "step": 1800, "tags": [ { "key": "instance", @@ -127,11 +164,101 @@ "value": "/^$instance$/" } ] + } + ], + "title": "QPS", + "type": "stat" + }, + { + "datasource": { + "uid": "$datasource" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 7, + "w": 20, + "x": 4, + "y": 0 + }, + "id": 1, + "links": [], + "options": { + "legend": { + "calcs": [ + "mean", + "max", + "min" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "9.4.3", + "targets": [ { "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_returned{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_tup_fetched{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -148,10 +275,10 @@ } ], "intervalFactor": 2, - "legendFormat": "returned", + "legendFormat": "fetched", "measurement": "postgresql", "policy": "default", - "refId": "B", + "refId": "A", "resultFormat": "time_series", "select": [ [ @@ -184,8 +311,11 @@ }, { "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_inserted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_tup_returned{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -202,10 +332,10 @@ } ], "intervalFactor": 2, - "legendFormat": "inserted", + "legendFormat": "returned", "measurement": "postgresql", "policy": "default", - "refId": "C", + "refId": "B", "resultFormat": "time_series", "select": [ [ @@ -238,8 +368,11 @@ }, { "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_updated{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_tup_inserted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -256,10 +389,10 @@ } ], "intervalFactor": 2, - "legendFormat": "updated", + "legendFormat": "inserted", "measurement": "postgresql", "policy": "default", - "refId": "D", + "refId": "C", "resultFormat": "time_series", "select": [ [ @@ -292,8 +425,11 @@ }, { "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_deleted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_tup_updated{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -310,10 +446,10 @@ } ], "intervalFactor": 2, - "legendFormat": "deleted", + "legendFormat": "updated", "measurement": "postgresql", "policy": "default", - "refId": "E", + "refId": "D", "resultFormat": "time_series", "select": [ [ @@ -343,124 +479,14 @@ "value": "/^$instance$/" } ] - } - ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, - "title": "Rows", - "tooltip": { - "msResolution": true, - "shared": true, - "sort": 0, - "value_type": "cumulative" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true }, { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } - }, - { - "cacheTimeout": null, - "colorBackground": false, - "colorValue": false, - "colors": [ - "rgba(245, 54, 54, 0.9)", - "rgba(237, 129, 40, 0.89)", - "rgba(50, 172, 45, 0.97)" - ], - "datasource": "$datasource", - "decimals": 0, - "editable": true, - "error": false, - "fieldConfig": { - "defaults": { - "custom": {} - }, - "overrides": [] - }, - "format": "none", - "gauge": { - "maxValue": 100, - "minValue": 0, - "show": false, - "thresholdLabels": false, - "thresholdMarkers": true - }, - "gridPos": { - "h": 3, - "w": 4, - "x": 20, - "y": 0 - }, - "height": "55px", - "id": 11, - "interval": null, - "isNew": true, - "links": [], - "mappingType": 1, - "mappingTypes": [ - { - "name": "value to text", - "value": 1 - }, - { - "name": "range to text", - "value": 2 - } - ], - "maxDataPoints": 100, - "nullPointMode": "connected", - "nullText": null, - "postfix": "", - "postfixFontSize": "50%", - "prefix": "", - "prefixFontSize": "50%", - "rangeMaps": [ - { - "from": "null", - "text": "N/A", - "to": "null" - } - ], - "sparkline": { - "fillColor": "rgba(31, 118, 189, 0.18)", - "full": true, - "lineColor": "rgb(31, 120, 193)", - "show": true - }, - "tableColumn": "", - "targets": [ - { + "alias": "fetched", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_xact_commit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum(irate(pg_stat_database_xact_rollback{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(irate(pg_stat_database_tup_deleted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -477,15 +503,16 @@ } ], "intervalFactor": 2, + "legendFormat": "deleted", "measurement": "postgresql", "policy": "default", - "refId": "A", + "refId": "E", "resultFormat": "time_series", "select": [ [ { "params": [ - "xact_commit" + "tup_fetched" ], "type": "field" }, @@ -501,7 +528,7 @@ } ] ], - "step": 1800, + "step": 120, "tags": [ { "key": "instance", @@ -511,80 +538,100 @@ ] } ], - "thresholds": "", - "title": "QPS", - "transparent": true, - "type": "singlestat", - "valueFontSize": "80%", - "valueMaps": [ - { - "op": "=", - "text": "N/A", - "value": "null" - } - ], - "valueName": "avg" + "title": "Rows", + "type": "timeseries" }, { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "$datasource", - "decimals": 1, - "editable": true, - "error": false, + "datasource": { + "uid": "$datasource" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, - "grid": {}, "gridPos": { "h": 7, "w": 12, "x": 0, "y": 7 }, - "hiddenSeries": false, "id": 2, - "isNew": true, - "legend": { - "alignAsTable": true, - "avg": true, - "current": false, - "hideZero": true, - "max": true, - "min": true, - "rightSide": false, - "show": true, - "total": false, - "values": true - }, - "lines": true, - "linewidth": 1, "links": [], - "nullPointMode": "connected", "options": { - "alertThreshold": true + "legend": { + "calcs": [ + "mean", + "max", + "min" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } }, - "percentage": false, - "pluginVersion": "7.2.1", - "pointradius": 5, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "pluginVersion": "9.4.3", "targets": [ { "alias": "Buffers Allocated", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_alloc{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", + "expr": "irate(pg_stat_bgwriter_buffers_alloc_total{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -635,8 +682,11 @@ }, { "alias": "Buffers Allocated", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_backend_fsync{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", + "expr": "irate(pg_stat_bgwriter_buffers_backend_fsync_total{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -687,8 +737,11 @@ }, { "alias": "Buffers Allocated", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_backend{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", + "expr": "irate(pg_stat_bgwriter_buffers_backend_total{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -739,8 +792,11 @@ }, { "alias": "Buffers Allocated", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_clean{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", + "expr": "irate(pg_stat_bgwriter_buffers_clean_total{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -791,8 +847,11 @@ }, { "alias": "Buffers Allocated", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_checkpoint{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", + "expr": "irate(pg_stat_bgwriter_buffers_checkpoint_total{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -842,104 +901,113 @@ ] } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, "title": "Buffers", - "tooltip": { - "msResolution": false, - "shared": true, - "sort": 0, - "value_type": "cumulative" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": 0, - "show": true - }, - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } + "type": "timeseries" }, { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "$datasource", - "editable": true, - "error": false, + "datasource": { + "type": "prometheus", + "uid": "$datasource" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" }, - "overrides": [] + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "deadlocks" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "light-red", + "mode": "fixed" + } + } + ] + } + ] }, - "fill": 1, - "fillGradient": 0, - "grid": {}, "gridPos": { "h": 7, "w": 12, "x": 12, "y": 7 }, - "hiddenSeries": false, "id": 3, - "isNew": true, - "legend": { - "avg": false, - "current": false, - "max": false, - "min": false, - "show": true, - "total": false, - "values": false - }, - "lines": true, - "linewidth": 2, "links": [], - "nullPointMode": "connected", "options": { - "alertThreshold": true + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } }, - "percentage": false, - "pluginVersion": "7.2.1", - "pointradius": 5, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "pluginVersion": "9.4.3", "targets": [ { "alias": "conflicts", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(rate(pg_stat_database_deadlocks{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "expr": "sum(pg_stat_database_deadlocks{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"})", "format": "time_series", "groupBy": [ { @@ -959,6 +1027,7 @@ "legendFormat": "deadlocks", "measurement": "postgresql", "policy": "default", + "range": true, "refId": "A", "resultFormat": "time_series", "select": [ @@ -990,8 +1059,12 @@ }, { "alias": "deadlocks", + "datasource": { + "uid": "$datasource" + }, "dsType": "prometheus", - "expr": "sum(rate(pg_stat_database_conflicts{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", + "editorMode": "code", + "expr": "sum(pg_stat_database_conflicts{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"})", "format": "time_series", "groupBy": [ { @@ -1011,6 +1084,7 @@ "legendFormat": "conflicts", "measurement": "postgresql", "policy": "default", + "range": true, "refId": "B", "resultFormat": "time_series", "select": [ @@ -1041,204 +1115,187 @@ ] } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, "title": "Conflicts/Deadlocks", - "tooltip": { - "msResolution": false, - "shared": true, - "sort": 0, - "value_type": "cumulative" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": 0, - "show": true - }, - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } + "type": "timeseries" }, { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "$datasource", - "editable": true, - "error": false, + "datasource": { + "uid": "$datasource" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, - "grid": {}, "gridPos": { "h": 7, "w": 12, "x": 0, "y": 14 }, - "hiddenSeries": false, "id": 12, - "isNew": true, - "legend": { - "avg": false, - "current": false, - "max": false, - "min": false, - "show": true, - "total": false, - "values": false - }, - "lines": true, - "linewidth": 2, "links": [], - "nullPointMode": "connected", "options": { - "alertThreshold": true + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } }, - "percentage": true, - "pluginVersion": "7.2.1", - "pointradius": 1, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "pluginVersion": "9.4.3", "targets": [ { - "expr": "sum by (datname) (rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) / (sum by (datname)(rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum by (datname)(rate(pg_stat_database_blks_read{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])))", + "datasource": { + "uid": "$datasource" + }, + "expr": "round(sum by (datname) (rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) / (sum by (datname)(rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum by (datname)(rate(pg_stat_database_blks_read{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])))*100,0.001)", "format": "time_series", - "intervalFactor": 2, "legendFormat": "{{datname}} - cache hit rate", "refId": "A", "step": 240 } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, "title": "Cache hit ratio", - "tooltip": { - "msResolution": false, - "shared": true, - "sort": 0, - "value_type": "cumulative" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "percentunit", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - }, - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } + "type": "timeseries" }, { - "aliasColors": {}, - "bars": false, - "dashLength": 10, - "dashes": false, - "datasource": "$datasource", - "editable": true, - "error": false, + "datasource": { + "uid": "$datasource" + }, "fieldConfig": { "defaults": { - "custom": {} + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" }, "overrides": [] }, - "fill": 1, - "fillGradient": 0, - "grid": {}, "gridPos": { "h": 7, "w": 12, "x": 12, "y": 14 }, - "hiddenSeries": false, "id": 13, - "isNew": true, - "legend": { - "avg": false, - "current": false, - "max": false, - "min": false, - "show": true, - "total": false, - "values": false - }, - "lines": true, - "linewidth": 2, "links": [], - "nullPointMode": "connected", "options": { - "alertThreshold": true + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } }, - "percentage": false, - "pluginVersion": "7.2.1", - "pointradius": 5, - "points": false, - "renderer": "flot", - "seriesOverrides": [], - "spaceLength": 10, - "stack": false, - "steppedLine": false, + "pluginVersion": "9.4.3", "targets": [ { + "datasource": { + "uid": "$datasource" + }, "expr": "pg_stat_database_numbackends{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}", "format": "time_series", "intervalFactor": 2, @@ -1247,51 +1304,13 @@ "step": 240 } ], - "thresholds": [], - "timeFrom": null, - "timeRegions": [], - "timeShift": null, "title": "Number of active connections", - "tooltip": { - "msResolution": false, - "shared": true, - "sort": 0, - "value_type": "cumulative" - }, - "type": "graph", - "xaxis": { - "buckets": null, - "mode": "time", - "name": null, - "show": true, - "values": [] - }, - "yaxes": [ - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - }, - { - "format": "short", - "label": null, - "logBase": 1, - "max": null, - "min": null, - "show": true - } - ], - "yaxis": { - "align": false, - "alignLevel": null - } + "type": "timeseries" } ], "refresh": false, - "schemaVersion": 26, + "revision": 1, + "schemaVersion": 38, "style": "dark", "tags": [ "postgres" @@ -1310,66 +1329,96 @@ "regex": "", "skipUrlSync": false, "type": "datasource" - }, + }, { "allValue": ".+", - "datasource": "$datasource", + "current": { + "selected": true, + "text": [ + "All" + ], + "value": [ + "$__all" + ] + }, + "datasource": { + "uid": "$datasource" + }, "definition": "label_values(pg_up, job)", "hide": 0, "includeAll": true, - "label": "job", + "label": "Job", "multi": true, "name": "job", "options": [], - "query": "label_values(pg_up, job)", - "refresh": 0, + "query": { + "query": "label_values(pg_up, job)", + "refId": "StandardVariableQuery" + }, + "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", - "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "allValue": ".+", - "datasource": "$datasource", - "definition": "", + "current": { + "selected": true, + "text": [ + "All" + ], + "value": [ + "$__all" + ] + }, + "datasource": { + "uid": "$datasource" + }, + "definition": "label_values(pg_up{job=~\"$job\"},instance)", "hide": 0, "includeAll": true, - "label": "instance", + "label": "Instance", "multi": true, "name": "instance", "options": [], - "query": "label_values(up{job=~\"$job\"},instance)", - "refresh": 1, + "query": { + "query": "label_values(pg_up{job=~\"$job\"},instance)", + "refId": "StandardVariableQuery" + }, + "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", - "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "allValue": ".+", - "datasource": "$datasource", - "definition": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", + "datasource": { + "uid": "$datasource" + }, + "definition": "label_values(pg_stat_database_tup_fetched{job=~\"$job\",instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", "hide": 0, "includeAll": true, - "label": "db", + "label": "Database", "multi": false, "name": "db", "options": [], - "query": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", - "refresh": 1, + "query": { + "query": "label_values(pg_stat_database_tup_fetched{job=~\"$job\",instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", + "refId": "StandardVariableQuery" + }, + "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", - "tags": [], "tagsQuery": "", "type": "query", "useTags": false @@ -1408,5 +1457,6 @@ "timezone": "browser", "title": "Postgres Overview", "uid": "wGgaPlciz", - "version": 5 -} + "version": 39, + "weekStart": "" +} \ No newline at end of file From cb0bac60e13efdcb9cfbad3c6de283e3ef8e23be Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 12 Aug 2025 22:35:52 -0400 Subject: [PATCH 046/113] Bump github.com/prometheus/client_golang from 1.22.0 to 1.23.0 (#1183) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.22.0...v1.23.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 4 ++-- go.sum | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/go.mod b/go.mod index 3ab61fb0c..3127f069e 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.22.0 + github.com/prometheus/client_golang v1.23.0 github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.65.0 github.com/prometheus/exporter-toolkit v0.14.0 @@ -33,7 +33,7 @@ require ( github.com/mdlayher/vsock v1.2.1 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f // indirect - github.com/prometheus/procfs v0.15.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect diff --git a/go.sum b/go.sum index e5351b6f3..5a44251f2 100644 --- a/go.sum +++ b/go.sum @@ -50,16 +50,16 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q= -github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= +github.com/prometheus/client_golang v1.23.0 h1:ust4zpdl9r4trLY/gSjlm07PuiBq2ynaXXlptpfy8Uc= +github.com/prometheus/client_golang v1.23.0/go.mod h1:i/o0R9ByOnHX0McrTMTyhYvKE4haaf2mW08I+jGAjEE= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.65.0 h1:QDwzd+G1twt//Kwj/Ww6E9FQq1iVMmODnILtW1t2VzE= github.com/prometheus/common v0.65.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8= github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= -github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= -github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= @@ -73,6 +73,8 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY= From 198454cc9e56141d5cc422149755fe8e80b3eeea Mon Sep 17 00:00:00 2001 From: Kaarel Moppel Date: Tue, 26 Aug 2025 04:05:25 +0300 Subject: [PATCH 047/113] Exclude the metrics fetching session's data from pg_stat_activity (#1185) To reduce the observer effect, filter out pg_stat_activity rows of the postgres_exporter session from all SA queries, based on pid / procpid. A bit annoying to see idling DBs showing pg_stat_activity_count "active" count of 1 Signed-off-by: Kaarel Moppel --- cmd/postgres_exporter/queries.go | 5 ++++- collector/pg_long_running_transactions.go | 3 ++- collector/pg_process_idle.go | 2 ++ 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/cmd/postgres_exporter/queries.go b/cmd/postgres_exporter/queries.go index 80be72d54..c0081fe02 100644 --- a/cmd/postgres_exporter/queries.go +++ b/cmd/postgres_exporter/queries.go @@ -142,6 +142,7 @@ var queryOverrides = map[string][]OverrideQuery{ count(*) AS count, MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration FROM pg_stat_activity + WHERE pid <> pg_backend_pid() GROUP BY datname,state,usename,application_name,backend_type,wait_event_type,wait_event) AS tmp2 ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname `, @@ -156,7 +157,9 @@ var queryOverrides = map[string][]OverrideQuery{ application_name, COALESCE(count(*),0) AS count, COALESCE(MAX(EXTRACT(EPOCH FROM now() - xact_start))::float,0) AS max_tx_duration - FROM pg_stat_activity GROUP BY datname,usename,application_name + FROM pg_stat_activity + WHERE procpid <> pg_backend_pid() + GROUP BY datname,usename,application_name `, }, }, diff --git a/collector/pg_long_running_transactions.go b/collector/pg_long_running_transactions.go index d7d1e6d30..072862f4e 100644 --- a/collector/pg_long_running_transactions.go +++ b/collector/pg_long_running_transactions.go @@ -56,7 +56,8 @@ var ( FROM pg_catalog.pg_stat_activity WHERE state IS DISTINCT FROM 'idle' AND query NOT LIKE 'autovacuum:%' -AND pg_stat_activity.xact_start IS NOT NULL; +AND pg_stat_activity.xact_start IS NOT NULL +AND pid <> pg_backend_pid(); ` ) diff --git a/collector/pg_process_idle.go b/collector/pg_process_idle.go index 7f3ff6f0f..1f9658aa6 100644 --- a/collector/pg_process_idle.go +++ b/collector/pg_process_idle.go @@ -56,6 +56,7 @@ func (PGProcessIdleCollector) Update(ctx context.Context, instance *instance, ch COUNT(*) AS process_idle_seconds_count FROM pg_stat_activity WHERE state ~ '^idle' + AND pid <> pg_backend_pid(); GROUP BY state, application_name ), buckets AS ( @@ -72,6 +73,7 @@ func (PGProcessIdleCollector) Update(ctx context.Context, instance *instance, ch FROM pg_stat_activity, UNNEST(ARRAY[1, 2, 5, 15, 30, 60, 90, 120, 300]) AS le + WHERE pid <> pg_backend_pid() GROUP BY state, application_name, le ORDER BY state, application_name, le ) From 1c6854ece5c8f74a5739a9b3dc97229841f932d4 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sat, 20 Sep 2025 08:40:37 +0200 Subject: [PATCH 048/113] Update common Prometheus files (#1191) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index d5d9ca2eb..1816a556b 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -30,7 +30,7 @@ jobs: - name: Install Go uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 with: - go-version: 1.24.x + go-version: 1.25.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' From 2850c3d988496983cd168fe9b5b064c4a11e28ae Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 23 Sep 2025 03:52:30 +0200 Subject: [PATCH 049/113] Update common Prometheus files (#1194) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 5 ++++- Makefile.common | 6 +++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 1816a556b..79891553f 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -34,8 +34,11 @@ jobs: - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' + - name: Get golangci-lint version + id: golangci-lint-version + run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT - name: Lint uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0 with: args: --verbose - version: v2.2.1 + version: ${{ steps.golangci-lint-version.outputs.version }} diff --git a/Makefile.common b/Makefile.common index 1f4c9025a..6762d0f83 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.2.1 +GOLANGCI_LINT_VERSION ?= v2.4.0 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. @@ -266,6 +266,10 @@ $(GOLANGCI_LINT): | sh -s -- -b $(FIRST_GOPATH)/bin $(GOLANGCI_LINT_VERSION) endif +.PHONY: common-print-golangci-lint-version +common-print-golangci-lint-version: + @echo $(GOLANGCI_LINT_VERSION) + .PHONY: precheck precheck:: From 105c422dfb3222940ced0b00023225866a86ebee Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Tue, 23 Sep 2025 20:38:16 +0200 Subject: [PATCH 050/113] Bump prometheus deps (#1195) - Update github.com/prometheus/client_golang to v1.23.1 - Update github.com/prometheus/common to v0.66.1 - Update github.com/prometheus/exporter-toolkit to v0.14.1 Signed-off-by: Cristian Greco --- go.mod | 21 +++++++++++---------- go.sum | 47 ++++++++++++++++++++++++----------------------- 2 files changed, 35 insertions(+), 33 deletions(-) diff --git a/go.mod b/go.mod index 3127f069e..02a1fec5f 100644 --- a/go.mod +++ b/go.mod @@ -9,10 +9,10 @@ require ( github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.23.0 + github.com/prometheus/client_golang v1.23.1 github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.65.0 - github.com/prometheus/exporter-toolkit v0.14.0 + github.com/prometheus/common v0.66.1 + github.com/prometheus/exporter-toolkit v0.14.1 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -23,7 +23,7 @@ require ( github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/coreos/go-systemd/v22 v22.5.0 // indirect + github.com/coreos/go-systemd/v22 v22.6.0 // indirect github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/jtolds/gls v4.20.0+incompatible // indirect @@ -37,11 +37,12 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - golang.org/x/crypto v0.38.0 // indirect - golang.org/x/net v0.40.0 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/crypto v0.41.0 // indirect + golang.org/x/net v0.43.0 // indirect golang.org/x/oauth2 v0.30.0 // indirect - golang.org/x/sync v0.14.0 // indirect - golang.org/x/sys v0.33.0 // indirect - golang.org/x/text v0.25.0 // indirect - google.golang.org/protobuf v1.36.6 // indirect + golang.org/x/sync v0.16.0 // indirect + golang.org/x/sys v0.35.0 // indirect + golang.org/x/text v0.28.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect ) diff --git a/go.sum b/go.sum index 5a44251f2..7feb5da55 100644 --- a/go.sum +++ b/go.sum @@ -10,13 +10,12 @@ github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs= -github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= +github.com/coreos/go-systemd/v22 v22.6.0 h1:aGVa/v8B7hpb0TKl0MWoAavPDmHvobFe5R5zn0bCJWo= +github.com/coreos/go-systemd/v22 v22.6.0/go.mod h1:iG+pp635Fo7ZmV/j14KUcmEyWF+0X7Lua8rrTWzYgWU= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= @@ -50,14 +49,14 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.23.0 h1:ust4zpdl9r4trLY/gSjlm07PuiBq2ynaXXlptpfy8Uc= -github.com/prometheus/client_golang v1.23.0/go.mod h1:i/o0R9ByOnHX0McrTMTyhYvKE4haaf2mW08I+jGAjEE= +github.com/prometheus/client_golang v1.23.1 h1:w6gXMLQGgd0jXXlote9lRHMe0nG01EbnJT+C0EJru2Y= +github.com/prometheus/client_golang v1.23.1/go.mod h1:br8j//v2eg2K5Vvna5klK8Ku5pcU5r4ll73v6ik5dIQ= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.65.0 h1:QDwzd+G1twt//Kwj/Ww6E9FQq1iVMmODnILtW1t2VzE= -github.com/prometheus/common v0.65.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8= -github.com/prometheus/exporter-toolkit v0.14.0 h1:NMlswfibpcZZ+H0sZBiTjrA3/aBFHkNZqE+iCj5EmRg= -github.com/prometheus/exporter-toolkit v0.14.0/go.mod h1:Gu5LnVvt7Nr/oqTBUC23WILZepW0nffNo10XdhQcwWA= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/exporter-toolkit v0.14.1 h1:uKPE4ewweVRWFainwvAcHs3uw15pjw2dk3I7b+aNo9o= +github.com/prometheus/exporter-toolkit v0.14.1/go.mod h1:di7yaAJiaMkcjcz48f/u4yRPwtyuxTU5Jr4EnM2mhtQ= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -69,26 +68,28 @@ github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sS github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= -github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= -golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= -golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY= -golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4= +golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc= +golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE= +golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg= golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI= golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU= -golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ= -golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= -golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= -golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= -golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4= -golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA= -google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY= -google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY= +golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= +golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng= +golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From e62fe086f0dc5a0866d4af494d34e0e3de0b16d7 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Thu, 25 Sep 2025 21:37:41 +0200 Subject: [PATCH 051/113] Prepare release `0.18.0` (#1196) * Prepare release `0.18.0` * [FEATURE] Add `stat_progress_vacuum` collector by @ianbibby * [FEATURE] Add `buffercache_summary` collector by @sfc-gh-pnuttall * [FEATURE] `stat_statements`: export query itself together with `queryId` by @Delorien84 * [ENHANCEMENT] Update Go version by @SuperQ * [ENHANCEMENT] Improve error handling for `Server.Scrape` by @BoweFlex * [ENHANCEMENT] `stat_user_tables`: record table-only size bytes in addition to the total size bytes by @Sticksman * [ENHANCEMENT] (chore) Fix a typo and use `slices.Contains` by @cristiangreco * [ENHANCEMENT] Update mixin to latest changes from `grafana/postgres_exporter` by @cristiangreco, @gaantunes, @v-zhuravlev and @mshahzeb * [ENHANCEMENT] Exclude the metrics fetching session's data from pg_stat_activity by @kmoppel * [BUGFIX] Ensure database connections are always closed by @cristiangreco and @dehaansa Signed-off-by: Cristian Greco * rm entry about go update Co-authored-by: Joe Adams Signed-off-by: Cristian Greco * add PRs Signed-off-by: Cristian Greco * update date Signed-off-by: Cristian Greco --------- Signed-off-by: Cristian Greco Co-authored-by: Joe Adams --- CHANGELOG.md | 20 ++++++++++++++++++++ VERSION | 2 +- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d01ceb504..4402fb354 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,23 @@ +## main / (unreleased) + +* [CHANGE] ... +* [FEATURE] ... +* [ENHANCEMENT] ... +* [BUGFIX] ... + +## 0.18.0 / 2025-09-25 + +* [FEATURE] Add `stat_progress_vacuum` collector by @ianbibby in https://github.com/prometheus-community/postgres_exporter/pull/1141 +* [FEATURE] Add `buffercache_summary` collector by @sfc-gh-pnuttall in https://github.com/prometheus-community/postgres_exporter/pull/1165 +* [FEATURE] `stat_statements`: export query itself together with `queryId` by @Delorien84 in https://github.com/prometheus-community/postgres_exporter/pull/940 +* [ENHANCEMENT] Improve error handling for `Server.Scrape` by @BoweFlex in https://github.com/prometheus-community/postgres_exporter/pull/1158 +* [ENHANCEMENT] `stat_user_tables`: record table-only size bytes in addition to the total size bytes by @Sticksman Sticksman in https://github.com/prometheus-community/postgres_exporter/pull/1149 +* [ENHANCEMENT] (chore) Fix a typo and use `slices.Contains` by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1176 +* [ENHANCEMENT] Update mixin to latest changes from `grafana/postgres_exporter` by @cristiangreco, @gaantunes, @v-zhuravlev and @mshahzeb in https://github.com/prometheus-community/postgres_exporter/pull/1179 +* [ENHANCEMENT] Exclude the metrics fetching session's data from pg_stat_activity by @kmoppel in https://github.com/prometheus-community/postgres_exporter/pull/1185 +* [BUGFIX] Ensure database connections are always closed by @cristiangreco and @dehaansa in https://github.com/prometheus-community/postgres_exporter/pull/1177 +* [BUGFIX] Fix superfluous semicolon breaking query in `process_idle` by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1197 + ## 0.17.1 / 2025-02-26 * [BUGFIX] Fix: Handle incoming labels with invalid UTF-8 #1131 diff --git a/VERSION b/VERSION index 7cca7711a..66333910a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.17.1 +0.18.0 From ef2736e7a6337615edf704066367d22ec2f4fa75 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Mon, 29 Sep 2025 19:15:17 +0300 Subject: [PATCH 052/113] Fix swapped `flushedLsn` and `receiveStartTli` for `wal_receiver` collector (#1198) In `pgStatWalReceiverQueryTemplate`, the order of the columns (when `hasFlushedLSN == true`) is: - ... - `receive_start_lsn` - `flushed_lsn` - `receive_start_tli` - ... However, columns were scanned in this order: - ... - `receive_start_lsn` -> `receiveStartLsn` - `receive_start_tli` -> `flushedLsn` (!) - `flushed_lsn` -> `receiveStartTli` (!) - ... This incorrect hydration of variables also manifests as swapped values for the `pg_stat_wal_receiver_flushed_lsn` and `pg_stat_wal_receiver_receive_start_tli` metrics. This seems to be a bug that has existed since the initial implementation: - 2d7e15275147ce7dfb7b7dc6f959d60e106c11a1 - https://github.com/prometheus-community/postgres_exporter/pull/844 In this patch, I'm: - fixing the `.Scan()`, so that it hydrates variables in the correct order - adjusting the order in which metrics are pushed out to the channel, to follow the order we consume them in (.., `receive_start_lsn`, `flushed_lsn`, `receive_start_tli`, ..) - adjusting the walreceiver tests, to follow the new order (which matches .`Scan()`) - fixing a small identation issue in `pgStatWalReceiverQueryTemplate` Signed-off-by: Slavi Pantaleev --- collector/pg_stat_walreceiver.go | 16 ++++++++-------- collector/pg_stat_walreceiver_test.go | 6 +++--- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/collector/pg_stat_walreceiver.go b/collector/pg_stat_walreceiver.go index ea0db4558..e6cc13652 100644 --- a/collector/pg_stat_walreceiver.go +++ b/collector/pg_stat_walreceiver.go @@ -108,7 +108,7 @@ var ( status, (receive_start_lsn- '0/0') %% (2^52)::bigint as receive_start_lsn, %s -receive_start_tli, + receive_start_tli, received_tli, extract(epoch from last_msg_send_time) as last_msg_send_time, extract(epoch from last_msg_receipt_time) as last_msg_receipt_time, @@ -147,7 +147,7 @@ func (c *PGStatWalReceiverCollector) Update(ctx context.Context, instance *insta var lastMsgSendTime, lastMsgReceiptTime, latestEndTime sql.NullFloat64 if hasFlushedLSN { - if err := rows.Scan(&upstreamHost, &slotName, &status, &receiveStartLsn, &receiveStartTli, &flushedLsn, &receivedTli, &lastMsgSendTime, &lastMsgReceiptTime, &latestEndLsn, &latestEndTime, &upstreamNode); err != nil { + if err := rows.Scan(&upstreamHost, &slotName, &status, &receiveStartLsn, &flushedLsn, &receiveStartTli, &receivedTli, &lastMsgSendTime, &lastMsgReceiptTime, &latestEndLsn, &latestEndTime, &upstreamNode); err != nil { return err } } else { @@ -209,12 +209,6 @@ func (c *PGStatWalReceiverCollector) Update(ctx context.Context, instance *insta float64(receiveStartLsn.Int64), labels...) - ch <- prometheus.MustNewConstMetric( - statWalReceiverReceiveStartTli, - prometheus.GaugeValue, - float64(receiveStartTli.Int64), - labels...) - if hasFlushedLSN { ch <- prometheus.MustNewConstMetric( statWalReceiverFlushedLSN, @@ -223,6 +217,12 @@ func (c *PGStatWalReceiverCollector) Update(ctx context.Context, instance *insta labels...) } + ch <- prometheus.MustNewConstMetric( + statWalReceiverReceiveStartTli, + prometheus.GaugeValue, + float64(receiveStartTli.Int64), + labels...) + ch <- prometheus.MustNewConstMetric( statWalReceiverReceivedTli, prometheus.GaugeValue, diff --git a/collector/pg_stat_walreceiver_test.go b/collector/pg_stat_walreceiver_test.go index c81c9ecae..13351255d 100644 --- a/collector/pg_stat_walreceiver_test.go +++ b/collector/pg_stat_walreceiver_test.go @@ -50,8 +50,8 @@ func TestPGStatWalReceiverCollectorWithFlushedLSN(t *testing.T) { "slot_name", "status", "receive_start_lsn", - "receive_start_tli", "flushed_lsn", + "receive_start_tli", "received_tli", "last_msg_send_time", "last_msg_receipt_time", @@ -65,8 +65,8 @@ func TestPGStatWalReceiverCollectorWithFlushedLSN(t *testing.T) { "bar", "stopping", int64(1200668684563608), - 1687321285, int64(1200668684563609), + 1687321285, 1687321280, 1687321275, 1687321276, @@ -88,8 +88,8 @@ func TestPGStatWalReceiverCollectorWithFlushedLSN(t *testing.T) { }() expected := []MetricResult{ {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1200668684563608, metricType: dto.MetricType_COUNTER}, - {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321285, metricType: dto.MetricType_GAUGE}, {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1200668684563609, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321285, metricType: dto.MetricType_GAUGE}, {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321280, metricType: dto.MetricType_GAUGE}, {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321275, metricType: dto.MetricType_COUNTER}, {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321276, metricType: dto.MetricType_COUNTER}, From b4c5250e034c26217f4d64253013a3eb42835722 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Mon, 29 Sep 2025 18:22:31 +0200 Subject: [PATCH 053/113] Fix #1185 bug (#1197) (#1201) This semicolon breaks the query Signed-off-by: Joe Adams Co-authored-by: Joe Adams --- collector/pg_process_idle.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/collector/pg_process_idle.go b/collector/pg_process_idle.go index 1f9658aa6..3cb08f114 100644 --- a/collector/pg_process_idle.go +++ b/collector/pg_process_idle.go @@ -56,7 +56,7 @@ func (PGProcessIdleCollector) Update(ctx context.Context, instance *instance, ch COUNT(*) AS process_idle_seconds_count FROM pg_stat_activity WHERE state ~ '^idle' - AND pid <> pg_backend_pid(); + AND pid <> pg_backend_pid() GROUP BY state, application_name ), buckets AS ( From 320b684f3dcd3f09b6e694e43f9b389b1a8199e5 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Mon, 29 Sep 2025 18:49:39 +0200 Subject: [PATCH 054/113] Prepare release v0.18.1 (#1202) * [BUGFIX] Fix swapped `flushedLsn` and `receiveStartTli` for `wal_receiver` collector by @spantaleev in https://github.com/prometheus-community/postgres_exporter/pull/1198 * [BUGFIX] Fix superfluous semicolon breaking query in `process_idle` by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1197 and https://github.com/prometheus-community/postgres_exporter/pull/1201 Signed-off-by: Cristian Greco --- CHANGELOG.md | 5 +++++ VERSION | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4402fb354..6cd2f553f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,11 @@ * [ENHANCEMENT] ... * [BUGFIX] ... +## 0.18.1 / 2025-09-29 + +* [BUGFIX] Fix swapped `flushedLsn` and `receiveStartTli` for `wal_receiver` collector by @spantaleev in https://github.com/prometheus-community/postgres_exporter/pull/1198 +* [BUGFIX] Fix superfluous semicolon breaking query in `process_idle` by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1197 and https://github.com/prometheus-community/postgres_exporter/pull/1201 + ## 0.18.0 / 2025-09-25 * [FEATURE] Add `stat_progress_vacuum` collector by @ianbibby in https://github.com/prometheus-community/postgres_exporter/pull/1141 diff --git a/VERSION b/VERSION index 66333910a..249afd517 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.18.0 +0.18.1 From a95b518a684cd05593a3c8661bbce8121b734675 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Tue, 30 Sep 2025 00:57:16 -0700 Subject: [PATCH 055/113] Update Go (#1203) * Update minimum supported Go to 1.24.0. * Update Go build to 1.25.x. * Update PostgreSQL testing versions. Signed-off-by: SuperQ --- .circleci/config.yml | 16 +++++++--------- .promu.yml | 2 +- README.md | 2 +- go.mod | 4 +--- 4 files changed, 10 insertions(+), 14 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index ee03ff8d2..707b71ee0 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -8,7 +8,7 @@ executors: # This must match .promu.yml. golang: docker: - - image: cimg/go:1.24 + - image: cimg/go:1.25 jobs: test: @@ -23,7 +23,7 @@ jobs: integration: docker: - - image: cimg/go:1.24 + - image: cimg/go:1.25 - image: << parameters.postgres_image >> environment: POSTGRES_DB: circle_test @@ -57,13 +57,11 @@ workflows: matrix: parameters: postgres_image: - - circleci/postgres:11 - - circleci/postgres:12 - - circleci/postgres:13 - - cimg/postgres:14.9 - - cimg/postgres:15.4 - - cimg/postgres:16.0 - - cimg/postgres:17.0 + - cimg/postgres:13.22 + - cimg/postgres:14.19 + - cimg/postgres:15.14 + - cimg/postgres:16.10 + - cimg/postgres:17.6 - prometheus/build: name: build parallelism: 3 diff --git a/.promu.yml b/.promu.yml index 5f915289f..c9bb51e78 100644 --- a/.promu.yml +++ b/.promu.yml @@ -1,6 +1,6 @@ go: # This must match .circle/config.yml. - version: 1.24 + version: 1.25 repository: path: github.com/prometheus-community/postgres_exporter build: diff --git a/README.md b/README.md index 01bd8b30d..beff2c793 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Prometheus exporter for PostgreSQL server metrics. -CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16`, `17`. +CI Tested PostgreSQL versions: `13`, `14`, `15`, `16`, `17`. ## Quick Start This package is available for Docker: diff --git a/go.mod b/go.mod index 02a1fec5f..a14e35022 100644 --- a/go.mod +++ b/go.mod @@ -1,8 +1,6 @@ module github.com/prometheus-community/postgres_exporter -go 1.23.0 - -toolchain go1.24.1 +go 1.24.0 require ( github.com/DATA-DOG/go-sqlmock v1.5.2 From c70a05907165216288593328c21a78ffe4af41f1 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 8 Oct 2025 04:06:13 +0200 Subject: [PATCH 056/113] feat: allow setting `limit` in `pg_stat_statements` (#1205) Add `.limit` CLI flag to `stat_statements` collector to allow setting a custom number of queries to be returned. Signed-off-by: Cristian Greco --- collector/pg_stat_statements.go | 33 +++-- collector/pg_stat_statements_test.go | 208 ++++++++++++++++++++++++--- 2 files changed, 216 insertions(+), 25 deletions(-) diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index d9a29ea65..824605190 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -24,11 +24,15 @@ import ( "github.com/prometheus/client_golang/prometheus" ) -const statStatementsSubsystem = "stat_statements" +const ( + statStatementsSubsystem = "stat_statements" + defaultStatementLimit = "100" +) var ( includeQueryFlag *bool = nil statementLengthFlag *uint = nil + statementLimitFlag *uint = nil ) func init() { @@ -47,12 +51,18 @@ func init() { "Maximum length of the statement text."). Default("120"). Uint() + statementLimitFlag = kingpin.Flag( + fmt.Sprint(collectorFlagPrefix, statStatementsSubsystem, ".limit"), + "Maximum number of statements to return."). + Default(defaultStatementLimit). + Uint() } type PGStatStatementsCollector struct { log *slog.Logger includeQueryStatement bool statementLength uint + statementLimit uint } func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { @@ -60,6 +70,7 @@ func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { log: config.logger, includeQueryStatement: *includeQueryFlag, statementLength: *statementLengthFlag, + statementLimit: *statementLimitFlag, }, nil } @@ -126,9 +137,9 @@ const ( FROM pg_stat_statements ) ORDER BY seconds_total DESC - LIMIT 100;` + LIMIT %s;` - pgStatStatementsNewQuery = `SELECT + pgStatStatementsQuery_PG13 = `SELECT pg_get_userbyid(userid) as user, pg_database.datname, pg_stat_statements.queryid, @@ -148,7 +159,7 @@ const ( FROM pg_stat_statements ) ORDER BY seconds_total DESC - LIMIT 100;` + LIMIT %s;` pgStatStatementsQuery_PG17 = `SELECT pg_get_userbyid(userid) as user, @@ -170,7 +181,7 @@ const ( FROM pg_stat_statements ) ORDER BY seconds_total DESC - LIMIT 100;` + LIMIT %s;` ) func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { @@ -179,20 +190,24 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc case instance.version.GE(semver.MustParse("17.0.0")): queryTemplate = pgStatStatementsQuery_PG17 case instance.version.GE(semver.MustParse("13.0.0")): - queryTemplate = pgStatStatementsNewQuery + queryTemplate = pgStatStatementsQuery_PG13 default: queryTemplate = pgStatStatementsQuery } - var querySelect = "" + querySelect := "" if c.includeQueryStatement { querySelect = fmt.Sprintf(pgStatStatementQuerySelect, c.statementLength) } - query := fmt.Sprintf(queryTemplate, querySelect) + statementLimit := defaultStatementLimit + if c.statementLimit > 0 { + statementLimit = fmt.Sprintf("%d", c.statementLimit) + } + query := fmt.Sprintf(queryTemplate, querySelect, statementLimit) db := instance.getDB() rows, err := db.QueryContext(ctx, query) - var presentQueryIds = make(map[string]struct{}) + presentQueryIds := make(map[string]struct{}) if err != nil { return err diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go index 0497ba380..8763f693d 100644 --- a/collector/pg_stat_statements_test.go +++ b/collector/pg_stat_statements_test.go @@ -24,7 +24,7 @@ import ( "github.com/smartystreets/goconvey/convey" ) -func TestPGStateStatementsCollector(t *testing.T) { +func TestPGStatStatementsCollector(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -36,7 +36,7 @@ func TestPGStateStatementsCollector(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, ""))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -67,7 +67,7 @@ func TestPGStateStatementsCollector(t *testing.T) { } } -func TestPGStateStatementsCollectorWithStatement(t *testing.T) { +func TestPGStatStatementsCollectorWithStatement(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -79,7 +79,7 @@ func TestPGStateStatementsCollectorWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 100) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100)))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -111,7 +111,51 @@ func TestPGStateStatementsCollectorWithStatement(t *testing.T) { } } -func TestPGStateStatementsCollectorNull(t *testing.T) { +func TestPGStatStatementsCollectorWithStatementAndLimit(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("12.0.0")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 100) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), "10"))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 100, statementLimit: 10} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollectorNull(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -123,7 +167,7 @@ func TestPGStateStatementsCollectorNull(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, ""))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -154,7 +198,7 @@ func TestPGStateStatementsCollectorNull(t *testing.T) { } } -func TestPGStateStatementsCollectorNullWithStatement(t *testing.T) { +func TestPGStatStatementsCollectorNullWithStatement(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -166,7 +210,7 @@ func TestPGStateStatementsCollectorNullWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, fmt.Sprintf(pgStatStatementQuerySelect, 200)))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -198,7 +242,51 @@ func TestPGStateStatementsCollectorNullWithStatement(t *testing.T) { } } -func TestPGStateStatementsCollectorNewPG(t *testing.T) { +func TestPGStatStatementsCollectorNullWithStatementAndLimit(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "10"))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 200, statementLimit: 10} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"queryid": "unknown", "query": "unknown"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollector_PG13(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -210,7 +298,7 @@ func TestPGStateStatementsCollectorNewPG(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, ""))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -241,7 +329,7 @@ func TestPGStateStatementsCollectorNewPG(t *testing.T) { } } -func TestPGStateStatementsCollectorNewPGWithStatement(t *testing.T) { +func TestPGStatStatementsCollector_PG13_WithStatement(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -253,7 +341,7 @@ func TestPGStateStatementsCollectorNewPGWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsNewQuery, fmt.Sprintf(pgStatStatementQuerySelect, 300)))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -285,7 +373,51 @@ func TestPGStateStatementsCollectorNewPGWithStatement(t *testing.T) { } } -func TestPGStateStatementsCollector_PG17(t *testing.T) { +func TestPGStatStatementsCollector_PG13_WithStatementAndLimit(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "10"))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 300, statementLimit: 10} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollector_PG17(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -297,7 +429,7 @@ func TestPGStateStatementsCollector_PG17(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, ""))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -328,7 +460,7 @@ func TestPGStateStatementsCollector_PG17(t *testing.T) { } } -func TestPGStateStatementsCollector_PG17_WithStatement(t *testing.T) { +func TestPGStatStatementsCollector_PG17_WithStatement(t *testing.T) { db, mock, err := sqlmock.New() if err != nil { t.Fatalf("Error opening a stub db connection: %s", err) @@ -340,7 +472,7 @@ func TestPGStateStatementsCollector_PG17_WithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300)))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -371,3 +503,47 @@ func TestPGStateStatementsCollector_PG17_WithStatement(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPGStatStatementsCollector_PG17_WithStatementAndLimit(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("17.0.0")} + + columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "10"))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{includeQueryStatement: true, statementLength: 300, statementLimit: 10} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + {labels: labelMap{"queryid": "1500", "query": "select 1 from foo"}, metricType: dto.MetricType_COUNTER, value: 1}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From 00b76dfa6a9a09a2d845d23b744d2cab0111e5dd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 12 Oct 2025 08:05:59 +0200 Subject: [PATCH 057/113] Bump github.com/prometheus/client_golang from 1.23.1 to 1.23.2 (#1204) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.23.1 to 1.23.2. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.23.1...v1.23.2) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.23.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index a14e35022..1698d7b3b 100644 --- a/go.mod +++ b/go.mod @@ -7,7 +7,7 @@ require ( github.com/alecthomas/kingpin/v2 v2.4.0 github.com/blang/semver/v4 v4.0.0 github.com/lib/pq v1.10.9 - github.com/prometheus/client_golang v1.23.1 + github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.66.1 github.com/prometheus/exporter-toolkit v0.14.1 diff --git a/go.sum b/go.sum index 7feb5da55..55486ac88 100644 --- a/go.sum +++ b/go.sum @@ -49,8 +49,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.23.1 h1:w6gXMLQGgd0jXXlote9lRHMe0nG01EbnJT+C0EJru2Y= -github.com/prometheus/client_golang v1.23.1/go.mod h1:br8j//v2eg2K5Vvna5klK8Ku5pcU5r4ll73v6ik5dIQ= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= From 37171f7741996d6f98e2335044fda194fbd58d25 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Tue, 21 Oct 2025 15:02:15 +0200 Subject: [PATCH 058/113] Enable pprof (#1212) Add the pprof package to allow debug profiling. Related: https://github.com/prometheus-community/postgres_exporter/issues/1189 Signed-off-by: SuperQ --- cmd/postgres_exporter/main.go | 1 + 1 file changed, 1 insertion(+) diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index 093ddd301..6b93725d4 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -16,6 +16,7 @@ package main import ( "fmt" "net/http" + _ "net/http/pprof" "os" "strings" From 9d9a9f9ef0887080d6486d1c1e800ef41b8f48cf Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 24 Oct 2025 03:19:02 +0200 Subject: [PATCH 059/113] Update common Prometheus files (#1213) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 79891553f..75f886d54 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -28,7 +28,7 @@ jobs: with: persist-credentials: false - name: Install Go - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 + uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0 with: go-version: 1.25.x - name: Install snmp_exporter/generator dependencies From eae2c87e90db9928503384ab60ce9f876110fc3b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 4 Nov 2025 20:30:23 -0500 Subject: [PATCH 060/113] Bump github.com/prometheus/common from 0.66.1 to 0.67.2 (#1216) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.66.1 to 0.67.2. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/common/compare/v0.66.1...v0.67.2) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.67.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 20 ++++++++++---------- go.sum | 50 ++++++++++++++++++++++++++++---------------------- 2 files changed, 38 insertions(+), 32 deletions(-) diff --git a/go.mod b/go.mod index 1698d7b3b..a5d9a1f4d 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.66.1 + github.com/prometheus/common v0.67.2 github.com/prometheus/exporter-toolkit v0.14.1 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -18,7 +18,7 @@ require ( ) require ( - github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect + github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/coreos/go-systemd/v22 v22.6.0 // indirect @@ -35,12 +35,12 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - go.yaml.in/yaml/v2 v2.4.2 // indirect - golang.org/x/crypto v0.41.0 // indirect - golang.org/x/net v0.43.0 // indirect - golang.org/x/oauth2 v0.30.0 // indirect - golang.org/x/sync v0.16.0 // indirect - golang.org/x/sys v0.35.0 // indirect - golang.org/x/text v0.28.0 // indirect - google.golang.org/protobuf v1.36.8 // indirect + go.yaml.in/yaml/v2 v2.4.3 // indirect + golang.org/x/crypto v0.43.0 // indirect + golang.org/x/net v0.46.0 // indirect + golang.org/x/oauth2 v0.32.0 // indirect + golang.org/x/sync v0.17.0 // indirect + golang.org/x/sys v0.37.0 // indirect + golang.org/x/text v0.30.0 // indirect + google.golang.org/protobuf v1.36.10 // indirect ) diff --git a/go.sum b/go.sum index 55486ac88..b450ac6c0 100644 --- a/go.sum +++ b/go.sum @@ -2,8 +2,8 @@ github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7Oputl github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU= github.com/alecthomas/kingpin/v2 v2.4.0 h1:f48lwail6p8zpO1bC4TxtqACaGqHYA22qkHjHpqDjYY= github.com/alecthomas/kingpin/v2 v2.4.0/go.mod h1:0gyi0zQnjuFk8xrkNKamJoyUo382HRL7ATRpFZCw6tE= -github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc= -github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8VM1Kc9e19TLln2VL61YJF0x1XFtfdL4JdbSyE= +github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0= +github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= @@ -53,8 +53,8 @@ github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= -github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/common v0.67.2 h1:PcBAckGFTIHt2+L3I33uNRTlKTplNzFctXcWhPyAEN8= +github.com/prometheus/common v0.67.2/go.mod h1:63W3KZb1JOKgcjlIr64WW/LvFGAqKPj0atm+knVGEko= github.com/prometheus/exporter-toolkit v0.14.1 h1:uKPE4ewweVRWFainwvAcHs3uw15pjw2dk3I7b+aNo9o= github.com/prometheus/exporter-toolkit v0.14.1/go.mod h1:di7yaAJiaMkcjcz48f/u4yRPwtyuxTU5Jr4EnM2mhtQ= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= @@ -67,34 +67,40 @@ github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+ github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY= github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= -go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= -golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4= -golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc= -golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE= -golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg= -golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI= -golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU= -golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= -golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= -golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= -golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= -golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng= -golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU= -google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= -google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= +go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= +golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04= +golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0= +golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4= +golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210= +golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= +golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= +golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= +golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= +golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= From 43720582adae852d82f37f5567e8e4093165ffd2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 4 Nov 2025 21:29:27 -0500 Subject: [PATCH 061/113] Bump github.com/prometheus/exporter-toolkit from 0.14.1 to 0.15.0 (#1215) Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.14.1 to 0.15.0. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.14.1...v0.15.0) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-version: 0.15.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 3 ++- go.sum | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index a5d9a1f4d..93ac4c507 100644 --- a/go.mod +++ b/go.mod @@ -10,7 +10,7 @@ require ( github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.67.2 - github.com/prometheus/exporter-toolkit v0.14.1 + github.com/prometheus/exporter-toolkit v0.15.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -42,5 +42,6 @@ require ( golang.org/x/sync v0.17.0 // indirect golang.org/x/sys v0.37.0 // indirect golang.org/x/text v0.30.0 // indirect + golang.org/x/time v0.13.0 // indirect google.golang.org/protobuf v1.36.10 // indirect ) diff --git a/go.sum b/go.sum index b450ac6c0..df5eb2e64 100644 --- a/go.sum +++ b/go.sum @@ -55,8 +55,8 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.67.2 h1:PcBAckGFTIHt2+L3I33uNRTlKTplNzFctXcWhPyAEN8= github.com/prometheus/common v0.67.2/go.mod h1:63W3KZb1JOKgcjlIr64WW/LvFGAqKPj0atm+knVGEko= -github.com/prometheus/exporter-toolkit v0.14.1 h1:uKPE4ewweVRWFainwvAcHs3uw15pjw2dk3I7b+aNo9o= -github.com/prometheus/exporter-toolkit v0.14.1/go.mod h1:di7yaAJiaMkcjcz48f/u4yRPwtyuxTU5Jr4EnM2mhtQ= +github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= +github.com/prometheus/exporter-toolkit v0.15.0/go.mod h1:OyRWd2iTo6Xge9Kedvv0IhCrJSBu36JCfJ2yVniRIYk= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -94,6 +94,8 @@ golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= +golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= From 974c9de42884c10daef43fef8b9a7227e9fcc099 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 18 Nov 2025 13:13:16 +0100 Subject: [PATCH 062/113] Synchronize common files from prometheus/prometheus (#1217) * Update common Prometheus files Signed-off-by: prombot * Fix linting issues. Signed-off-by: SuperQ --------- Signed-off-by: prombot Signed-off-by: SuperQ Co-authored-by: SuperQ --- Makefile.common | 2 +- cmd/postgres_exporter/pg_setting_test.go | 1 - cmd/postgres_exporter/postgres_exporter_integration_test.go | 1 - cmd/postgres_exporter/postgres_exporter_test.go | 1 - 4 files changed, 1 insertion(+), 4 deletions(-) diff --git a/Makefile.common b/Makefile.common index 6762d0f83..143bf03fb 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.4.0 +GOLANGCI_LINT_VERSION ?= v2.6.0 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. diff --git a/cmd/postgres_exporter/pg_setting_test.go b/cmd/postgres_exporter/pg_setting_test.go index 6923da630..9acdeaed7 100644 --- a/cmd/postgres_exporter/pg_setting_test.go +++ b/cmd/postgres_exporter/pg_setting_test.go @@ -12,7 +12,6 @@ // limitations under the License. //go:build !integration -// +build !integration package main diff --git a/cmd/postgres_exporter/postgres_exporter_integration_test.go b/cmd/postgres_exporter/postgres_exporter_integration_test.go index 7043c1e88..39032f333 100644 --- a/cmd/postgres_exporter/postgres_exporter_integration_test.go +++ b/cmd/postgres_exporter/postgres_exporter_integration_test.go @@ -15,7 +15,6 @@ // a lot of additional work to keep the external docker environment they require // working. //go:build integration -// +build integration package main diff --git a/cmd/postgres_exporter/postgres_exporter_test.go b/cmd/postgres_exporter/postgres_exporter_test.go index 0f36febf4..5b2879d94 100644 --- a/cmd/postgres_exporter/postgres_exporter_test.go +++ b/cmd/postgres_exporter/postgres_exporter_test.go @@ -12,7 +12,6 @@ // limitations under the License. //go:build !integration -// +build !integration package main From 5483697c036f5cd8e4229eae12aa8a0583117d9d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 1 Dec 2025 14:35:42 -0500 Subject: [PATCH 063/113] Bump github.com/prometheus/common from 0.67.2 to 0.67.4 (#1224) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.67.2 to 0.67.4. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/common/compare/v0.67.2...v0.67.4) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.67.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 4 +++- go.sum | 8 ++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 93ac4c507..ecd8bec79 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.67.2 + github.com/prometheus/common v0.67.4 github.com/prometheus/exporter-toolkit v0.15.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -22,6 +22,8 @@ require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/coreos/go-systemd/v22 v22.6.0 // indirect + github.com/golang-jwt/jwt/v5 v5.3.0 // indirect + github.com/google/uuid v1.6.0 // indirect github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/jtolds/gls v4.20.0+incompatible // indirect diff --git a/go.sum b/go.sum index df5eb2e64..1dfc429b6 100644 --- a/go.sum +++ b/go.sum @@ -16,8 +16,12 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo= +github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= @@ -53,8 +57,8 @@ github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.67.2 h1:PcBAckGFTIHt2+L3I33uNRTlKTplNzFctXcWhPyAEN8= -github.com/prometheus/common v0.67.2/go.mod h1:63W3KZb1JOKgcjlIr64WW/LvFGAqKPj0atm+knVGEko= +github.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc= +github.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= github.com/prometheus/exporter-toolkit v0.15.0/go.mod h1:OyRWd2iTo6Xge9Kedvv0IhCrJSBu36JCfJ2yVniRIYk= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= From 7c167f7add794da1181422cd38c5cf99c37686d7 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 10 Dec 2025 09:06:54 -0500 Subject: [PATCH 064/113] Bump golang.org/x/crypto from 0.43.0 to 0.45.0 (#1220) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0. - [Commits](https://github.com/golang/crypto/compare/v0.43.0...v0.45.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.45.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 10 +++++----- go.sum | 20 ++++++++++---------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/go.mod b/go.mod index ecd8bec79..6ec5291d5 100644 --- a/go.mod +++ b/go.mod @@ -38,12 +38,12 @@ require ( github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect go.yaml.in/yaml/v2 v2.4.3 // indirect - golang.org/x/crypto v0.43.0 // indirect - golang.org/x/net v0.46.0 // indirect + golang.org/x/crypto v0.45.0 // indirect + golang.org/x/net v0.47.0 // indirect golang.org/x/oauth2 v0.32.0 // indirect - golang.org/x/sync v0.17.0 // indirect - golang.org/x/sys v0.37.0 // indirect - golang.org/x/text v0.30.0 // indirect + golang.org/x/sync v0.18.0 // indirect + golang.org/x/sys v0.38.0 // indirect + golang.org/x/text v0.31.0 // indirect golang.org/x/time v0.13.0 // indirect google.golang.org/protobuf v1.36.10 // indirect ) diff --git a/go.sum b/go.sum index 1dfc429b6..acf1db306 100644 --- a/go.sum +++ b/go.sum @@ -86,18 +86,18 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= -golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04= -golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0= -golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4= -golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210= +golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= +golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= +golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= +golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= -golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= -golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= -golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= -golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= -golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= +golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= +golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= +golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= From d613418b0031209ab0cd16a34cbcb76cb3a88750 Mon Sep 17 00:00:00 2001 From: Joe Adams Date: Fri, 12 Dec 2025 04:13:36 -0500 Subject: [PATCH 065/113] Update GitHub Actions to run all CI (#1231) * Update GitHub Actions config to run tests Signed-off-by: Joe Adams * Add CI steps for release and include all postgres versions Signed-off-by: Joe Adams * Disable circleci jobs This will stop the CI jobs but leave existing PRs in place until we can get those updated. Signed-off-by: Joe Adams --------- Signed-off-by: Joe Adams --- .circleci/config.yml | 106 +++++----------------------- .github/workflows/ci.yml | 146 +++++++++++++++++++++++++++++++++++++++ queries.yaml | 2 +- 3 files changed, 165 insertions(+), 89 deletions(-) create mode 100644 .github/workflows/ci.yml diff --git a/.circleci/config.yml b/.circleci/config.yml index 707b71ee0..965b27394 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -1,103 +1,33 @@ --- -version: 2.1 +# Prometheus has switched to GitHub action. +# Circle CI is not disabled repository-wise so that previous pull requests +# continue working. +# This file does not generate any CircleCI workflow. -orbs: - prometheus: prometheus/prometheus@0.17.1 +version: 2.1 executors: - # This must match .promu.yml. golang: docker: - - image: cimg/go:1.25 + - image: busybox jobs: - test: + noopjob: executor: golang steps: - - prometheus/setup_environment - - run: GOHOSTARCH=386 GOARCH=386 make test - - run: make - - prometheus/store_artifact: - file: postgres_exporter - - integration: - docker: - - image: cimg/go:1.25 - - image: << parameters.postgres_image >> - environment: - POSTGRES_DB: circle_test - POSTGRES_USER: postgres - POSTGRES_PASSWORD: test - - parameters: - postgres_image: - type: string - - environment: - DATA_SOURCE_NAME: 'postgresql://postgres:test@localhost:5432/circle_test?sslmode=disable' - GOOPTS: '-v -tags integration' - - steps: - - checkout - - setup_remote_docker - - run: docker version - - run: make build - - run: make test + - run: + command: "true" workflows: version: 2 - postgres_exporter: + prometheus: jobs: - - test: - filters: - tags: - only: /.*/ - - integration: - matrix: - parameters: - postgres_image: - - cimg/postgres:13.22 - - cimg/postgres:14.19 - - cimg/postgres:15.14 - - cimg/postgres:16.10 - - cimg/postgres:17.6 - - prometheus/build: - name: build - parallelism: 3 - promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" - filters: - tags: - ignore: /^v.*/ - branches: - ignore: /^(main|master|release-.*|.*build-all.*)$/ - - prometheus/build: - name: build_all - parallelism: 12 - filters: - branches: - only: /^(main|master|release-.*|.*build-all.*)$/ - tags: - only: /^v.*/ - - prometheus/publish_master: - context: org-context - docker_hub_organization: prometheuscommunity - quay_io_organization: prometheuscommunity - requires: - - test - - build_all - filters: - branches: - only: master - - prometheus/publish_release: - context: org-context - docker_hub_organization: prometheuscommunity - quay_io_organization: prometheuscommunity - requires: - - test - - build_all - filters: - tags: - only: /^v.*/ - branches: - ignore: /.*/ + - noopjob + triggers: + - schedule: + cron: "0 0 30 2 *" + filters: + branches: + only: + - main diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 000000000..3150f65f3 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,146 @@ +--- +name: CI +on: + pull_request: + push: + +jobs: + test_go: + name: Go tests + runs-on: ubuntu-latest + container: + # Whenever the Go version is updated here, .promu.yml + # should also be updated. + image: quay.io/prometheus/golang-builder:1.25-base + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/setup_environment + - run: make test + - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1 + integration_tests: + name: Integration tests + runs-on: ubuntu-latest + strategy: + matrix: + # Define Postgres versions to test against. + postgres_version: + - 13.22 + - 14.19 + - 15.14 + - 16.10 + - 17.6 + services: + postgres: + image: postgres:${{ matrix.postgres_version }} + env: + POSTGRES_DB: circle_test + POSTGRES_USER: postgres + POSTGRES_PASSWORD: test + # options: >- + # --health-cmd="pg_isready -U postgres -d circle_test" + # --health-interval=10s + # --health-timeout=5s + # --health-retries=5 + container: + # Whenever the Go version is updated here, .promu.yml + # should also be updated. + image: quay.io/prometheus/golang-builder:1.25-base + env: + DATA_SOURCE_NAME: 'postgresql://postgres:test@postgres:5432/circle_test?sslmode=disable' + GOOPTS: '-v -tags integration' + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/setup_environment + - run: make build + - run: make test + + + build: + name: Build Prometheus for common architectures + runs-on: ubuntu-latest + if: | + !(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) + && + !(github.event_name == 'pull_request' && startsWith(github.event.pull_request.base.ref, 'release-')) + && + !(github.event_name == 'push' && github.event.ref == 'refs/heads/main') + && + !(github.event_name == 'push' && github.event.ref == 'refs/heads/master') + strategy: + matrix: + thread: [ 0, 1, 2 ] + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/build + with: + promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" + parallelism: 3 + thread: ${{ matrix.thread }} + + build_all: + name: Build Prometheus for all architectures + runs-on: ubuntu-latest + if: | + (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) + || + (github.event_name == 'pull_request' && startsWith(github.event.pull_request.base.ref, 'release-')) + || + (github.event_name == 'push' && github.event.ref == 'refs/heads/main') + || + (github.event_name == 'push' && github.event.ref == 'refs/heads/master') + strategy: + matrix: + thread: [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 ] + + # Whenever the Go version is updated here, .promu.yml + # should also be updated. + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/build + with: + parallelism: 12 + thread: ${{ matrix.thread }} + + publish_main: + # https://github.com/prometheus/promci/blob/52c7012f5f0070d7281b8db4a119e21341d43c91/actions/publish_main/action.yml + name: Publish main branch artifacts + runs-on: ubuntu-latest + needs: [test_go, build_all] + if: | + (github.event_name == 'push' && github.event.ref == 'refs/heads/main') + || + (github.event_name == 'push' && github.event.ref == 'refs/heads/master') + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/publish_main + with: + docker_hub_organization: prometheuscommunity + docker_hub_login: ${{ secrets.docker_hub_login }} + docker_hub_password: ${{ secrets.docker_hub_password }} + quay_io_organization: prometheuscommunity + quay_io_login: ${{ secrets.quay_io_login }} + quay_io_password: ${{ secrets.quay_io_password }} + + publish_release: + name: Publish release artefacts + runs-on: ubuntu-latest + needs: [test_go, build_all] + if: | + (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) + steps: + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: ./.github/promci/actions/publish_release + with: + docker_hub_organization: prometheuscommunity + docker_hub_login: ${{ secrets.docker_hub_login }} + docker_hub_password: ${{ secrets.docker_hub_password }} + quay_io_organization: prometheuscommunity + quay_io_login: ${{ secrets.quay_io_login }} + quay_io_password: ${{ secrets.quay_io_password }} + github_token: ${{ secrets.PROMBOT_GITHUB_TOKEN }} diff --git a/queries.yaml b/queries.yaml index 189ce0866..9e9ee167c 100644 --- a/queries.yaml +++ b/queries.yaml @@ -1,2 +1,2 @@ # Adding queries to this file is deprecated -# Example queries have been transformed into collectors. \ No newline at end of file +# Example queries have been transformed into collectors. From 778660147e6b6af297005f2c2f51d8fce45380e5 Mon Sep 17 00:00:00 2001 From: Joe Adams Date: Sat, 13 Dec 2025 07:46:30 -0500 Subject: [PATCH 066/113] Fix NULL on long_running_transactions collector (#1230) When there are no long running transactions, the oldest timestamp will be NULL. This sets the metrics value to 0 (age). Also adds a test for this behavior. Fixes #1223 Signed-off-by: Joe Adams --- collector/pg_long_running_transactions.go | 15 +++++-- .../pg_long_running_transactions_test.go | 41 +++++++++++++++++++ 2 files changed, 53 insertions(+), 3 deletions(-) diff --git a/collector/pg_long_running_transactions.go b/collector/pg_long_running_transactions.go index 072862f4e..ac1eaa70e 100644 --- a/collector/pg_long_running_transactions.go +++ b/collector/pg_long_running_transactions.go @@ -15,6 +15,7 @@ package collector import ( "context" + "database/sql" "log/slog" "github.com/prometheus/client_golang/prometheus" @@ -50,7 +51,7 @@ var ( ) longRunningTransactionsQuery = ` - SELECT + SELECT COUNT(*) as transactions, MAX(EXTRACT(EPOCH FROM clock_timestamp() - pg_stat_activity.xact_start)) AS oldest_timestamp_seconds FROM pg_catalog.pg_stat_activity @@ -72,12 +73,20 @@ func (PGLongRunningTransactionsCollector) Update(ctx context.Context, instance * defer rows.Close() for rows.Next() { - var transactions, ageInSeconds float64 + var transactions float64 + var ageInSeconds sql.NullFloat64 if err := rows.Scan(&transactions, &ageInSeconds); err != nil { return err } + // If there are no long running transactions, ageInSeconds will be NULL + // so we set it to 0 + age := 0.0 + if ageInSeconds.Valid { + age = ageInSeconds.Float64 + } + ch <- prometheus.MustNewConstMetric( longRunningTransactionsCount, prometheus.GaugeValue, @@ -86,7 +95,7 @@ func (PGLongRunningTransactionsCollector) Update(ctx context.Context, instance * ch <- prometheus.MustNewConstMetric( longRunningTransactionsAgeInSeconds, prometheus.GaugeValue, - ageInSeconds, + age, ) } if err := rows.Err(); err != nil { diff --git a/collector/pg_long_running_transactions_test.go b/collector/pg_long_running_transactions_test.go index eedda7c65..ba7bc6a52 100644 --- a/collector/pg_long_running_transactions_test.go +++ b/collector/pg_long_running_transactions_test.go @@ -61,3 +61,44 @@ func TestPGLongRunningTransactionsCollector(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPGLongRunningTransactionsCollectorNull(t *testing.T) { + // Test when no long running transactions are present + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "transactions", + "age_in_seconds", + } + rows := sqlmock.NewRows(columns). + AddRow(0, nil) + + mock.ExpectQuery(sanitizeQuery(longRunningTransactionsQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGLongRunningTransactionsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGLongRunningTransactionsCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{}, value: 0, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{}, value: 0, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From f2203bab20820b4e2b390e80ebf6f4168a126951 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 19 Dec 2025 10:09:12 +0100 Subject: [PATCH 067/113] Update common Prometheus files (#1222) Signed-off-by: prombot Co-authored-by: Cristian Greco --- Makefile.common | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile.common b/Makefile.common index 143bf03fb..3ed717b46 100644 --- a/Makefile.common +++ b/Makefile.common @@ -112,7 +112,7 @@ common-all: precheck style check_license lint yamllint unused build test .PHONY: common-style common-style: @echo ">> checking code style" - @fmtRes=$$($(GOFMT) -d $$(find . -path ./vendor -prune -o -name '*.go' -print)); \ + @fmtRes=$$($(GOFMT) -d $$(git ls-files '*.go' ':!:vendor/*' || find . -path ./vendor -prune -o -name '*.go' -print)); \ if [ -n "$${fmtRes}" ]; then \ echo "gofmt checking failed!"; echo "$${fmtRes}"; echo; \ echo "Please ensure you are using $$($(GO) version) for formatting code."; \ @@ -122,7 +122,7 @@ common-style: .PHONY: common-check_license common-check_license: @echo ">> checking license header" - @licRes=$$(for file in $$(find . -type f -iname '*.go' ! -path './vendor/*') ; do \ + @licRes=$$(for file in $$(git ls-files '*.go' ':!:vendor/*' || find . -path ./vendor -prune -o -type f -iname '*.go' -print) ; do \ awk 'NR<=3' $$file | grep -Eq "(Copyright|generated|GENERATED)" || echo $$file; \ done); \ if [ -n "$${licRes}" ]; then \ From 97fb8714ccaa9fb888881d689638c69f09dcbd6f Mon Sep 17 00:00:00 2001 From: Fabio Rueda Date: Fri, 19 Dec 2025 10:17:49 +0100 Subject: [PATCH 068/113] Document stat_checkpointer collector option (#1226) Add documentation for stat_checkpointer collector Signed-off-by: Fabio Rueda --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index beff2c793..4144029cb 100644 --- a/README.md +++ b/README.md @@ -141,6 +141,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.stat_bgwriter` Enable the `stat_bgwriter` collector (default: enabled). +* `[no-]collector.stat_checkpointer` + Enable the `stat_checkpointer` collector (default: disabled). + * `[no-]collector.stat_database` Enable the `stat_database` collector (default: enabled). From 5724e0c821f0cfb69e2caed7ac214e2679e26dd7 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 23 Dec 2025 04:09:32 +0100 Subject: [PATCH 069/113] Update common Prometheus files (#1233) Signed-off-by: prombot --- Makefile.common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index 3ed717b46..840bc0ea7 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.6.0 +GOLANGCI_LINT_VERSION ?= v2.6.2 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. From 73a157ed0ed430bd6328e5a0984de7b81095877b Mon Sep 17 00:00:00 2001 From: Pierre Souchay Date: Tue, 23 Dec 2025 04:21:46 +0100 Subject: [PATCH 070/113] Ensure collection of stats succeed in a maximum duration to avoid exhausting PG connections (#1229) * Ensure collection returns within a specified delay **Context**: As explained in https://github.com/prometheus-community/postgres_exporter/issues/1228, when database is very slow to answer, every call to prometheus exporter might consume a new connection and possibly consume all available connections **Solution**: Ensure a Context with a specified Timeout is specified, so the connection will end if duration of collection is too long Signed-off-by: Pierre Souchay * test: added TestPGDatabaseTimeout to ensure our timing behaves as it should Signed-off-by: Pierre Souchay * document the new variable `PG_EXPORTER_COLLECTION_TIMEOUT` in the README.md Signed-off-by: Pierre SOUCHAY * CollectionTimeout -> WithCollectionTimeout Signed-off-by: Pierre SOUCHAY * Document and report 0 value for PG_EXPORTER_COLLECTION_TIMEOUT Signed-off-by: Pierre SOUCHAY * move test to the right place Signed-off-by: Pierre SOUCHAY * ensure that UTest do not take too much time Signed-off-by: Pierre SOUCHAY --------- Signed-off-by: Pierre Souchay Signed-off-by: Pierre SOUCHAY Co-authored-by: Pierre SOUCHAY --- README.md | 8 ++++ cmd/postgres_exporter/main.go | 3 +- collector/collector.go | 26 +++++++++++-- collector/collector_test.go | 73 +++++++++++++++++++++++++++++++++++ 4 files changed, 106 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 4144029cb..2654b319f 100644 --- a/README.md +++ b/README.md @@ -258,6 +258,14 @@ The following environment variables configure the exporter: * `DATA_SOURCE_PASS_FILE` The same as above but reads the password from a file. +* `PG_EXPORTER_COLLECTION_TIMEOUT` + Timeout duration to use when collecting the statistics, default to `1m`. + When the timeout is reached, the database connection will be dropped. + It avoids connections stacking when the database answers too slowly + (for instance if the database creates/drop a huge table and locks the tables) + and will avoid exhausting the pool of connections of the database. + Value of `0` or less than `1ms` is considered invalid and will report an error. + * `PG_EXPORTER_WEB_TELEMETRY_PATH` Path under which to expose metrics. Default is `/metrics`. diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index 6b93725d4..c901ec722 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -50,6 +50,7 @@ var ( excludeDatabases = kingpin.Flag("exclude-databases", "A list of databases to remove when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXCLUDE_DATABASES").String() includeDatabases = kingpin.Flag("include-databases", "A list of databases to include when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_INCLUDE_DATABASES").String() metricPrefix = kingpin.Flag("metric-prefix", "A metric prefix can be used to have non-default (not \"pg\") prefixes for each of the metrics").Default("pg").Envar("PG_EXPORTER_METRIC_PREFIX").String() + collectionTimeout = kingpin.Flag("collection-timeout", "Timeout for collecting the statistics when the database is slow").Default("1m").Envar("PG_EXPORTER_COLLECTION_TIMEOUT").String() logger = promslog.NewNopLogger() ) @@ -137,7 +138,7 @@ func main() { excludedDatabases, dsn, []string{}, - ) + collector.WithCollectionTimeout(*collectionTimeout)) if err != nil { logger.Warn("Failed to create PostgresCollector", "err", err.Error()) } else { diff --git a/collector/collector.go b/collector/collector.go index de7203486..130f5313b 100644 --- a/collector/collector.go +++ b/collector/collector.go @@ -92,7 +92,8 @@ type PostgresCollector struct { Collectors map[string]Collector logger *slog.Logger - instance *instance + instance *instance + CollectionTimeout time.Duration } type Option func(*PostgresCollector) error @@ -158,6 +159,20 @@ func NewPostgresCollector(logger *slog.Logger, excludeDatabases []string, dsn st return p, nil } +func WithCollectionTimeout(s string) Option { + return func(e *PostgresCollector) error { + duration, err := time.ParseDuration(s) + if err != nil { + return err + } + if duration < 1*time.Millisecond { + return errors.New("timeout must be greater than 1ms") + } + e.CollectionTimeout = duration + return nil + } +} + // Describe implements the prometheus.Collector interface. func (p PostgresCollector) Describe(ch chan<- *prometheus.Desc) { ch <- scrapeDurationDesc @@ -166,8 +181,6 @@ func (p PostgresCollector) Describe(ch chan<- *prometheus.Desc) { // Collect implements the prometheus.Collector interface. func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) { - ctx := context.TODO() - // copy the instance so that concurrent scrapes have independent instances inst := p.instance.copy() @@ -178,6 +191,13 @@ func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) { p.logger.Error("Error opening connection to database", "err", err) return } + p.collectFromConnection(inst, ch) +} + +func (p PostgresCollector) collectFromConnection(inst *instance, ch chan<- prometheus.Metric) { + // Eventually, connect this to the http scraping context + ctx, cancel := context.WithTimeout(context.Background(), p.CollectionTimeout) + defer cancel() wg := sync.WaitGroup{} wg.Add(len(p.Collectors)) diff --git a/collector/collector_test.go b/collector/collector_test.go index d3b473b43..984b2c0ff 100644 --- a/collector/collector_test.go +++ b/collector/collector_test.go @@ -14,9 +14,13 @@ package collector import ( "strings" + "testing" + "time" + "github.com/DATA-DOG/go-sqlmock" "github.com/prometheus/client_golang/prometheus" dto "github.com/prometheus/client_model/go" + "github.com/prometheus/common/promslog" ) type labelMap map[string]string @@ -60,3 +64,72 @@ func sanitizeQuery(q string) string { q = strings.ReplaceAll(q, "$", "\\$") return q } + +// We ensure that when the database respond after a long time +// The collection process still occurs in a predictable manner +// Will avoid accumulation of queries on a completely frozen DB +func TestWithConnectionTimeout(t *testing.T) { + + timeoutForQuery := time.Duration(100 * time.Millisecond) + + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"pg_roles.rolname", "pg_roles.rolconnlimit"} + rows := sqlmock.NewRows(columns).AddRow("role1", 2) + mock.ExpectQuery(pgRolesConnectionLimitsQuery). + WillDelayFor(30 * time.Second). + WillReturnRows(rows) + + log_config := promslog.Config{} + + logger := promslog.New(&log_config) + + c, err := NewPostgresCollector(logger, []string{}, "postgresql://local", []string{}, WithCollectionTimeout(timeoutForQuery.String())) + if err != nil { + t.Fatalf("error creating NewPostgresCollector: %s", err) + } + collector_config := collectorConfig{ + logger: logger, + excludeDatabases: []string{}, + } + + collector, err := NewPGRolesCollector(collector_config) + if err != nil { + t.Fatalf("error creating collector: %s", err) + } + c.Collectors["test"] = collector + c.instance = inst + + ch := make(chan prometheus.Metric) + defer close(ch) + + go func() { + for { + <-ch + time.Sleep(1 * time.Millisecond) + } + }() + + startTime := time.Now() + c.collectFromConnection(inst, ch) + elapsed := time.Since(startTime) + + if elapsed <= timeoutForQuery { + t.Errorf("elapsed time was %v, should be bigger than timeout=%v", elapsed, timeoutForQuery) + } + + // Ensure we took more than timeout, but not too much + if elapsed >= timeoutForQuery+500*time.Millisecond { + t.Errorf("elapsed time was %v, should not be much bigger than timeout=%v", elapsed, timeoutForQuery) + } + + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From 30e8ab3856b4c94694f2866d21521819b7bd52ff Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sun, 28 Dec 2025 02:38:46 +0100 Subject: [PATCH 071/113] Update common Prometheus files (#1234) Signed-off-by: prombot --- .github/workflows/container_description.yml | 4 ++-- .github/workflows/golangci-lint.yml | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/container_description.yml b/.github/workflows/container_description.yml index 7de8bb8da..7b46e9532 100644 --- a/.github/workflows/container_description.yml +++ b/.github/workflows/container_description.yml @@ -18,7 +18,7 @@ jobs: if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks. steps: - name: git checkout - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 with: persist-credentials: false - name: Set docker hub repo name @@ -42,7 +42,7 @@ jobs: if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks. steps: - name: git checkout - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 with: persist-credentials: false - name: Set quay.io org name diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 75f886d54..2736e69b7 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -24,11 +24,11 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout repository - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 with: persist-credentials: false - name: Install Go - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0 + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0 with: go-version: 1.25.x - name: Install snmp_exporter/generator dependencies @@ -38,7 +38,7 @@ jobs: id: golangci-lint-version run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT - name: Lint - uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0 + uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0 with: args: --verbose version: ${{ steps.golangci-lint-version.outputs.version }} From c7a09a2563ee62ad6c589b99641c233e7fd81478 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sun, 4 Jan 2026 23:22:46 +0100 Subject: [PATCH 072/113] Update common Prometheus files (#1236) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- Makefile.common | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 2736e69b7..ae5fdc80e 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -38,7 +38,7 @@ jobs: id: golangci-lint-version run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT - name: Lint - uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0 + uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0 with: args: --verbose version: ${{ steps.golangci-lint-version.outputs.version }} diff --git a/Makefile.common b/Makefile.common index 840bc0ea7..998da2309 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.6.2 +GOLANGCI_LINT_VERSION ?= v2.7.2 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. From c285030bc69ea9a0a3fcda9f92356748ff1f1697 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 6 Jan 2026 09:34:38 +0100 Subject: [PATCH 073/113] Update common Prometheus files (#1237) Signed-off-by: prombot --- Makefile.common | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index 998da2309..7beae6e58 100644 --- a/Makefile.common +++ b/Makefile.common @@ -1,4 +1,4 @@ -# Copyright 2018 The Prometheus Authors +# Copyright The Prometheus Authors # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at @@ -129,6 +129,12 @@ common-check_license: echo "license header checking failed:"; echo "$${licRes}"; \ exit 1; \ fi + @echo ">> checking for copyright years 2026 or later" + @futureYearRes=$$(git grep -E 'Copyright (202[6-9]|20[3-9][0-9])' -- '*.go' ':!:vendor/*' || true); \ + if [ -n "$${futureYearRes}" ]; then \ + echo "Files with copyright year 2026 or later found (should use 'Copyright The Prometheus Authors'):"; echo "$${futureYearRes}"; \ + exit 1; \ + fi .PHONY: common-deps common-deps: From 7b2975a88371711dea73fe5ca578f3d441523833 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Sat, 10 Jan 2026 19:39:29 +0100 Subject: [PATCH 074/113] feat: allow excluding databases or users names in `stat_statements` (#1232) * feat: allow excluding databases or users names in `stat_statements` Add a couple of of options to the `stat_statements` collector that allow excluding certain database names or user names from metrics. This is useful if you don't care about certain preconfigured databases with admin activity. Signed-off-by: Cristian Greco * address review feedback Signed-off-by: Cristian Greco --------- Signed-off-by: Cristian Greco --- README.md | 9 + collector/pg_stat_statements.go | 65 ++++++- collector/pg_stat_statements_test.go | 265 +++++++++++++++++++++++++-- 3 files changed, 322 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 2654b319f..7a61135bb 100644 --- a/README.md +++ b/README.md @@ -159,6 +159,15 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `--collector.stat_statements.query_length` Maximum length of the statement text. Default is 120. +* `--collector.stat_statements.limit` + Maximum number of statements to return. Default is 100. + +* `--collector.stat_statements.exclude_databases` + Comma-separated list of database names to exclude from `pg_stat_statements` metrics. Default is none. + +* `--collector.stat_statements.exclude_users` + Comma-separated list of user names to exclude from `pg_stat_statements` metrics. Default is none. + * `[no-]collector.stat_user_tables` Enable the `stat_user_tables` collector (default: enabled). diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index 824605190..9227f5c04 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -18,6 +18,7 @@ import ( "database/sql" "fmt" "log/slog" + "strings" "github.com/alecthomas/kingpin/v2" "github.com/blang/semver/v4" @@ -30,9 +31,11 @@ const ( ) var ( - includeQueryFlag *bool = nil - statementLengthFlag *uint = nil - statementLimitFlag *uint = nil + includeQueryFlag *bool = nil + statementLengthFlag *uint = nil + statementLimitFlag *uint = nil + excludedDatabasesFlag *string = nil + excludedUsersFlag *string = nil ) func init() { @@ -56,6 +59,16 @@ func init() { "Maximum number of statements to return."). Default(defaultStatementLimit). Uint() + excludedDatabasesFlag = kingpin.Flag( + fmt.Sprint(collectorFlagPrefix, statStatementsSubsystem, ".exclude_databases"), + "Comma-separated list of database names to exclude. (default: none)"). + Default(""). + String() + excludedUsersFlag = kingpin.Flag( + fmt.Sprint(collectorFlagPrefix, statStatementsSubsystem, ".exclude_users"), + "Comma-separated list of user names to exclude. (default: none)"). + Default(""). + String() } type PGStatStatementsCollector struct { @@ -63,14 +76,36 @@ type PGStatStatementsCollector struct { includeQueryStatement bool statementLength uint statementLimit uint + excludedDatabases []string + excludedUsers []string } func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { + var excludedDatabases []string + if *excludedDatabasesFlag != "" { + for db := range strings.SplitSeq(*excludedDatabasesFlag, ",") { + if trimmed := strings.TrimSpace(db); trimmed != "" { + excludedDatabases = append(excludedDatabases, trimmed) + } + } + } + + var excludedUsers []string + if *excludedUsersFlag != "" { + for user := range strings.SplitSeq(*excludedUsersFlag, ",") { + if trimmed := strings.TrimSpace(user); trimmed != "" { + excludedUsers = append(excludedUsers, trimmed) + } + } + } + return &PGStatStatementsCollector{ log: config.logger, includeQueryStatement: *includeQueryFlag, statementLength: *statementLengthFlag, statementLimit: *statementLimitFlag, + excludedDatabases: excludedDatabases, + excludedUsers: excludedUsers, }, nil } @@ -115,7 +150,9 @@ var ( ) const ( - pgStatStatementQuerySelect = `LEFT(pg_stat_statements.query, %d) as query,` + pgStatStatementQuerySelect = `LEFT(pg_stat_statements.query, %d) as query,` + pgStatStatementExcludeDatabases = `AND pg_database.datname NOT IN (%s) ` + pgStatStatementExcludeUsers = `AND pg_get_userbyid(userid) NOT IN (%s) ` pgStatStatementsQuery = `SELECT pg_get_userbyid(userid) as user, @@ -136,6 +173,7 @@ const ( WITHIN GROUP (ORDER BY total_time) FROM pg_stat_statements ) + %s %s ORDER BY seconds_total DESC LIMIT %s;` @@ -158,6 +196,7 @@ const ( WITHIN GROUP (ORDER BY total_exec_time) FROM pg_stat_statements ) + %s %s ORDER BY seconds_total DESC LIMIT %s;` @@ -180,6 +219,7 @@ const ( WITHIN GROUP (ORDER BY total_exec_time) FROM pg_stat_statements ) + %s %s ORDER BY seconds_total DESC LIMIT %s;` ) @@ -198,11 +238,13 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc if c.includeQueryStatement { querySelect = fmt.Sprintf(pgStatStatementQuerySelect, c.statementLength) } + databaseFilter := c.buildExclusionClause(c.excludedDatabases, pgStatStatementExcludeDatabases) + userFilter := c.buildExclusionClause(c.excludedUsers, pgStatStatementExcludeUsers) statementLimit := defaultStatementLimit if c.statementLimit > 0 { statementLimit = fmt.Sprintf("%d", c.statementLimit) } - query := fmt.Sprintf(queryTemplate, querySelect, statementLimit) + query := fmt.Sprintf(queryTemplate, querySelect, databaseFilter, userFilter, statementLimit) db := instance.getDB() rows, err := db.QueryContext(ctx, query) @@ -319,3 +361,16 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc } return nil } + +func (c PGStatStatementsCollector) buildExclusionClause(identifiers []string, clauseTemplate string) string { + if len(identifiers) == 0 { + return "" + } + + escaped := make([]string, 0, len(identifiers)) + for _, identifier := range identifiers { + escaped = append(escaped, fmt.Sprintf("'%s'", strings.ReplaceAll(identifier, "'", "''"))) + } + + return fmt.Sprintf(clauseTemplate, strings.Join(escaped, ", ")) +} diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go index 8763f693d..aea08b244 100644 --- a/collector/pg_stat_statements_test.go +++ b/collector/pg_stat_statements_test.go @@ -36,7 +36,7 @@ func TestPGStatStatementsCollector(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, "", "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -79,7 +79,7 @@ func TestPGStatStatementsCollectorWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 100) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -123,7 +123,7 @@ func TestPGStatStatementsCollectorWithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 100) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, fmt.Sprintf(pgStatStatementQuerySelect, 100), "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -167,7 +167,7 @@ func TestPGStatStatementsCollectorNull(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -210,7 +210,7 @@ func TestPGStatStatementsCollectorNullWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -254,7 +254,7 @@ func TestPGStatStatementsCollectorNullWithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -298,7 +298,7 @@ func TestPGStatStatementsCollector_PG13(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -341,7 +341,7 @@ func TestPGStatStatementsCollector_PG13_WithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -385,7 +385,7 @@ func TestPGStatStatementsCollector_PG13_WithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -429,7 +429,7 @@ func TestPGStatStatementsCollector_PG17(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -472,7 +472,7 @@ func TestPGStatStatementsCollector_PG17_WithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -516,7 +516,7 @@ func TestPGStatStatementsCollector_PG17_WithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -547,3 +547,244 @@ func TestPGStatStatementsCollector_PG17_WithStatementAndLimit(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPGStatStatementsCollectorWithExcludedDatabases(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("12.0.0")} + + excludedDatabases := []string{"rdsadmin", "cloudsqladmin"} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + + // Expected query should include database exclusion filters + expectedFilter := " AND pg_database.datname NOT IN ('rdsadmin', 'cloudsqladmin')" + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, "", expectedFilter, "", defaultStatementLimit))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{excludedDatabases: excludedDatabases} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison with excluded databases", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollectorWithExcludedDatabasesSQLEscaping(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.0.0")} + + // exclude a database name with a single quote (SQL injection test) + excludedDatabases := []string{"test'db"} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + + expectedFilter := " AND pg_database.datname NOT IN ('test''db')" + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", expectedFilter, "", defaultStatementLimit))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{excludedDatabases: excludedDatabases} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison with SQL escaping", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollectorWithExcludedUsers(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("12.0.0")} + + excludedUsers := []string{"monitoring", "readonly"} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + + expectedFilter := " AND pg_get_userbyid(userid) NOT IN ('monitoring', 'readonly')" + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery, "", "", expectedFilter, defaultStatementLimit))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{excludedUsers: excludedUsers} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison with excluded users", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollectorWithExcludedUsersSQLEscaping(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.0.0")} + + // exclude a user name with a single quote (SQL injection test) + excludedUsers := []string{"test'user"} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + + expectedFilter := " AND pg_get_userbyid(userid) NOT IN ('test''user')" + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", expectedFilter, defaultStatementLimit))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{excludedUsers: excludedUsers} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison with SQL escaping for users", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatStatementsCollectorWithExcludedDatabasesAndUsers(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("17.0.0")} + + // exclude both databases and users + excludedDatabases := []string{"rdsadmin", "cloudsqladmin"} + excludedUsers := []string{"monitoring", "readonly"} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + + expectedDbFilter := " AND pg_database.datname NOT IN ('rdsadmin', 'cloudsqladmin')" + expectedUserFilter := " AND pg_get_userbyid(userid) NOT IN ('monitoring', 'readonly')" + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", expectedDbFilter, expectedUserFilter, defaultStatementLimit))).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{excludedDatabases: excludedDatabases, excludedUsers: excludedUsers} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison with excluded databases and users", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From de76d83da78aa2ba915853c113323f62bdc6cb22 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Fri, 16 Jan 2026 20:07:56 +0100 Subject: [PATCH 075/113] ci: add postgres 18.1 to test matrix (#1239) Signed-off-by: Cristian Greco --- .github/workflows/ci.yml | 1 + README.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3150f65f3..9d759a627 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -30,6 +30,7 @@ jobs: - 15.14 - 16.10 - 17.6 + - 18.1 services: postgres: image: postgres:${{ matrix.postgres_version }} diff --git a/README.md b/README.md index 7a61135bb..d0cdf5947 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Prometheus exporter for PostgreSQL server metrics. -CI Tested PostgreSQL versions: `13`, `14`, `15`, `16`, `17`. +CI Tested PostgreSQL versions: `13`, `14`, `15`, `16`, `17`, `18`. ## Quick Start This package is available for Docker: From 9599702cbd67cd7a8bb26dddcf4b62867900f9bd Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Thu, 22 Jan 2026 09:53:34 +0100 Subject: [PATCH 076/113] Update common Prometheus files (#1243) Signed-off-by: prombot --- Makefile.common | 97 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 86 insertions(+), 11 deletions(-) diff --git a/Makefile.common b/Makefile.common index 7beae6e58..e3adb8958 100644 --- a/Makefile.common +++ b/Makefile.common @@ -82,11 +82,32 @@ endif PREFIX ?= $(shell pwd) BIN_DIR ?= $(shell pwd) DOCKER_IMAGE_TAG ?= $(subst /,-,$(shell git rev-parse --abbrev-ref HEAD)) -DOCKERFILE_PATH ?= ./Dockerfile DOCKERBUILD_CONTEXT ?= ./ DOCKER_REPO ?= prom +# Check if deprecated DOCKERFILE_PATH is set +ifdef DOCKERFILE_PATH +$(error DOCKERFILE_PATH is deprecated. Use DOCKERFILE_VARIANTS ?= $(DOCKERFILE_PATH) in the Makefile) +endif + DOCKER_ARCHS ?= amd64 +DOCKERFILE_VARIANTS ?= Dockerfile $(wildcard Dockerfile.*) + +# Function to extract variant from Dockerfile label. +# Returns the variant name from io.prometheus.image.variant label, or "default" if not found. +define dockerfile_variant +$(strip $(or $(shell sed -n 's/.*io\.prometheus\.image\.variant="\([^"]*\)".*/\1/p' $(1)),default)) +endef + +# Check for duplicate variant names (including default for Dockerfiles without labels). +DOCKERFILE_VARIANT_NAMES := $(foreach df,$(DOCKERFILE_VARIANTS),$(call dockerfile_variant,$(df))) +DOCKERFILE_VARIANT_NAMES_SORTED := $(sort $(DOCKERFILE_VARIANT_NAMES)) +ifneq ($(words $(DOCKERFILE_VARIANT_NAMES)),$(words $(DOCKERFILE_VARIANT_NAMES_SORTED))) +$(error Duplicate variant names found. Each Dockerfile must have a unique io.prometheus.image.variant label, and only one can be without a label (default)) +endif + +# Build variant:dockerfile pairs for shell iteration. +DOCKERFILE_VARIANTS_WITH_NAMES := $(foreach df,$(DOCKERFILE_VARIANTS),$(call dockerfile_variant,$(df)):$(df)) BUILD_DOCKER_ARCHS = $(addprefix common-docker-,$(DOCKER_ARCHS)) PUBLISH_DOCKER_ARCHS = $(addprefix common-docker-publish-,$(DOCKER_ARCHS)) @@ -226,28 +247,82 @@ common-docker-repo-name: .PHONY: common-docker $(BUILD_DOCKER_ARCHS) common-docker: $(BUILD_DOCKER_ARCHS) $(BUILD_DOCKER_ARCHS): common-docker-%: - docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" \ - -f $(DOCKERFILE_PATH) \ - --build-arg ARCH="$*" \ - --build-arg OS="linux" \ - $(DOCKERBUILD_CONTEXT) + @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ + dockerfile=$${variant#*:}; \ + variant_name=$${variant%%:*}; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Building default variant ($$variant_name) for linux-$* using $$dockerfile"; \ + docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" \ + -f $$dockerfile \ + --build-arg ARCH="$*" \ + --build-arg OS="linux" \ + $(DOCKERBUILD_CONTEXT); \ + if [ "$$variant_name" != "default" ]; then \ + echo "Tagging default variant with $$variant_name suffix"; \ + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" \ + "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ + fi; \ + else \ + echo "Building $$variant_name variant for linux-$* using $$dockerfile"; \ + docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" \ + -f $$dockerfile \ + --build-arg ARCH="$*" \ + --build-arg OS="linux" \ + $(DOCKERBUILD_CONTEXT); \ + fi; \ + done .PHONY: common-docker-publish $(PUBLISH_DOCKER_ARCHS) common-docker-publish: $(PUBLISH_DOCKER_ARCHS) $(PUBLISH_DOCKER_ARCHS): common-docker-publish-%: - docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" + @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ + dockerfile=$${variant#*:}; \ + variant_name=$${variant%%:*}; \ + if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ + echo "Pushing $$variant_name variant for linux-$*"; \ + docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ + fi; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Pushing default variant ($$variant_name) for linux-$*"; \ + docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)"; \ + fi; \ + done DOCKER_MAJOR_VERSION_TAG = $(firstword $(subst ., ,$(shell cat VERSION))) .PHONY: common-docker-tag-latest $(TAG_DOCKER_ARCHS) common-docker-tag-latest: $(TAG_DOCKER_ARCHS) $(TAG_DOCKER_ARCHS): common-docker-tag-latest-%: - docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest" - docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)" + @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ + dockerfile=$${variant#*:}; \ + variant_name=$${variant%%:*}; \ + if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ + echo "Tagging $$variant_name variant for linux-$* as latest"; \ + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest-$$variant_name"; \ + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ + fi; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Tagging default variant ($$variant_name) for linux-$* as latest"; \ + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest"; \ + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)"; \ + fi; \ + done .PHONY: common-docker-manifest common-docker-manifest: - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)) - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" + @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ + dockerfile=$${variant#*:}; \ + variant_name=$${variant%%:*}; \ + if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ + echo "Creating manifest for $$variant_name variant"; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name); \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ + fi; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Creating default variant ($$variant_name) manifest"; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)); \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)"; \ + fi; \ + done .PHONY: promu promu: $(PROMU) From 47d0100bc35cde5c6ed628d4b5808101d6c1b228 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 23 Jan 2026 09:32:09 +0100 Subject: [PATCH 077/113] Update common Prometheus files (#1244) Signed-off-by: prombot --- Makefile.common | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/Makefile.common b/Makefile.common index e3adb8958..b8c9b3844 100644 --- a/Makefile.common +++ b/Makefile.common @@ -250,12 +250,17 @@ $(BUILD_DOCKER_ARCHS): common-docker-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ + distroless_arch="$*"; \ + if [ "$*" = "armv7" ]; then \ + distroless_arch="arm"; \ + fi; \ if [ "$$dockerfile" = "Dockerfile" ]; then \ echo "Building default variant ($$variant_name) for linux-$* using $$dockerfile"; \ docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" \ -f $$dockerfile \ --build-arg ARCH="$*" \ --build-arg OS="linux" \ + --build-arg DISTROLESS_ARCH="$$distroless_arch" \ $(DOCKERBUILD_CONTEXT); \ if [ "$$variant_name" != "default" ]; then \ echo "Tagging default variant with $$variant_name suffix"; \ @@ -268,6 +273,7 @@ $(BUILD_DOCKER_ARCHS): common-docker-%: -f $$dockerfile \ --build-arg ARCH="$*" \ --build-arg OS="linux" \ + --build-arg DISTROLESS_ARCH="$$distroless_arch" \ $(DOCKERBUILD_CONTEXT); \ fi; \ done @@ -286,6 +292,16 @@ $(PUBLISH_DOCKER_ARCHS): common-docker-publish-%: echo "Pushing default variant ($$variant_name) for linux-$*"; \ docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)"; \ fi; \ + if [ "$(DOCKER_IMAGE_TAG)" = "latest" ]; then \ + if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ + echo "Pushing $$variant_name variant version tags for linux-$*"; \ + docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ + fi; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Pushing default variant version tag for linux-$*"; \ + docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)"; \ + fi; \ + fi; \ done DOCKER_MAJOR_VERSION_TAG = $(firstword $(subst ., ,$(shell cat VERSION))) @@ -322,6 +338,18 @@ common-docker-manifest: DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)); \ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)"; \ fi; \ + if [ "$(DOCKER_IMAGE_TAG)" = "latest" ]; then \ + if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ + echo "Creating manifest for $$variant_name variant version tag"; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name); \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ + fi; \ + if [ "$$dockerfile" = "Dockerfile" ]; then \ + echo "Creating default variant version tag manifest"; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):v$(DOCKER_MAJOR_VERSION_TAG)); \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)"; \ + fi; \ + fi; \ done .PHONY: promu From 3c0720a1bc0d02587617d058b79cf1083b851122 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 28 Jan 2026 21:42:24 +0100 Subject: [PATCH 078/113] feat: reorganize code to remove the main package (#1238) * feat: reorganize code to remove the main package Move all `main` code to `exporter`, for easier code embedding. Signed-off-by: Cristian Greco * move logger args to NewExporter signature Signed-off-by: Cristian Greco --------- Signed-off-by: Cristian Greco --- cmd/postgres_exporter/main.go | 44 +++++------- cmd/postgres_exporter/probe.go | 22 +++--- .../datasource.go | 14 ++-- .../namespace.go | 20 +++--- .../pg_setting.go | 4 +- .../pg_setting_test.go | 2 +- .../postgres_exporter.go | 71 +++++++++++++------ .../postgres_exporter_integration_test.go | 13 ++-- .../postgres_exporter_test.go | 25 ++++--- .../postgres_exporter => exporter}/queries.go | 21 +++--- {cmd/postgres_exporter => exporter}/server.go | 20 ++++-- .../docker-postgres-replication/Dockerfile | 0 .../docker-postgres-replication/Dockerfile.p2 | 0 .../docker-postgres-replication/README.md | 0 .../docker-compose.yml | 0 .../docker-entrypoint.sh | 0 .../setup-replication.sh | 0 .../tests/test-smoke | 0 .../tests/user_queries_ok.yaml | 0 .../tests/user_queries_test.yaml | 0 .../tests/username_file | 0 .../tests/userpass_file | 0 {cmd/postgres_exporter => exporter}/util.go | 7 +- 23 files changed, 149 insertions(+), 114 deletions(-) rename {cmd/postgres_exporter => exporter}/datasource.go (89%) rename {cmd/postgres_exporter => exporter}/namespace.go (92%) rename {cmd/postgres_exporter => exporter}/pg_setting.go (98%) rename {cmd/postgres_exporter => exporter}/pg_setting_test.go (99%) rename {cmd/postgres_exporter => exporter}/postgres_exporter.go (93%) rename {cmd/postgres_exporter => exporter}/postgres_exporter_integration_test.go (92%) rename {cmd/postgres_exporter => exporter}/postgres_exporter_test.go (95%) rename {cmd/postgres_exporter => exporter}/queries.go (92%) rename {cmd/postgres_exporter => exporter}/server.go (89%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/Dockerfile (100%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/Dockerfile.p2 (100%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/README.md (100%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/docker-compose.yml (100%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/docker-entrypoint.sh (100%) rename {cmd/postgres_exporter => exporter}/tests/docker-postgres-replication/setup-replication.sh (100%) rename {cmd/postgres_exporter => exporter}/tests/test-smoke (100%) rename {cmd/postgres_exporter => exporter}/tests/user_queries_ok.yaml (100%) rename {cmd/postgres_exporter => exporter}/tests/user_queries_test.yaml (100%) rename {cmd/postgres_exporter => exporter}/tests/username_file (100%) rename {cmd/postgres_exporter => exporter}/tests/userpass_file (100%) rename {cmd/postgres_exporter => exporter}/util.go (96%) diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index c901ec722..0c1bd003e 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -23,6 +23,7 @@ import ( "github.com/alecthomas/kingpin/v2" "github.com/prometheus-community/postgres_exporter/collector" "github.com/prometheus-community/postgres_exporter/config" + "github.com/prometheus-community/postgres_exporter/exporter" "github.com/prometheus/client_golang/prometheus" versioncollector "github.com/prometheus/client_golang/prometheus/collectors/version" "github.com/prometheus/client_golang/prometheus/promhttp" @@ -54,20 +55,8 @@ var ( logger = promslog.NewNopLogger() ) -// Metric name parts. -const ( - // Namespace for all metrics. - namespace = "pg" - // Subsystems. - exporter = "exporter" - // The name of the exporter. - exporterName = "postgres_exporter" - // Metric label used for static string data thats handy to send to Prometheus - // e.g. version - staticLabelName = "static" - // Metric label used for server identification. - serverLabelName = "server" -) +// The name of the exporter. +const exporterName = "postgres_exporter" func main() { kingpin.Version(version.Print(exporterName)) @@ -78,7 +67,7 @@ func main() { logger = promslog.New(promslogConfig) if *onlyDumpMaps { - dumpMaps() + exporter.DumpMaps() return } @@ -87,7 +76,7 @@ func main() { logger.Warn("Error loading config", "err", err) } - dsns, err := getDataSources() + dsns, err := exporter.GetDataSources() if err != nil { logger.Error("Failed reading data sources", "err", err.Error()) os.Exit(1) @@ -108,19 +97,20 @@ func main() { logger.Warn("Constant labels on all metrics is DEPRECATED") } - opts := []ExporterOpt{ - DisableDefaultMetrics(*disableDefaultMetrics), - DisableSettingsMetrics(*disableSettingsMetrics), - AutoDiscoverDatabases(*autoDiscoverDatabases), - WithUserQueriesPath(*queriesPath), - WithConstantLabels(*constantLabelsList), - ExcludeDatabases(excludedDatabases), - IncludeDatabases(*includeDatabases), + opts := []exporter.ExporterOpt{ + exporter.DisableDefaultMetrics(*disableDefaultMetrics), + exporter.DisableSettingsMetrics(*disableSettingsMetrics), + exporter.AutoDiscoverDatabases(*autoDiscoverDatabases), + exporter.WithUserQueriesPath(*queriesPath), + exporter.WithConstantLabels(*constantLabelsList), + exporter.ExcludeDatabases(excludedDatabases), + exporter.IncludeDatabases(*includeDatabases), + exporter.WithMetricPrefix(*metricPrefix), } - exporter := NewExporter(dsns, opts...) + exporter := exporter.NewExporter(dsns, logger, opts...) defer func() { - exporter.servers.Close() + exporter.CloseServers() }() prometheus.MustRegister(versioncollector.NewCollector(exporterName)) diff --git a/cmd/postgres_exporter/probe.go b/cmd/postgres_exporter/probe.go index 2c8c7652e..66219c07d 100644 --- a/cmd/postgres_exporter/probe.go +++ b/cmd/postgres_exporter/probe.go @@ -20,6 +20,7 @@ import ( "github.com/prometheus-community/postgres_exporter/collector" "github.com/prometheus-community/postgres_exporter/config" + "github.com/prometheus-community/postgres_exporter/exporter" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promhttp" ) @@ -64,20 +65,21 @@ func handleProbe(logger *slog.Logger, excludeDatabases []string) http.HandlerFun registry := prometheus.NewRegistry() - opts := []ExporterOpt{ - DisableDefaultMetrics(*disableDefaultMetrics), - DisableSettingsMetrics(*disableSettingsMetrics), - AutoDiscoverDatabases(*autoDiscoverDatabases), - WithUserQueriesPath(*queriesPath), - WithConstantLabels(*constantLabelsList), - ExcludeDatabases(excludeDatabases), - IncludeDatabases(*includeDatabases), + opts := []exporter.ExporterOpt{ + exporter.DisableDefaultMetrics(*disableDefaultMetrics), + exporter.DisableSettingsMetrics(*disableSettingsMetrics), + exporter.AutoDiscoverDatabases(*autoDiscoverDatabases), + exporter.WithUserQueriesPath(*queriesPath), + exporter.WithConstantLabels(*constantLabelsList), + exporter.ExcludeDatabases(excludeDatabases), + exporter.IncludeDatabases(*includeDatabases), + exporter.WithMetricPrefix(*metricPrefix), } dsns := []string{dsn.GetConnectionString()} - exporter := NewExporter(dsns, opts...) + exporter := exporter.NewExporter(dsns, logger, opts...) defer func() { - exporter.servers.Close() + exporter.CloseServers() }() registry.MustRegister(exporter) diff --git a/cmd/postgres_exporter/datasource.go b/exporter/datasource.go similarity index 89% rename from cmd/postgres_exporter/datasource.go rename to exporter/datasource.go index cebe5072e..9ae3410fe 100644 --- a/cmd/postgres_exporter/datasource.go +++ b/exporter/datasource.go @@ -11,7 +11,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "fmt" @@ -39,19 +39,19 @@ func (e *Exporter) discoverDatabaseDSNs() []string { var err error dsnURI, err = url.Parse(dsn) if err != nil { - logger.Error("Unable to parse DSN as URI", "dsn", loggableDSN(dsn), "err", err) + e.logger.Error("Unable to parse DSN as URI", "dsn", loggableDSN(dsn), "err", err) continue } } else if connstringRe.MatchString(dsn) { dsnConnstring = dsn } else { - logger.Error("Unable to parse DSN as either URI or connstring", "dsn", loggableDSN(dsn)) + e.logger.Error("Unable to parse DSN as either URI or connstring", "dsn", loggableDSN(dsn)) continue } server, err := e.servers.GetServer(dsn) if err != nil { - logger.Error("Error opening connection to database", "dsn", loggableDSN(dsn), "err", err) + e.logger.Error("Error opening connection to database", "dsn", loggableDSN(dsn), "err", err) continue } dsns[dsn] = struct{}{} @@ -61,7 +61,7 @@ func (e *Exporter) discoverDatabaseDSNs() []string { databaseNames, err := queryDatabases(server) if err != nil { - logger.Error("Error querying databases", "dsn", loggableDSN(dsn), "err", err) + e.logger.Error("Error querying databases", "dsn", loggableDSN(dsn), "err", err) continue } for _, databaseName := range databaseNames { @@ -109,7 +109,7 @@ func (e *Exporter) scrapeDSN(ch chan<- prometheus.Metric, dsn string) error { // Check if map versions need to be updated if err := e.checkMapVersions(ch, server); err != nil { - logger.Warn("Proceeding with outdated query maps, as the Postgres version could not be determined", "err", err) + e.logger.Warn("Proceeding with outdated query maps, as the Postgres version could not be determined", "err", err) } return server.Scrape(ch, e.disableSettingsMetrics) @@ -119,7 +119,7 @@ func (e *Exporter) scrapeDSN(ch chan<- prometheus.Metric, dsn string) error { // DATA_SOURCE_NAME always wins so we do not break older versions // reading secrets from files wins over secrets in environment variables // DATA_SOURCE_NAME > DATA_SOURCE_{USER|PASS}_FILE > DATA_SOURCE_{USER|PASS} -func getDataSources() ([]string, error) { +func GetDataSources() ([]string, error) { var dsn = os.Getenv("DATA_SOURCE_NAME") if len(dsn) != 0 { return strings.Split(dsn, ","), nil diff --git a/cmd/postgres_exporter/namespace.go b/exporter/namespace.go similarity index 92% rename from cmd/postgres_exporter/namespace.go rename to exporter/namespace.go index ac7a23739..5f6bd8edc 100644 --- a/cmd/postgres_exporter/namespace.go +++ b/exporter/namespace.go @@ -11,7 +11,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "database/sql" @@ -129,7 +129,7 @@ func queryNamespaceMapping(server *Server, namespace string, mapping MetricMapNa nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Missing column: ", namespace, columnName+"_sum"))) continue } - sum, ok := dbToFloat64(columnData[idx]) + sum, ok := dbToFloat64(columnData[idx], server.logger) if !ok { nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Unexpected error parsing column: ", namespace, columnName+"_sum", columnData[idx]))) continue @@ -140,7 +140,7 @@ func queryNamespaceMapping(server *Server, namespace string, mapping MetricMapNa nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Missing column: ", namespace, columnName+"_count"))) continue } - count, ok := dbToUint64(columnData[idx]) + count, ok := dbToUint64(columnData[idx], server.logger) if !ok { nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Unexpected error parsing column: ", namespace, columnName+"_count", columnData[idx]))) continue @@ -152,7 +152,7 @@ func queryNamespaceMapping(server *Server, namespace string, mapping MetricMapNa labels..., ) } else { - value, ok := dbToFloat64(columnData[idx]) + value, ok := dbToFloat64(columnData[idx], server.logger) if !ok { nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Unexpected error parsing column: ", namespace, columnName, columnData[idx]))) continue @@ -167,7 +167,7 @@ func queryNamespaceMapping(server *Server, namespace string, mapping MetricMapNa // Its not an error to fail here, since the values are // unexpected anyway. - value, ok := dbToFloat64(columnData[idx]) + value, ok := dbToFloat64(columnData[idx], server.logger) if !ok { nonfatalErrors = append(nonfatalErrors, errors.New(fmt.Sprintln("Unparseable column type - discarding: ", namespace, columnName, err))) continue @@ -189,10 +189,10 @@ func queryNamespaceMappings(ch chan<- prometheus.Metric, server *Server) map[str scrapeStart := time.Now() for namespace, mapping := range server.metricMap { - logger.Debug("Querying namespace", "namespace", namespace) + server.logger.Debug("Querying namespace", "namespace", namespace) if mapping.master && !server.master { - logger.Debug("Query skipped...") + server.logger.Debug("Query skipped...") continue } @@ -201,7 +201,7 @@ func queryNamespaceMappings(ch chan<- prometheus.Metric, server *Server) map[str serVersion, _ := semver.Parse(server.lastMapVersion.String()) runServerRange, _ := semver.ParseRange(server.runonserver) if !runServerRange(serVersion) { - logger.Debug("Query skipped for this database version", "version", server.lastMapVersion.String(), "target_version", server.runonserver) + server.logger.Debug("Query skipped for this database version", "version", server.lastMapVersion.String(), "target_version", server.runonserver) continue } } @@ -232,12 +232,12 @@ func queryNamespaceMappings(ch chan<- prometheus.Metric, server *Server) map[str // Serious error - a namespace disappeared if err != nil { namespaceErrors[namespace] = err - logger.Info("error finding namespace", "err", err) + server.logger.Info("error finding namespace", "err", err) } // Non-serious errors - likely version or parsing problems. if len(nonFatalErrors) > 0 { for _, err := range nonFatalErrors { - logger.Info("error querying namespace", "err", err) + server.logger.Info("error querying namespace", "err", err) } } diff --git a/cmd/postgres_exporter/pg_setting.go b/exporter/pg_setting.go similarity index 98% rename from cmd/postgres_exporter/pg_setting.go rename to exporter/pg_setting.go index 5b13e160f..b7adfa25f 100644 --- a/cmd/postgres_exporter/pg_setting.go +++ b/exporter/pg_setting.go @@ -11,7 +11,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "fmt" @@ -31,7 +31,7 @@ var ( // Query the pg_settings view containing runtime variables func querySettings(ch chan<- prometheus.Metric, server *Server) error { - logger.Debug("Querying pg_setting view", "server", server) + server.logger.Debug("Querying pg_setting view", "server", server) // pg_settings docs: https://www.postgresql.org/docs/current/static/view-pg-settings.html // diff --git a/cmd/postgres_exporter/pg_setting_test.go b/exporter/pg_setting_test.go similarity index 99% rename from cmd/postgres_exporter/pg_setting_test.go rename to exporter/pg_setting_test.go index 9acdeaed7..7038959fd 100644 --- a/cmd/postgres_exporter/pg_setting_test.go +++ b/exporter/pg_setting_test.go @@ -13,7 +13,7 @@ //go:build !integration -package main +package exporter import ( "github.com/prometheus/client_golang/prometheus" diff --git a/cmd/postgres_exporter/postgres_exporter.go b/exporter/postgres_exporter.go similarity index 93% rename from cmd/postgres_exporter/postgres_exporter.go rename to exporter/postgres_exporter.go index a76479611..720a4667f 100644 --- a/cmd/postgres_exporter/postgres_exporter.go +++ b/exporter/postgres_exporter.go @@ -11,13 +11,14 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "crypto/sha256" "database/sql" "errors" "fmt" + "log/slog" "math" "os" "regexp" @@ -28,6 +29,19 @@ import ( "github.com/prometheus/client_golang/prometheus" ) +// Metric name parts. +const ( + // Namespace for all metrics. + namespace = "pg" + // Subsystems. + exporter = "exporter" + // Metric label used for static string data thats handy to send to Prometheus + // e.g. version + staticLabelName = "static" + // Metric label used for server identification. + serverLabelName = "server" +) + // ColumnUsage should be one of several enum values which describe how a // queried row is to be converted to a Prometheus metric. type ColumnUsage int @@ -142,7 +156,7 @@ func (e *ErrorConnectToServer) Error() string { } // TODO: revisit this with the semver system -func dumpMaps() { +func DumpMaps() { // TODO: make this function part of the exporter for name, cmap := range builtinMetricMaps { query, ok := queryOverrides[name] @@ -263,13 +277,13 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ } // Turn the MetricMap column mapping into a prometheus descriptor mapping. -func makeDescMap(pgVersion semver.Version, serverLabels prometheus.Labels, metricMaps map[string]intermediateMetricMap) map[string]MetricMapNamespace { +func makeDescMap(pgVersion semver.Version, serverLabels prometheus.Labels, metricMaps map[string]intermediateMetricMap, logger *slog.Logger, metricPrefix string) map[string]MetricMapNamespace { var metricMap = make(map[string]MetricMapNamespace) for namespace, intermediateMappings := range metricMaps { thisMap := make(map[string]MetricMap) - namespace = strings.Replace(namespace, "pg", *metricPrefix, 1) + namespace = strings.Replace(namespace, "pg", metricPrefix, 1) // Get the constant labels var variableLabels []string @@ -312,7 +326,7 @@ func makeDescMap(pgVersion semver.Version, serverLabels prometheus.Labels, metri vtype: prometheus.CounterValue, desc: prometheus.NewDesc(fmt.Sprintf("%s_%s", namespace, columnName), columnMapping.description, variableLabels, serverLabels), conversion: func(in interface{}) (float64, bool) { - return dbToFloat64(in) + return dbToFloat64(in, logger) }, } case GAUGE: @@ -320,7 +334,7 @@ func makeDescMap(pgVersion semver.Version, serverLabels prometheus.Labels, metri vtype: prometheus.GaugeValue, desc: prometheus.NewDesc(fmt.Sprintf("%s_%s", namespace, columnName), columnMapping.description, variableLabels, serverLabels), conversion: func(in interface{}) (float64, bool) { - return dbToFloat64(in) + return dbToFloat64(in, logger) }, } case HISTOGRAM: @@ -329,7 +343,7 @@ func makeDescMap(pgVersion semver.Version, serverLabels prometheus.Labels, metri vtype: prometheus.UntypedValue, desc: prometheus.NewDesc(fmt.Sprintf("%s_%s", namespace, columnName), columnMapping.description, variableLabels, serverLabels), conversion: func(in interface{}) (float64, bool) { - return dbToFloat64(in) + return dbToFloat64(in, logger) }, } thisMap[columnName+"_bucket"] = MetricMap{ @@ -425,6 +439,9 @@ type Exporter struct { // servers are used to allow re-using the DB connection between scrapes. // servers contains metrics map and query overrides. servers *Servers + + logger *slog.Logger + metricPrefix string } // ExporterOpt configures Exporter. @@ -477,11 +494,18 @@ func WithUserQueriesPath(p string) ExporterOpt { // WithConstantLabels configures constant labels. func WithConstantLabels(s string) ExporterOpt { return func(e *Exporter) { - e.constantLabels = parseConstLabels(s) + e.constantLabels = parseConstLabels(s, e.logger) + } +} + +// WithMetricPrefix configures metric prefix. +func WithMetricPrefix(prefix string) ExporterOpt { + return func(e *Exporter) { + e.metricPrefix = prefix } } -func parseConstLabels(s string) prometheus.Labels { +func parseConstLabels(s string, logger *slog.Logger) prometheus.Labels { labels := make(prometheus.Labels) s = strings.TrimSpace(s) @@ -508,10 +532,11 @@ func parseConstLabels(s string) prometheus.Labels { } // NewExporter returns a new PostgreSQL exporter for the provided DSN. -func NewExporter(dsn []string, opts ...ExporterOpt) *Exporter { +func NewExporter(dsn []string, logger *slog.Logger, opts ...ExporterOpt) *Exporter { e := &Exporter{ dsn: dsn, builtinMetricMaps: builtinMetricMaps, + logger: logger, } for _, opt := range opts { @@ -519,7 +544,7 @@ func NewExporter(dsn []string, opts ...ExporterOpt) *Exporter { } e.setupInternalMetrics() - e.servers = NewServers(ServerWithLabels(e.constantLabels)) + e.servers = NewServers(ServerWithLabels(e.constantLabels), ServerWithLogger(e.logger)) return e } @@ -576,6 +601,10 @@ func (e *Exporter) Collect(ch chan<- prometheus.Metric) { e.userQueriesError.Collect(ch) } +func (e *Exporter) CloseServers() { + e.servers.Close() +} + func newDesc(subsystem, name, help string, labels prometheus.Labels) *prometheus.Desc { return prometheus.NewDesc( prometheus.BuildFQName(namespace, subsystem, name), @@ -583,7 +612,7 @@ func newDesc(subsystem, name, help string, labels prometheus.Labels) *prometheus ) } -func checkPostgresVersion(db *sql.DB, server string) (semver.Version, string, error) { +func checkPostgresVersion(db *sql.DB, server string, logger *slog.Logger) (semver.Version, string, error) { logger.Debug("Querying PostgreSQL version", "server", server) versionRow := db.QueryRow("SELECT version();") var versionString string @@ -601,24 +630,24 @@ func checkPostgresVersion(db *sql.DB, server string) (semver.Version, string, er // Check and update the exporters query maps if the version has changed. func (e *Exporter) checkMapVersions(ch chan<- prometheus.Metric, server *Server) error { - semanticVersion, versionString, err := checkPostgresVersion(server.db, server.String()) + semanticVersion, versionString, err := checkPostgresVersion(server.db, server.String(), e.logger) if err != nil { return fmt.Errorf("Error fetching version string on %q: %v", server, err) } if !e.disableDefaultMetrics && semanticVersion.LT(lowestSupportedVersion) { - logger.Warn("PostgreSQL version is lower than our lowest supported version", "server", server, "version", semanticVersion, "lowest_supported_version", lowestSupportedVersion) + e.logger.Warn("PostgreSQL version is lower than our lowest supported version", "server", server, "version", semanticVersion, "lowest_supported_version", lowestSupportedVersion) } // Check if semantic version changed and recalculate maps if needed. if semanticVersion.NE(server.lastMapVersion) || server.metricMap == nil { - logger.Info("Semantic version changed", "server", server, "from", server.lastMapVersion, "to", semanticVersion) + e.logger.Info("Semantic version changed", "server", server, "from", server.lastMapVersion, "to", semanticVersion) server.mappingMtx.Lock() // Get Default Metrics only for master database if !e.disableDefaultMetrics && server.master { - server.metricMap = makeDescMap(semanticVersion, server.labels, e.builtinMetricMaps) - server.queryOverrides = makeQueryOverrideMap(semanticVersion, queryOverrides) + server.metricMap = makeDescMap(semanticVersion, server.labels, e.builtinMetricMaps, server.logger, e.metricPrefix) + server.queryOverrides = makeQueryOverrideMap(semanticVersion, queryOverrides, server.logger) } else { server.metricMap = make(map[string]MetricMapNamespace) server.queryOverrides = make(map[string]string) @@ -633,13 +662,13 @@ func (e *Exporter) checkMapVersions(ch chan<- prometheus.Metric, server *Server) // Calculate the hashsum of the useQueries userQueriesData, err := os.ReadFile(e.userQueriesPath) if err != nil { - logger.Error("Failed to reload user queries", "path", e.userQueriesPath, "err", err) + e.logger.Error("Failed to reload user queries", "path", e.userQueriesPath, "err", err) e.userQueriesError.WithLabelValues(e.userQueriesPath, "").Set(1) } else { hashsumStr := fmt.Sprintf("%x", sha256.Sum256(userQueriesData)) - if err := addQueries(userQueriesData, semanticVersion, server); err != nil { - logger.Error("Failed to reload user queries", "path", e.userQueriesPath, "err", err) + if err := addQueries(userQueriesData, semanticVersion, server, e.metricPrefix); err != nil { + e.logger.Error("Failed to reload user queries", "path", e.userQueriesPath, "err", err) e.userQueriesError.WithLabelValues(e.userQueriesPath, hashsumStr).Set(1) } else { // Mark user queries as successfully loaded @@ -681,7 +710,7 @@ func (e *Exporter) scrape(ch chan<- prometheus.Metric) { if err := e.scrapeDSN(ch, dsn); err != nil { errorsCount++ - logger.Error("error scraping dsn", "err", err, "dsn", loggableDSN(dsn)) + e.logger.Error("error scraping dsn", "err", err, "dsn", loggableDSN(dsn)) if _, ok := err.(*ErrorConnectToServer); ok { connectionErrorsCount++ diff --git a/cmd/postgres_exporter/postgres_exporter_integration_test.go b/exporter/postgres_exporter_integration_test.go similarity index 92% rename from cmd/postgres_exporter/postgres_exporter_integration_test.go rename to exporter/postgres_exporter_integration_test.go index 39032f333..a65dbdaab 100644 --- a/cmd/postgres_exporter/postgres_exporter_integration_test.go +++ b/exporter/postgres_exporter_integration_test.go @@ -16,7 +16,7 @@ // working. //go:build integration -package main +package exporter import ( "fmt" @@ -26,6 +26,7 @@ import ( _ "github.com/lib/pq" "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/promslog" . "gopkg.in/check.v1" ) @@ -42,7 +43,7 @@ func (s *IntegrationSuite) SetUpSuite(c *C) { dsn := os.Getenv("DATA_SOURCE_NAME") c.Assert(dsn, Not(Equals), "") - exporter := NewExporter(strings.Split(dsn, ",")) + exporter := NewExporter(strings.Split(dsn, ","), promslog.NewNopLogger()) c.Assert(exporter, NotNil) // Assign the exporter to the suite s.e = exporter @@ -99,12 +100,12 @@ func (s *IntegrationSuite) TestInvalidDsnDoesntCrash(c *C) { }() // Send a bad DSN - exporter := NewExporter([]string{"invalid dsn"}) + exporter := NewExporter([]string{"invalid dsn"}, promslog.NewNopLogger()) c.Assert(exporter, NotNil) exporter.scrape(ch) // Send a DSN to a non-listening port. - exporter = NewExporter([]string{"postgresql://nothing:nothing@127.0.0.1:1/nothing"}) + exporter = NewExporter([]string{"postgresql://nothing:nothing@127.0.0.1:1/nothing"}, promslog.NewNopLogger()) c.Assert(exporter, NotNil) exporter.scrape(ch) } @@ -122,7 +123,7 @@ func (s *IntegrationSuite) TestUnknownMetricParsingDoesntCrash(c *C) { dsn := os.Getenv("DATA_SOURCE_NAME") c.Assert(dsn, Not(Equals), "") - exporter := NewExporter(strings.Split(dsn, ",")) + exporter := NewExporter(strings.Split(dsn, ","), promslog.NewNopLogger()) c.Assert(exporter, NotNil) // Convert the default maps into a list of empty maps. @@ -155,6 +156,7 @@ func (s *IntegrationSuite) TestExtendQueriesDoesntCrash(c *C) { exporter := NewExporter( strings.Split(dsn, ","), + promslog.NewNopLogger(), WithUserQueriesPath("../user_queries_test.yaml"), ) c.Assert(exporter, NotNil) @@ -168,6 +170,7 @@ func (s *IntegrationSuite) TestAutoDiscoverDatabases(c *C) { exporter := NewExporter( strings.Split(dsn, ","), + promslog.NewNopLogger(), ) c.Assert(exporter, NotNil) diff --git a/cmd/postgres_exporter/postgres_exporter_test.go b/exporter/postgres_exporter_test.go similarity index 95% rename from cmd/postgres_exporter/postgres_exporter_test.go rename to exporter/postgres_exporter_test.go index 5b2879d94..6364fc4f9 100644 --- a/cmd/postgres_exporter/postgres_exporter_test.go +++ b/exporter/postgres_exporter_test.go @@ -13,7 +13,7 @@ //go:build !integration -package main +package exporter import ( "math" @@ -24,6 +24,7 @@ import ( "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/promslog" . "gopkg.in/check.v1" ) @@ -36,7 +37,6 @@ type FunctionalSuite struct { var _ = Suite(&FunctionalSuite{}) func (s *FunctionalSuite) SetUpSuite(c *C) { - } func (s *FunctionalSuite) TestSemanticVersionColumnDiscard(c *C) { @@ -53,7 +53,7 @@ func (s *FunctionalSuite) TestSemanticVersionColumnDiscard(c *C) { { // No metrics should be eliminated - resultMap := makeDescMap(semver.MustParse("0.0.1"), prometheus.Labels{}, testMetricMap) + resultMap := makeDescMap(semver.MustParse("0.0.1"), prometheus.Labels{}, testMetricMap, promslog.NewNopLogger(), "pg") c.Check( resultMap["test_namespace"].columnMappings["metric_which_stays"].discard, Equals, @@ -74,7 +74,7 @@ func (s *FunctionalSuite) TestSemanticVersionColumnDiscard(c *C) { testMetricMap["test_namespace"].columnMappings["metric_which_discards"] = discardableMetric // Discard metric should be discarded - resultMap := makeDescMap(semver.MustParse("0.0.1"), prometheus.Labels{}, testMetricMap) + resultMap := makeDescMap(semver.MustParse("0.0.1"), prometheus.Labels{}, testMetricMap, promslog.NewNopLogger(), "pg") c.Check( resultMap["test_namespace"].columnMappings["metric_which_stays"].discard, Equals, @@ -95,7 +95,7 @@ func (s *FunctionalSuite) TestSemanticVersionColumnDiscard(c *C) { testMetricMap["test_namespace"].columnMappings["metric_which_discards"] = discardableMetric // Discard metric should be discarded - resultMap := makeDescMap(semver.MustParse("0.0.2"), prometheus.Labels{}, testMetricMap) + resultMap := makeDescMap(semver.MustParse("0.0.2"), prometheus.Labels{}, testMetricMap, promslog.NewNopLogger(), "pg") c.Check( resultMap["test_namespace"].columnMappings["metric_which_stays"].discard, Equals, @@ -125,7 +125,7 @@ func (s *FunctionalSuite) TestEnvironmentSettingWithSecretsFiles(c *C) { var expected = "postgresql://custom_username$&+,%2F%3A;=%3F%40:custom_password$&+,%2F%3A;=%3F%40@localhost:5432/?sslmode=disable" - dsn, err := getDataSources() + dsn, err := GetDataSources() if err != nil { c.Errorf("Unexpected error reading datasources") } @@ -145,7 +145,7 @@ func (s *FunctionalSuite) TestEnvironmentSettingWithDns(c *C) { c.Assert(err, IsNil) defer UnsetEnvironment(c, "DATA_SOURCE_NAME") - dsn, err := getDataSources() + dsn, err := GetDataSources() if err != nil { c.Errorf("Unexpected error reading datasources") } @@ -173,7 +173,7 @@ func (s *FunctionalSuite) TestEnvironmentSettingWithDnsAndSecrets(c *C) { c.Assert(err, IsNil) defer UnsetEnvironment(c, "DATA_SOURCE_PASS") - dsn, err := getDataSources() + dsn, err := GetDataSources() if err != nil { c.Errorf("Unexpected error reading datasources") } @@ -289,7 +289,7 @@ func (s *FunctionalSuite) TestParseConstLabels(c *C) { } for _, cs := range cases { - labels := parseConstLabels(cs.s) + labels := parseConstLabels(cs.s, promslog.NewNopLogger()) if !reflect.DeepEqual(labels, cs.labels) { c.Fatalf("labels not equal (%v -> %v)", labels, cs.labels) } @@ -319,7 +319,6 @@ func (checker *isNaNChecker) Check(params []interface{}, names []string) (result // test boolean metric type gets converted to float func (s *FunctionalSuite) TestBooleanConversionToValueAndString(c *C) { - type TestCase struct { input interface{} expectedString string @@ -388,7 +387,7 @@ func (s *FunctionalSuite) TestBooleanConversionToValueAndString(c *C) { } for _, cs := range cases { - value, ok := dbToFloat64(cs.input) + value, ok := dbToFloat64(cs.input, promslog.NewNopLogger()) if math.IsNaN(cs.expectedValue) { c.Assert(value, IsNaN) } else { @@ -396,7 +395,7 @@ func (s *FunctionalSuite) TestBooleanConversionToValueAndString(c *C) { } c.Assert(ok, Equals, cs.expectedOK) - count, ok := dbToUint64(cs.input) + count, ok := dbToUint64(cs.input, promslog.NewNopLogger()) c.Assert(count, Equals, cs.expectedCount) c.Assert(ok, Equals, cs.expectedOK) @@ -409,7 +408,7 @@ func (s *FunctionalSuite) TestBooleanConversionToValueAndString(c *C) { func (s *FunctionalSuite) TestParseUserQueries(c *C) { userQueriesData, err := os.ReadFile("./tests/user_queries_ok.yaml") if err == nil { - metricMaps, newQueryOverrides, err := parseUserQueries(userQueriesData) + metricMaps, newQueryOverrides, err := parseUserQueries(userQueriesData, promslog.NewNopLogger()) c.Assert(err, Equals, nil) c.Assert(metricMaps, NotNil) c.Assert(newQueryOverrides, NotNil) diff --git a/cmd/postgres_exporter/queries.go b/exporter/queries.go similarity index 92% rename from cmd/postgres_exporter/queries.go rename to exporter/queries.go index c0081fe02..b0f64f657 100644 --- a/cmd/postgres_exporter/queries.go +++ b/exporter/queries.go @@ -11,11 +11,12 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "errors" "fmt" + "log/slog" "github.com/blang/semver/v4" "gopkg.in/yaml.v2" @@ -167,7 +168,7 @@ var queryOverrides = map[string][]OverrideQuery{ // Convert the query override file to the version-specific query override file // for the exporter. -func makeQueryOverrideMap(pgVersion semver.Version, queryOverrides map[string][]OverrideQuery) map[string]string { +func makeQueryOverrideMap(pgVersion semver.Version, queryOverrides map[string][]OverrideQuery, logger *slog.Logger) map[string]string { resultMap := make(map[string]string) for name, overrideDef := range queryOverrides { // Find a matching semver. We make it an error to have overlapping @@ -189,7 +190,7 @@ func makeQueryOverrideMap(pgVersion semver.Version, queryOverrides map[string][] return resultMap } -func parseUserQueries(content []byte) (map[string]intermediateMetricMap, map[string]string, error) { +func parseUserQueries(content []byte, logger *slog.Logger) (map[string]intermediateMetricMap, map[string]string, error) { var userQueries UserQueries err := yaml.Unmarshal(content, &userQueries) @@ -242,21 +243,21 @@ func parseUserQueries(content []byte) (map[string]intermediateMetricMap, map[str // queries. // TODO: test code for all cu. // TODO: the YAML this supports is "non-standard" - we should move away from it. -func addQueries(content []byte, pgVersion semver.Version, server *Server) error { - metricMaps, newQueryOverrides, err := parseUserQueries(content) +func addQueries(content []byte, pgVersion semver.Version, server *Server, metricPrefix string) error { + metricMaps, newQueryOverrides, err := parseUserQueries(content, server.logger) if err != nil { return err } // Convert the loaded metric map into exporter representation - partialExporterMap := makeDescMap(pgVersion, server.labels, metricMaps) + partialExporterMap := makeDescMap(pgVersion, server.labels, metricMaps, server.logger, metricPrefix) // Merge the two maps (which are now quite flatteend) for k, v := range partialExporterMap { _, found := server.metricMap[k] if found { - logger.Debug("Overriding metric from user YAML file", "metric", k) + server.logger.Debug("Overriding metric from user YAML file", "metric", k) } else { - logger.Debug("Adding new metric from user YAML file", "metric", k) + server.logger.Debug("Adding new metric from user YAML file", "metric", k) } server.metricMap[k] = v } @@ -265,9 +266,9 @@ func addQueries(content []byte, pgVersion semver.Version, server *Server) error for k, v := range newQueryOverrides { _, found := server.queryOverrides[k] if found { - logger.Debug("Overriding query override from user YAML file", "query_override", k) + server.logger.Debug("Overriding query override from user YAML file", "query_override", k) } else { - logger.Debug("Adding new query override from user YAML file", "query_override", k) + server.logger.Debug("Adding new query override from user YAML file", "query_override", k) } server.queryOverrides[k] = v } diff --git a/cmd/postgres_exporter/server.go b/exporter/server.go similarity index 89% rename from cmd/postgres_exporter/server.go rename to exporter/server.go index a90183cd7..2aa2c6d67 100644 --- a/cmd/postgres_exporter/server.go +++ b/exporter/server.go @@ -11,16 +11,18 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "database/sql" "fmt" + "log/slog" "sync" "time" "github.com/blang/semver/v4" "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/promslog" ) // Server describes a connection to Postgres. @@ -42,6 +44,7 @@ type Server struct { // Currently cached metrics metricCache map[string]cachedMetrics cacheMtx sync.Mutex + logger *slog.Logger } // ServerOpt configures a server. @@ -56,6 +59,12 @@ func ServerWithLabels(labels prometheus.Labels) ServerOpt { } } +func ServerWithLogger(logger *slog.Logger) ServerOpt { + return func(s *Server) { + s.logger = logger + } +} + // NewServer establishes a new connection using DSN. func NewServer(dsn string, opts ...ServerOpt) (*Server, error) { fingerprint, err := parseFingerprint(dsn) @@ -70,8 +79,6 @@ func NewServer(dsn string, opts ...ServerOpt) (*Server, error) { db.SetMaxOpenConns(1) db.SetMaxIdleConns(1) - logger.Info("Established new database connection", "fingerprint", fingerprint) - s := &Server{ db: db, master: false, @@ -79,12 +86,15 @@ func NewServer(dsn string, opts ...ServerOpt) (*Server, error) { serverLabelName: fingerprint, }, metricCache: make(map[string]cachedMetrics), + logger: promslog.NewNopLogger(), } for _, opt := range opts { opt(s) } + s.logger.Info("Established new database connection", "fingerprint", fingerprint) + return s, nil } @@ -97,7 +107,7 @@ func (s *Server) Close() error { func (s *Server) Ping() error { if err := s.db.Ping(); err != nil { if cerr := s.Close(); cerr != nil { - logger.Error("Error while closing non-pinging DB connection", "server", s, "err", cerr) + s.logger.Error("Error while closing non-pinging DB connection", "server", s, "err", cerr) } return err } @@ -189,7 +199,7 @@ func (s *Servers) Close() { defer s.m.Unlock() for _, server := range s.servers { if err := server.Close(); err != nil { - logger.Error("Failed to close connection", "server", server, "err", err) + server.logger.Error("Failed to close connection", "server", server, "err", err) } } } diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/Dockerfile b/exporter/tests/docker-postgres-replication/Dockerfile similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/Dockerfile rename to exporter/tests/docker-postgres-replication/Dockerfile diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/Dockerfile.p2 b/exporter/tests/docker-postgres-replication/Dockerfile.p2 similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/Dockerfile.p2 rename to exporter/tests/docker-postgres-replication/Dockerfile.p2 diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/README.md b/exporter/tests/docker-postgres-replication/README.md similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/README.md rename to exporter/tests/docker-postgres-replication/README.md diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/docker-compose.yml b/exporter/tests/docker-postgres-replication/docker-compose.yml similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/docker-compose.yml rename to exporter/tests/docker-postgres-replication/docker-compose.yml diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/docker-entrypoint.sh b/exporter/tests/docker-postgres-replication/docker-entrypoint.sh similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/docker-entrypoint.sh rename to exporter/tests/docker-postgres-replication/docker-entrypoint.sh diff --git a/cmd/postgres_exporter/tests/docker-postgres-replication/setup-replication.sh b/exporter/tests/docker-postgres-replication/setup-replication.sh similarity index 100% rename from cmd/postgres_exporter/tests/docker-postgres-replication/setup-replication.sh rename to exporter/tests/docker-postgres-replication/setup-replication.sh diff --git a/cmd/postgres_exporter/tests/test-smoke b/exporter/tests/test-smoke similarity index 100% rename from cmd/postgres_exporter/tests/test-smoke rename to exporter/tests/test-smoke diff --git a/cmd/postgres_exporter/tests/user_queries_ok.yaml b/exporter/tests/user_queries_ok.yaml similarity index 100% rename from cmd/postgres_exporter/tests/user_queries_ok.yaml rename to exporter/tests/user_queries_ok.yaml diff --git a/cmd/postgres_exporter/tests/user_queries_test.yaml b/exporter/tests/user_queries_test.yaml similarity index 100% rename from cmd/postgres_exporter/tests/user_queries_test.yaml rename to exporter/tests/user_queries_test.yaml diff --git a/cmd/postgres_exporter/tests/username_file b/exporter/tests/username_file similarity index 100% rename from cmd/postgres_exporter/tests/username_file rename to exporter/tests/username_file diff --git a/cmd/postgres_exporter/tests/userpass_file b/exporter/tests/userpass_file similarity index 100% rename from cmd/postgres_exporter/tests/userpass_file rename to exporter/tests/userpass_file diff --git a/cmd/postgres_exporter/util.go b/exporter/util.go similarity index 96% rename from cmd/postgres_exporter/util.go rename to exporter/util.go index fa692c652..895c2d07e 100644 --- a/cmd/postgres_exporter/util.go +++ b/exporter/util.go @@ -11,10 +11,11 @@ // See the License for the specific language governing permissions and // limitations under the License. -package main +package exporter import ( "fmt" + "log/slog" "math" "net/url" "strconv" @@ -59,7 +60,7 @@ func stringToColumnUsage(s string) (ColumnUsage, error) { // Convert database.sql types to float64s for Prometheus consumption. Null types are mapped to NaN. string and []byte // types are mapped as NaN and !ok -func dbToFloat64(t interface{}) (float64, bool) { +func dbToFloat64(t interface{}, logger *slog.Logger) (float64, bool) { switch v := t.(type) { case int64: return float64(v), true @@ -97,7 +98,7 @@ func dbToFloat64(t interface{}) (float64, bool) { // Convert database.sql types to uint64 for Prometheus consumption. Null types are mapped to 0. string and []byte // types are mapped as 0 and !ok -func dbToUint64(t interface{}) (uint64, bool) { +func dbToUint64(t interface{}, logger *slog.Logger) (uint64, bool) { switch v := t.(type) { case uint64: return v, true From e27c227e471ae45c93af32297a870cc57395ed27 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 30 Jan 2026 12:35:37 +0100 Subject: [PATCH 079/113] Update common Prometheus files (#1251) Signed-off-by: prombot --- .yamllint | 1 + 1 file changed, 1 insertion(+) diff --git a/.yamllint b/.yamllint index 8d09c375f..b329f464f 100644 --- a/.yamllint +++ b/.yamllint @@ -2,6 +2,7 @@ extends: default ignore: | **/node_modules + web/api/v1/testdata/openapi_*_golden.yaml rules: braces: From 9cb964898624da7206fa11165c2e4452c42d9a63 Mon Sep 17 00:00:00 2001 From: Joe Adams Date: Mon, 2 Feb 2026 11:34:03 -0500 Subject: [PATCH 080/113] fix: Do not crash on bad pg_settings value (#1252) * fix: Do not crash on bad pg_settings value pg_settings metrics would panic when the parsing fails. This is a poor experience for users. An exporter should not panic. This skips unparsable metrics and instead logs a warning. Fixes: #1240 --------- Signed-off-by: Joe Adams --- exporter/pg_setting.go | 22 +++++++++++++--------- exporter/pg_setting_test.go | 16 +++++++--------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/exporter/pg_setting.go b/exporter/pg_setting.go index b7adfa25f..8a49021d3 100644 --- a/exporter/pg_setting.go +++ b/exporter/pg_setting.go @@ -51,8 +51,15 @@ func querySettings(ch chan<- prometheus.Metric, server *Server) error { if err != nil { return fmt.Errorf("Error retrieving rows on %q: %s %v", server, namespace, err) } - - ch <- s.metric(server.labels) + metric, err := s.metric(server.labels) + if err != nil { + // Log the error and continue + // This could be due to a bad value in the setting + // but we should not fail the entire scrape or panic + server.logger.Warn("Error normalising unit for setting", "setting", s.name, "value", s.setting, "unit", s.unit, "error", err) + continue + } + ch <- metric } return nil @@ -64,7 +71,7 @@ type pgSetting struct { name, setting, unit, shortDesc, vartype string } -func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { +func (s *pgSetting) metric(labels prometheus.Labels) (prometheus.Metric, error) { var ( err error name = strings.ReplaceAll(strings.ReplaceAll(s.name, ".", "_"), "-", "_") @@ -81,9 +88,7 @@ func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { } case "integer", "real": if val, unit, err = s.normaliseUnit(); err != nil { - // Panic, since we should recognise all units - // and don't want to silently exlude metrics - panic(err) + return nil, err } if len(unit) > 0 { @@ -91,12 +96,11 @@ func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { shortDesc = fmt.Sprintf("%s [Units converted to %s.]", shortDesc, unit) } default: - // Panic because we got a type we didn't ask for - panic(fmt.Sprintf("Unsupported vartype %q", s.vartype)) + return nil, fmt.Errorf("pgsetting: unsupported vartype %q", s.vartype) } desc := newDesc(subsystem, name, shortDesc, labels) - return prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, val) + return prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, val), nil } // Removes units from any of the setting values. diff --git a/exporter/pg_setting_test.go b/exporter/pg_setting_test.go index 7038959fd..20fa65c07 100644 --- a/exporter/pg_setting_test.go +++ b/exporter/pg_setting_test.go @@ -237,17 +237,15 @@ func (s *PgSettingSuite) TestNormaliseUnit(c *C) { } func (s *PgSettingSuite) TestMetric(c *C) { - defer func() { - if r := recover(); r != nil { - if r.(error).Error() != `unknown unit for runtime variable: "nonexistent"` { - panic(r) - } - } - }() - for _, f := range fixtures { + if f.n.err != "" { + continue + } d := &dto.Metric{} - m := f.p.metric(prometheus.Labels{}) + m, err := f.p.metric(prometheus.Labels{}) + if err != nil { + c.Fatalf("Error creating metric: %v", err) + } m.Write(d) // nolint: errcheck c.Check(m.Desc().String(), Equals, f.d) From 7bf8aeb5cba51d5d812421877fca6874b39ecef5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 4 Feb 2026 16:19:37 +0100 Subject: [PATCH 081/113] Bump github.com/prometheus/common from 0.67.4 to 0.67.5 (#1255) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.67.4 to 0.67.5. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/common/compare/v0.67.4...v0.67.5) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.67.5 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 16 ++++++++-------- go.sum | 32 ++++++++++++++++---------------- 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/go.mod b/go.mod index 6ec5291d5..1a0c83ea8 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/lib/pq v1.10.9 github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.67.4 + github.com/prometheus/common v0.67.5 github.com/prometheus/exporter-toolkit v0.15.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c @@ -38,12 +38,12 @@ require ( github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect go.yaml.in/yaml/v2 v2.4.3 // indirect - golang.org/x/crypto v0.45.0 // indirect - golang.org/x/net v0.47.0 // indirect - golang.org/x/oauth2 v0.32.0 // indirect - golang.org/x/sync v0.18.0 // indirect - golang.org/x/sys v0.38.0 // indirect - golang.org/x/text v0.31.0 // indirect + golang.org/x/crypto v0.46.0 // indirect + golang.org/x/net v0.48.0 // indirect + golang.org/x/oauth2 v0.34.0 // indirect + golang.org/x/sync v0.19.0 // indirect + golang.org/x/sys v0.39.0 // indirect + golang.org/x/text v0.32.0 // indirect golang.org/x/time v0.13.0 // indirect - google.golang.org/protobuf v1.36.10 // indirect + google.golang.org/protobuf v1.36.11 // indirect ) diff --git a/go.sum b/go.sum index acf1db306..ffb6162fd 100644 --- a/go.sum +++ b/go.sum @@ -57,8 +57,8 @@ github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc= -github.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= +github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4= +github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw= github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= github.com/prometheus/exporter-toolkit v0.15.0/go.mod h1:OyRWd2iTo6Xge9Kedvv0IhCrJSBu36JCfJ2yVniRIYk= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= @@ -86,22 +86,22 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= -golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= -golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= -golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= -golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= -golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= -golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= -golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= -golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= -golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= -golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= -golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= +golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU= +golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0= +golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU= +golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY= +golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= +golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= +golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= +golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= +golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY= golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= -google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= -google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= +google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= From d762363b9029c28c0eab103ec80ac0bcedba26ef Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 4 Feb 2026 16:29:38 +0100 Subject: [PATCH 082/113] Bump github.com/prometheus/exporter-toolkit from 0.15.0 to 0.15.1 (#1254) Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.15.0 to 0.15.1. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.15.0...v0.15.1) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-version: 0.15.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 4 ++-- go.sum | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/go.mod b/go.mod index 1a0c83ea8..c68d6d359 100644 --- a/go.mod +++ b/go.mod @@ -10,7 +10,7 @@ require ( github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.67.5 - github.com/prometheus/exporter-toolkit v0.15.0 + github.com/prometheus/exporter-toolkit v0.15.1 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -44,6 +44,6 @@ require ( golang.org/x/sync v0.19.0 // indirect golang.org/x/sys v0.39.0 // indirect golang.org/x/text v0.32.0 // indirect - golang.org/x/time v0.13.0 // indirect + golang.org/x/time v0.14.0 // indirect google.golang.org/protobuf v1.36.11 // indirect ) diff --git a/go.sum b/go.sum index ffb6162fd..a40e54c06 100644 --- a/go.sum +++ b/go.sum @@ -59,8 +59,8 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4= github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw= -github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= -github.com/prometheus/exporter-toolkit v0.15.0/go.mod h1:OyRWd2iTo6Xge9Kedvv0IhCrJSBu36JCfJ2yVniRIYk= +github.com/prometheus/exporter-toolkit v0.15.1 h1:XrGGr/qWl8Gd+pqJqTkNLww9eG8vR/CoRk0FubOKfLE= +github.com/prometheus/exporter-toolkit v0.15.1/go.mod h1:P/NR9qFRGbCFgpklyhix9F6v6fFr/VQB/CVsrMDGKo4= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -98,8 +98,8 @@ golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY= -golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= -golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= +golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= From a433d58515763835dd4b3bf9c4f3a005c2a758b4 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 4 Feb 2026 18:44:53 +0100 Subject: [PATCH 083/113] Prepare release 0.19.0 (#1256) Signed-off-by: Cristian Greco --- CHANGELOG.md | 11 +++++++++++ VERSION | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cd2f553f..aa42e5000 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,17 @@ * [ENHANCEMENT] ... * [BUGFIX] ... +## 0.19.0 / 2026-02-03 + +* [CHANGE] Reorganize code to remove the main package by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1238 +* [FEATURE] Allow setting `limit` in `pg_stat_statements` collector by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1205 +* [FEATURE] Allow excluding databases or users names in `stat_statements` collector by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1232 +* [ENHANCEMENT] Enable pprof endpoints for profiling by @SuperQ in https://github.com/prometheus-community/postgres_exporter/pull/1212 +* [ENHANCEMENT] Document `stat_checkpointer` collector option by @fabiorueda in https://github.com/prometheus-community/postgres_exporter/pull/1226 +* [ENHANCEMENT] Ensure collection of stats succeed in a maximum duration to avoid exhausting PG connections by @pierresouchay in https://github.com/prometheus-community/postgres_exporter/pull/1229 +* [BUGFIX] Fix NULL handling on `long_running_transactions` collector by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1230 +* [BUGFIX] Do not crash on bad `pg_settings` value by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1252 + ## 0.18.1 / 2025-09-29 * [BUGFIX] Fix swapped `flushedLsn` and `receiveStartTli` for `wal_receiver` collector by @spantaleev in https://github.com/prometheus-community/postgres_exporter/pull/1198 diff --git a/VERSION b/VERSION index 249afd517..1cf0537c3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.18.1 +0.19.0 From a00749f9eae4772c09d3284002be6831a35c6cfb Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Mon, 16 Feb 2026 18:59:36 +0100 Subject: [PATCH 084/113] Update common Prometheus files (#1260) Signed-off-by: prombot --- .github/workflows/golangci-lint.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index ae5fdc80e..16467b897 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -28,7 +28,7 @@ jobs: with: persist-credentials: false - name: Install Go - uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0 + uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0 with: go-version: 1.25.x - name: Install snmp_exporter/generator dependencies From d14e06ca63778a7c190e99d57b285634bfcf71db Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 18 Feb 2026 15:57:15 +0100 Subject: [PATCH 085/113] fix: ignore setting `google_dataplex.max_messages` (#1261) * fix: ignore setting `google_dataplex.max_messages` Followup of https://github.com/prometheus-community/postgres_exporter/pull/1252 Ignore this setting altogether, as logging a warn at each scrape for a cloud provider specific entry seems a waste of resources. Signed-off-by: Cristian Greco * add comment Signed-off-by: Cristian Greco --------- Signed-off-by: Cristian Greco --- exporter/pg_setting.go | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/exporter/pg_setting.go b/exporter/pg_setting.go index 8a49021d3..770536c4b 100644 --- a/exporter/pg_setting.go +++ b/exporter/pg_setting.go @@ -37,7 +37,11 @@ func querySettings(ch chan<- prometheus.Metric, server *Server) error { // // NOTE: If you add more vartypes here, you must update the supported // types in normaliseUnit() below - query := "SELECT name, setting, COALESCE(unit, ''), short_desc, vartype FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real') AND name != 'sync_commit_cancel_wait';" + // + // Settings intentionally ignored due to invalid format: + // - `sync_commit_cancel_wait`, specific to Azure Postgres, see https://github.com/prometheus-community/postgres_exporter/issues/523 + // - `google_dataplex.max_messages`, specific to Google Cloud SQL, see https://github.com/prometheus-community/postgres_exporter/issues/1240 + query := "SELECT name, setting, COALESCE(unit, ''), short_desc, vartype FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real') AND name NOT IN ('sync_commit_cancel_wait', 'google_dataplex.max_messages');" rows, err := server.db.Query(query) if err != nil { From ee441c1fdbd79c81436082a5d80e02509ae526cc Mon Sep 17 00:00:00 2001 From: dannotripp Date: Fri, 20 Feb 2026 06:17:40 -0800 Subject: [PATCH 086/113] wal: Fix collector NULL SUM(size) (#1265) * Fix wal collector NULL SUM(size) * Use sql.NullInt64 for WAL size, skip metric when NULL Ref: prometheus-community/postgres_exporter#1264 --------- Signed-off-by: Danno Tripp --- collector/pg_wal.go | 13 ++++++++----- collector/pg_wal_test.go | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 5 deletions(-) diff --git a/collector/pg_wal.go b/collector/pg_wal.go index afa8fcef6..b76b7118b 100644 --- a/collector/pg_wal.go +++ b/collector/pg_wal.go @@ -15,6 +15,7 @@ package collector import ( "context" + "database/sql" "github.com/prometheus/client_golang/prometheus" ) @@ -67,7 +68,7 @@ func (c PGWALCollector) Update(ctx context.Context, instance *instance, ch chan< ) var segments uint64 - var size uint64 + var size sql.NullInt64 err := row.Scan(&segments, &size) if err != nil { return err @@ -76,9 +77,11 @@ func (c PGWALCollector) Update(ctx context.Context, instance *instance, ch chan< pgWALSegments, prometheus.GaugeValue, float64(segments), ) - ch <- prometheus.MustNewConstMetric( - pgWALSize, - prometheus.GaugeValue, float64(size), - ) + if size.Valid { + ch <- prometheus.MustNewConstMetric( + pgWALSize, + prometheus.GaugeValue, float64(size.Int64), + ) + } return nil } diff --git a/collector/pg_wal_test.go b/collector/pg_wal_test.go index 745105a13..8109953ef 100644 --- a/collector/pg_wal_test.go +++ b/collector/pg_wal_test.go @@ -61,3 +61,42 @@ func TestPgWALCollector(t *testing.T) { t.Errorf("there were unfulfilled exceptions: %s", err) } } + +func TestPgWALCollectorZeroSegments(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"segments", "size"} + rows := sqlmock.NewRows(columns). + AddRow(0, nil) + mock.ExpectQuery(sanitizeQuery(pgWALQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGWALCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGWALCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, value: 0, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison (zero segments)", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} From 6afd07898e61aa6fa7efc2e578d91decfbe5a634 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20F=2E=20Sch=C3=B6nitzer?= Date: Tue, 24 Feb 2026 13:06:48 +0100 Subject: [PATCH 087/113] Filter and warn about duplicates in pg_stat_statements (#998) (#1259) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Filter and warn about duplicates in pg_stat_statements Signed-off-by: Michael Schönitzer Signed-off-by: Michael F. Schönitzer * Structured log message Co-authored-by: Cristian Greco Signed-off-by: Michael F. Schönitzer * Use struct instead of string for map of seen objects Signed-off-by: Michael F. Schönitzer * Switch to a map(string)struct{} data structure Signed-off-by: Michael F. Schönitzer --------- Signed-off-by: Michael Schönitzer Signed-off-by: Michael F. Schönitzer Co-authored-by: Cristian Greco --- collector/pg_stat_statements.go | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index 9227f5c04..2bfac2df4 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -251,6 +251,8 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc presentQueryIds := make(map[string]struct{}) + seen := make(map[string]struct{}) // to track duplicates by (user, datname, queryid) + if err != nil { return err } @@ -282,6 +284,14 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc queryidLabel = queryid.String } + key := fmt.Sprintf("%s|%s|%s", userLabel, datnameLabel, queryidLabel) + _, ok := seen[key] + if ok { + c.log.Warn("Duplicate found", "user", userLabel, "datname", datnameLabel, "queryid", queryidLabel) + continue + } + seen[key] = struct{}{} + callsTotalMetric := 0.0 if callsTotal.Valid { callsTotalMetric = float64(callsTotal.Int64) From 4a776beb674c8488ade1c0bf91810b7a4b17db6f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Emin=20Umut=20Ger=C3=A7ek?= Date: Wed, 25 Feb 2026 10:57:38 +0300 Subject: [PATCH 088/113] Optimize pg_stat_statements queries (#1267) Signed-off-by: Emin Umut Gercek --- collector/pg_stat_statements.go | 28 ++++++++++++++-------------- collector/pg_stat_statements_test.go | 24 ++++++++++++------------ 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go index 2bfac2df4..5c5c036d1 100644 --- a/collector/pg_stat_statements.go +++ b/collector/pg_stat_statements.go @@ -187,14 +187,14 @@ const ( pg_stat_statements.rows as rows_total, pg_stat_statements.blk_read_time / 1000.0 as block_read_seconds_total, pg_stat_statements.blk_write_time / 1000.0 as block_write_seconds_total - FROM pg_stat_statements + FROM pg_stat_statements(%t) JOIN pg_database ON pg_database.oid = pg_stat_statements.dbid WHERE total_exec_time > ( SELECT percentile_cont(0.1) WITHIN GROUP (ORDER BY total_exec_time) - FROM pg_stat_statements + FROM pg_stat_statements(false) ) %s %s ORDER BY seconds_total DESC @@ -210,14 +210,14 @@ const ( pg_stat_statements.rows as rows_total, pg_stat_statements.shared_blk_read_time / 1000.0 as block_read_seconds_total, pg_stat_statements.shared_blk_write_time / 1000.0 as block_write_seconds_total - FROM pg_stat_statements + FROM pg_stat_statements(%t) JOIN pg_database ON pg_database.oid = pg_stat_statements.dbid WHERE total_exec_time > ( SELECT percentile_cont(0.1) WITHIN GROUP (ORDER BY total_exec_time) - FROM pg_stat_statements + FROM pg_stat_statements(false) ) %s %s ORDER BY seconds_total DESC @@ -225,15 +225,6 @@ const ( ) func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { - var queryTemplate string - switch { - case instance.version.GE(semver.MustParse("17.0.0")): - queryTemplate = pgStatStatementsQuery_PG17 - case instance.version.GE(semver.MustParse("13.0.0")): - queryTemplate = pgStatStatementsQuery_PG13 - default: - queryTemplate = pgStatStatementsQuery - } querySelect := "" if c.includeQueryStatement { querySelect = fmt.Sprintf(pgStatStatementQuerySelect, c.statementLength) @@ -244,7 +235,16 @@ func (c PGStatStatementsCollector) Update(ctx context.Context, instance *instanc if c.statementLimit > 0 { statementLimit = fmt.Sprintf("%d", c.statementLimit) } - query := fmt.Sprintf(queryTemplate, querySelect, databaseFilter, userFilter, statementLimit) + + var query string + switch { + case instance.version.GE(semver.MustParse("17.0.0")): + query = fmt.Sprintf(pgStatStatementsQuery_PG17, querySelect, c.includeQueryStatement, databaseFilter, userFilter, statementLimit) + case instance.version.GE(semver.MustParse("13.0.0")): + query = fmt.Sprintf(pgStatStatementsQuery_PG13, querySelect, c.includeQueryStatement, databaseFilter, userFilter, statementLimit) + default: + query = fmt.Sprintf(pgStatStatementsQuery, querySelect, databaseFilter, userFilter, statementLimit) + } db := instance.getDB() rows, err := db.QueryContext(ctx, query) diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go index aea08b244..00171c83c 100644 --- a/collector/pg_stat_statements_test.go +++ b/collector/pg_stat_statements_test.go @@ -167,7 +167,7 @@ func TestPGStatStatementsCollectorNull(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", false, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -210,7 +210,7 @@ func TestPGStatStatementsCollectorNullWithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), true, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -254,7 +254,7 @@ func TestPGStatStatementsCollectorNullWithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 200) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), "", "", "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 200), true, "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -298,7 +298,7 @@ func TestPGStatStatementsCollector_PG13(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", false, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -341,7 +341,7 @@ func TestPGStatStatementsCollector_PG13_WithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), true, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -385,7 +385,7 @@ func TestPGStatStatementsCollector_PG13_WithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, fmt.Sprintf(pgStatStatementQuerySelect, 300), true, "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -429,7 +429,7 @@ func TestPGStatStatementsCollector_PG17(t *testing.T) { columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", false, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -472,7 +472,7 @@ func TestPGStatStatementsCollector_PG17_WithStatement(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), true, "", "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -516,7 +516,7 @@ func TestPGStatStatementsCollector_PG17_WithStatementAndLimit(t *testing.T) { columns := []string{"user", "datname", "queryid", "LEFT(pg_stat_statements.query, 300) as query", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} rows := sqlmock.NewRows(columns). AddRow("postgres", "postgres", 1500, "select 1 from foo", 5, 0.4, 100, 0.1, 0.2) - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), "", "", "10"))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, fmt.Sprintf(pgStatStatementQuerySelect, 300), true, "", "", "10"))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -613,7 +613,7 @@ func TestPGStatStatementsCollectorWithExcludedDatabasesSQLEscaping(t *testing.T) AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) expectedFilter := " AND pg_database.datname NOT IN ('test''db')" - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", expectedFilter, "", defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", false, expectedFilter, "", defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -708,7 +708,7 @@ func TestPGStatStatementsCollectorWithExcludedUsersSQLEscaping(t *testing.T) { AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) expectedFilter := " AND pg_get_userbyid(userid) NOT IN ('test''user')" - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", "", expectedFilter, defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG13, "", false, "", expectedFilter, defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { @@ -758,7 +758,7 @@ func TestPGStatStatementsCollectorWithExcludedDatabasesAndUsers(t *testing.T) { expectedDbFilter := " AND pg_database.datname NOT IN ('rdsadmin', 'cloudsqladmin')" expectedUserFilter := " AND pg_get_userbyid(userid) NOT IN ('monitoring', 'readonly')" - mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", expectedDbFilter, expectedUserFilter, defaultStatementLimit))).WillReturnRows(rows) + mock.ExpectQuery(sanitizeQuery(fmt.Sprintf(pgStatStatementsQuery_PG17, "", false, expectedDbFilter, expectedUserFilter, defaultStatementLimit))).WillReturnRows(rows) ch := make(chan prometheus.Metric) go func() { From e9aa46ccf6ff650aca32c0f4438b7752cc215b01 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Wed, 25 Feb 2026 09:01:38 +0100 Subject: [PATCH 089/113] Update common Prometheus files (#1266) Signed-off-by: prombot Co-authored-by: Cristian Greco --- Makefile.common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index b8c9b3844..18f20f79a 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.7.2 +GOLANGCI_LINT_VERSION ?= v2.10.1 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. From 333363aeb548b66340a61a7fb2b3f0e8a8d39f90 Mon Sep 17 00:00:00 2001 From: Cristian Greco Date: Wed, 25 Feb 2026 14:05:43 +0100 Subject: [PATCH 090/113] Prepare release v0.19.1 (#1268) Signed-off-by: Cristian Greco --- CHANGELOG.md | 7 +++++++ VERSION | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa42e5000..8cc3b9c47 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,13 @@ * [ENHANCEMENT] ... * [BUGFIX] ... +## 0.19.1 / 2026-02-25 + +* [CHANGE] Filter and warn about duplicates in pg_stat_statements (#998) by @Nudin in https://github.com/prometheus-community/postgres_exporter/pull/1259 +* [ENHANCEMENT] perf/reliability: Optimize pg_stat_statements queries that uses too many temp files by @eugercek in https://github.com/prometheus-community/postgres_exporter/pull/1267 +* [BUGFIX] fix: ignore setting `google_dataplex.max_messages` by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1261 +* [BUGFIX] wal: Fix collector NULL SUM(size) by @dannotripp in https://github.com/prometheus-community/postgres_exporter/pull/1265 + ## 0.19.0 / 2026-02-03 * [CHANGE] Reorganize code to remove the main package by @cristiangreco in https://github.com/prometheus-community/postgres_exporter/pull/1238 diff --git a/VERSION b/VERSION index 1cf0537c3..41915c799 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.19.0 +0.19.1 From 4f6941ffe9cb147d34547724314c24487b37b320 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 3 Mar 2026 10:34:13 +0100 Subject: [PATCH 091/113] Update common Prometheus files (#1269) Signed-off-by: prombot --- .github/workflows/container_description.yml | 4 ++-- .github/workflows/golangci-lint.yml | 6 ++++-- Makefile.common | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/.github/workflows/container_description.yml b/.github/workflows/container_description.yml index 7b46e9532..d7b879f9c 100644 --- a/.github/workflows/container_description.yml +++ b/.github/workflows/container_description.yml @@ -18,7 +18,7 @@ jobs: if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks. steps: - name: git checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: persist-credentials: false - name: Set docker hub repo name @@ -42,7 +42,7 @@ jobs: if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks. steps: - name: git checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: persist-credentials: false - name: Set quay.io org name diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 16467b897..dc8ffd02d 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -3,6 +3,7 @@ name: golangci-lint on: push: + branches: [main, master, 'release-*'] paths: - "go.sum" - "go.mod" @@ -10,6 +11,7 @@ on: - "scripts/errcheck_excludes.txt" - ".github/workflows/golangci-lint.yml" - ".golangci.yml" + tags: ['v*'] pull_request: permissions: # added using https://github.com/step-security/secure-repo @@ -24,13 +26,13 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout repository - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: persist-credentials: false - name: Install Go uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0 with: - go-version: 1.25.x + go-version: 1.26.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' diff --git a/Makefile.common b/Makefile.common index 18f20f79a..d19d390d3 100644 --- a/Makefile.common +++ b/Makefile.common @@ -55,7 +55,7 @@ ifneq ($(shell command -v gotestsum 2> /dev/null),) endif endif -PROMU_VERSION ?= 0.17.0 +PROMU_VERSION ?= 0.18.0 PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_VERSION)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM).tar.gz SKIP_GOLANGCI_LINT := From f0117d7562b5a84002d00f2cca9fe6580db9fc0e Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Wed, 4 Mar 2026 03:10:27 +0100 Subject: [PATCH 092/113] Update common Prometheus files (#1272) Signed-off-by: prombot --- Makefile.common | 110 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 106 insertions(+), 4 deletions(-) diff --git a/Makefile.common b/Makefile.common index d19d390d3..cce3ef1d1 100644 --- a/Makefile.common +++ b/Makefile.common @@ -109,6 +109,24 @@ endif # Build variant:dockerfile pairs for shell iteration. DOCKERFILE_VARIANTS_WITH_NAMES := $(foreach df,$(DOCKERFILE_VARIANTS),$(call dockerfile_variant,$(df)):$(df)) +# Shell helper to check whether a dockerfile/arch pair is excluded. +define dockerfile_arch_is_excluded +case " $(DOCKERFILE_ARCH_EXCLUSIONS) " in \ + *" $$dockerfile:$(1) "*) true ;; \ + *) false ;; \ +esac +endef + +# Shell helper to check whether a registry/arch pair is excluded. +# Extracts registry from DOCKER_REPO (e.g., quay.io/prometheus -> quay.io) +define registry_arch_is_excluded +registry=$$(echo "$(DOCKER_REPO)" | cut -d'/' -f1); \ +case " $(DOCKER_REGISTRY_ARCH_EXCLUSIONS) " in \ + *" $$registry:$(1) "*) true ;; \ + *) false ;; \ +esac +endef + BUILD_DOCKER_ARCHS = $(addprefix common-docker-,$(DOCKER_ARCHS)) PUBLISH_DOCKER_ARCHS = $(addprefix common-docker-publish-,$(DOCKER_ARCHS)) TAG_DOCKER_ARCHS = $(addprefix common-docker-tag-latest-,$(DOCKER_ARCHS)) @@ -250,6 +268,10 @@ $(BUILD_DOCKER_ARCHS): common-docker-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ + if $(call dockerfile_arch_is_excluded,$*); then \ + echo "Skipping $$variant_name variant for linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ distroless_arch="$*"; \ if [ "$*" = "armv7" ]; then \ distroless_arch="arm"; \ @@ -284,6 +306,14 @@ $(PUBLISH_DOCKER_ARCHS): common-docker-publish-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ + if $(call dockerfile_arch_is_excluded,$*); then \ + echo "Skipping push for $$variant_name variant on linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$*); then \ + echo "Skipping push for $$variant_name variant on linux-$* to $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Pushing $$variant_name variant for linux-$*"; \ docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ @@ -311,6 +341,14 @@ $(TAG_DOCKER_ARCHS): common-docker-tag-latest-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ + if $(call dockerfile_arch_is_excluded,$*); then \ + echo "Skipping tag for $$variant_name variant on linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$*); then \ + echo "Skipping tag for $$variant_name variant on linux-$* for $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Tagging $$variant_name variant for linux-$* as latest"; \ docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest-$$variant_name"; \ @@ -330,23 +368,87 @@ common-docker-manifest: variant_name=$${variant%%:*}; \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Creating manifest for $$variant_name variant"; \ - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name); \ + refs=""; \ + for arch in $(DOCKER_ARCHS); do \ + if $(call dockerfile_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for $$variant_name (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for $$variant_name on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ + done; \ + if [ -z "$$refs" ]; then \ + echo "Skipping manifest for $$variant_name variant (no supported architectures)"; \ + continue; \ + fi; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" $$refs; \ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ fi; \ if [ "$$dockerfile" = "Dockerfile" ]; then \ echo "Creating default variant ($$variant_name) manifest"; \ - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)); \ + refs=""; \ + for arch in $(DOCKER_ARCHS); do \ + if $(call dockerfile_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for default variant (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for default variant on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:$(SANITIZED_DOCKER_IMAGE_TAG)"; \ + done; \ + if [ -z "$$refs" ]; then \ + echo "Skipping default variant manifest (no supported architectures)"; \ + continue; \ + fi; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $$refs; \ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)"; \ fi; \ if [ "$(DOCKER_IMAGE_TAG)" = "latest" ]; then \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Creating manifest for $$variant_name variant version tag"; \ - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name); \ + refs=""; \ + for arch in $(DOCKER_ARCHS); do \ + if $(call dockerfile_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for $$variant_name version tag (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for $$variant_name version tag on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ + done; \ + if [ -z "$$refs" ]; then \ + echo "Skipping version-tag manifest for $$variant_name variant (no supported architectures)"; \ + continue; \ + fi; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name" $$refs; \ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ fi; \ if [ "$$dockerfile" = "Dockerfile" ]; then \ echo "Creating default variant version tag manifest"; \ - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):v$(DOCKER_MAJOR_VERSION_TAG)); \ + refs=""; \ + for arch in $(DOCKER_ARCHS); do \ + if $(call dockerfile_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for default variant version tag (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + if $(call registry_arch_is_excluded,$$arch); then \ + echo " Skipping $$arch for default variant version tag on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ + continue; \ + fi; \ + refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:v$(DOCKER_MAJOR_VERSION_TAG)"; \ + done; \ + if [ -z "$$refs" ]; then \ + echo "Skipping default variant version-tag manifest (no supported architectures)"; \ + continue; \ + fi; \ + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)" $$refs; \ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):v$(DOCKER_MAJOR_VERSION_TAG)"; \ fi; \ fi; \ From 9dd194f4d768c350879df39e0b9cac1aef981e1d Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Wed, 1 Apr 2026 15:55:31 +0200 Subject: [PATCH 093/113] Update build (#1279) * Update Go minimum version to 1.25.0. * Update Go build to 1.26.x. * Update PromCI Actions. * Enable dependabot for GitHub Actions. Signed-off-by: SuperQ --- .github/dependabot.yml | 8 ++++++++ .github/workflows/ci.yml | 28 +++++++++++++--------------- .promu.yml | 2 +- go.mod | 2 +- 4 files changed, 23 insertions(+), 17 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 202ae2366..76e99715c 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -4,3 +4,11 @@ updates: directory: "/" schedule: interval: "monthly" + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "monthly" + # Exclude configs synced from upstream prometheus/prometheus. + exclude-paths: + - .github/workflows/container_description.yml + - .github/workflows/golangci-lint.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9d759a627..449774254 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -3,6 +3,11 @@ name: CI on: pull_request: push: + branches: [main, master, 'release-*'] + tags: ['v*'] + +permissions: + contents: read jobs: test_go: @@ -11,11 +16,10 @@ jobs: container: # Whenever the Go version is updated here, .promu.yml # should also be updated. - image: quay.io/prometheus/golang-builder:1.25-base + image: quay.io/prometheus/golang-builder:1.26-base steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/setup_environment + - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make test - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1 integration_tests: @@ -46,14 +50,13 @@ jobs: container: # Whenever the Go version is updated here, .promu.yml # should also be updated. - image: quay.io/prometheus/golang-builder:1.25-base + image: quay.io/prometheus/golang-builder:1.26-base env: DATA_SOURCE_NAME: 'postgresql://postgres:test@postgres:5432/circle_test?sslmode=disable' GOOPTS: '-v -tags integration' steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/setup_environment + - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make build - run: make test @@ -74,8 +77,7 @@ jobs: thread: [ 0, 1, 2 ] steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/build + - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" parallelism: 3 @@ -100,14 +102,12 @@ jobs: # should also be updated. steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/build + - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: parallelism: 12 thread: ${{ matrix.thread }} publish_main: - # https://github.com/prometheus/promci/blob/52c7012f5f0070d7281b8db4a119e21341d43c91/actions/publish_main/action.yml name: Publish main branch artifacts runs-on: ubuntu-latest needs: [test_go, build_all] @@ -117,8 +117,7 @@ jobs: (github.event_name == 'push' && github.event.ref == 'refs/heads/master') steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/publish_main + - uses: prometheus/promci/publish_main@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: docker_hub_organization: prometheuscommunity docker_hub_login: ${{ secrets.docker_hub_login }} @@ -135,8 +134,7 @@ jobs: (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 - - uses: ./.github/promci/actions/publish_release + - uses: prometheus/promci/publish_release@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: docker_hub_organization: prometheuscommunity docker_hub_login: ${{ secrets.docker_hub_login }} diff --git a/.promu.yml b/.promu.yml index c9bb51e78..f3b76a765 100644 --- a/.promu.yml +++ b/.promu.yml @@ -1,6 +1,6 @@ go: # This must match .circle/config.yml. - version: 1.25 + version: 1.26 repository: path: github.com/prometheus-community/postgres_exporter build: diff --git a/go.mod b/go.mod index c68d6d359..2844bbc62 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/prometheus-community/postgres_exporter -go 1.24.0 +go 1.25.0 require ( github.com/DATA-DOG/go-sqlmock v1.5.2 From faf05a07afea4cf78a9ac0d8fcaa2b1d65ab993e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 1 Apr 2026 16:04:38 +0200 Subject: [PATCH 094/113] Bump actions/checkout from 5.0.0 to 6.0.2 (#1281) Bumps [actions/checkout](https://github.com/actions/checkout) from 5.0.0 to 6.0.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/08c6903cd8c0fde910a37f88322edcfb5dd907a8...de0fac2e4500dabe0009e67214ff5f5447ce83dd) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 449774254..b93ec99aa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,7 +18,7 @@ jobs: # should also be updated. image: quay.io/prometheus/golang-builder:1.26-base steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make test - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1 @@ -55,7 +55,7 @@ jobs: DATA_SOURCE_NAME: 'postgresql://postgres:test@postgres:5432/circle_test?sslmode=disable' GOOPTS: '-v -tags integration' steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make build - run: make test @@ -76,7 +76,7 @@ jobs: matrix: thread: [ 0, 1, 2 ] steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" @@ -101,7 +101,7 @@ jobs: # Whenever the Go version is updated here, .promu.yml # should also be updated. steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: parallelism: 12 @@ -116,7 +116,7 @@ jobs: || (github.event_name == 'push' && github.event.ref == 'refs/heads/master') steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci/publish_main@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: docker_hub_organization: prometheuscommunity @@ -133,7 +133,7 @@ jobs: if: | (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) steps: - - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci/publish_release@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 with: docker_hub_organization: prometheuscommunity From 4a014e911f3bf63db9365095b0e0bae7e002d5ec Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 1 Apr 2026 21:57:12 +0200 Subject: [PATCH 095/113] Bump prometheus/promci from 0.6.0 to 0.6.1 (#1280) Bumps [prometheus/promci](https://github.com/prometheus/promci) from 0.6.0 to 0.6.1. - [Release notes](https://github.com/prometheus/promci/releases) - [Commits](https://github.com/prometheus/promci/compare/769ee18070cd21cfc2a24fa912349fd3e48dee58...42c3c84c865e5c1ab78543b929f7341eb2ef6123) --- updated-dependencies: - dependency-name: prometheus/promci dependency-version: 0.6.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b93ec99aa..e6073baa6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -77,7 +77,7 @@ jobs: thread: [ 0, 1, 2 ] steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 + - uses: prometheus/promci/build@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" parallelism: 3 @@ -102,7 +102,7 @@ jobs: # should also be updated. steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/build@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 + - uses: prometheus/promci/build@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 with: parallelism: 12 thread: ${{ matrix.thread }} @@ -117,7 +117,7 @@ jobs: (github.event_name == 'push' && github.event.ref == 'refs/heads/master') steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_main@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 + - uses: prometheus/promci/publish_main@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 with: docker_hub_organization: prometheuscommunity docker_hub_login: ${{ secrets.docker_hub_login }} @@ -134,7 +134,7 @@ jobs: (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_release@769ee18070cd21cfc2a24fa912349fd3e48dee58 # v0.6.0 + - uses: prometheus/promci/publish_release@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 with: docker_hub_organization: prometheuscommunity docker_hub_login: ${{ secrets.docker_hub_login }} From ee51a26d96b7c3cbcac03772232bcb7a2b2eb41f Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Wed, 8 Apr 2026 10:07:12 +0200 Subject: [PATCH 096/113] Update common Prometheus files (#1285) Signed-off-by: prombot --- Makefile.common | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile.common b/Makefile.common index cce3ef1d1..52fe9daaf 100644 --- a/Makefile.common +++ b/Makefile.common @@ -55,7 +55,7 @@ ifneq ($(shell command -v gotestsum 2> /dev/null),) endif endif -PROMU_VERSION ?= 0.18.0 +PROMU_VERSION ?= 0.18.1 PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_VERSION)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM).tar.gz SKIP_GOLANGCI_LINT := @@ -91,7 +91,7 @@ $(error DOCKERFILE_PATH is deprecated. Use DOCKERFILE_VARIANTS ?= $(DOCKERFILE_P endif DOCKER_ARCHS ?= amd64 -DOCKERFILE_VARIANTS ?= Dockerfile $(wildcard Dockerfile.*) +DOCKERFILE_VARIANTS ?= $(wildcard Dockerfile Dockerfile.*) # Function to extract variant from Dockerfile label. # Returns the variant name from io.prometheus.image.variant label, or "default" if not found. From ca518649ff786a6372e9f7080bb52ab4271e1e01 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Wed, 8 Apr 2026 22:03:52 +0200 Subject: [PATCH 097/113] Update common Prometheus files (#1286) Signed-off-by: prombot --- Makefile.common | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index 52fe9daaf..907950846 100644 --- a/Makefile.common +++ b/Makefile.common @@ -90,7 +90,9 @@ ifdef DOCKERFILE_PATH $(error DOCKERFILE_PATH is deprecated. Use DOCKERFILE_VARIANTS ?= $(DOCKERFILE_PATH) in the Makefile) endif -DOCKER_ARCHS ?= amd64 +DOCKER_ARCHS ?= amd64 armv7 arm64 ppc64le riscv64 s390x +DOCKERFILE_ARCH_EXCLUSIONS ?= +DOCKER_REGISTRY_ARCH_EXCLUSIONS ?= quay.io:riscv64 DOCKERFILE_VARIANTS ?= $(wildcard Dockerfile Dockerfile.*) # Function to extract variant from Dockerfile label. From 4255a338b7687a60baa840db16b9739d37a033cc Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sat, 11 Apr 2026 08:54:12 +0200 Subject: [PATCH 098/113] Update common Prometheus files (#1287) Signed-off-by: prombot --- .dockerignore | 10 ++++++++++ .github/workflows/golangci-lint.yml | 2 +- Makefile.common | 4 ++-- SECURITY.md | 2 +- 4 files changed, 14 insertions(+), 4 deletions(-) create mode 100644 .dockerignore diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000..096709915 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,10 @@ +data/ +.build/ +.tarballs/ + +!.build/linux-amd64/ +!.build/linux-arm64/ +!.build/linux-armv7/ +!.build/linux-ppc64le/ +!.build/linux-riscv64/ +!.build/linux-s390x/ diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index dc8ffd02d..d7477caf7 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -30,7 +30,7 @@ jobs: with: persist-credentials: false - name: Install Go - uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0 + uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 with: go-version: 1.26.x - name: Install snmp_exporter/generator dependencies diff --git a/Makefile.common b/Makefile.common index 907950846..42dc653d3 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.10.1 +GOLANGCI_LINT_VERSION ?= v2.11.4 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. @@ -90,7 +90,7 @@ ifdef DOCKERFILE_PATH $(error DOCKERFILE_PATH is deprecated. Use DOCKERFILE_VARIANTS ?= $(DOCKERFILE_PATH) in the Makefile) endif -DOCKER_ARCHS ?= amd64 armv7 arm64 ppc64le riscv64 s390x +DOCKER_ARCHS ?= amd64 arm64 armv7 ppc64le riscv64 s390x DOCKERFILE_ARCH_EXCLUSIONS ?= DOCKER_REGISTRY_ARCH_EXCLUSIONS ?= quay.io:riscv64 DOCKERFILE_VARIANTS ?= $(wildcard Dockerfile Dockerfile.*) diff --git a/SECURITY.md b/SECURITY.md index fed02d85c..5e6f976db 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -3,4 +3,4 @@ The Prometheus security policy, including how to report vulnerabilities, can be found here: - +[https://prometheus.io/docs/operating/security/](https://prometheus.io/docs/operating/security/) From 363b3aeddda964e166545809bd9f07d659ecb305 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Fri, 17 Apr 2026 17:04:17 +0200 Subject: [PATCH 099/113] Cleanup CI (#1290) * Cleanup old CircleCI related config. * Split integration tests to separate workflow file. Signed-off-by: SuperQ --- .circleci/config.yml | 33 ----------- .github/workflows/ci.yml | 92 +++---------------------------- .github/workflows/integration.yml | 49 ++++++++++++++++ .promu.yml | 2 +- README.md | 5 +- 5 files changed, 61 insertions(+), 120 deletions(-) delete mode 100644 .circleci/config.yml create mode 100644 .github/workflows/integration.yml diff --git a/.circleci/config.yml b/.circleci/config.yml deleted file mode 100644 index 965b27394..000000000 --- a/.circleci/config.yml +++ /dev/null @@ -1,33 +0,0 @@ ---- -# Prometheus has switched to GitHub action. -# Circle CI is not disabled repository-wise so that previous pull requests -# continue working. -# This file does not generate any CircleCI workflow. - -version: 2.1 - -executors: - golang: - docker: - - image: busybox - -jobs: - noopjob: - executor: golang - - steps: - - run: - command: "true" - -workflows: - version: 2 - prometheus: - jobs: - - noopjob - triggers: - - schedule: - cron: "0 0 30 2 *" - filters: - branches: - only: - - main diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e6073baa6..06ad115f4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -20,126 +20,50 @@ jobs: steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - - run: make test - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1 - integration_tests: - name: Integration tests - runs-on: ubuntu-latest - strategy: - matrix: - # Define Postgres versions to test against. - postgres_version: - - 13.22 - - 14.19 - - 15.14 - - 16.10 - - 17.6 - - 18.1 - services: - postgres: - image: postgres:${{ matrix.postgres_version }} - env: - POSTGRES_DB: circle_test - POSTGRES_USER: postgres - POSTGRES_PASSWORD: test - # options: >- - # --health-cmd="pg_isready -U postgres -d circle_test" - # --health-interval=10s - # --health-timeout=5s - # --health-retries=5 - container: - # Whenever the Go version is updated here, .promu.yml - # should also be updated. - image: quay.io/prometheus/golang-builder:1.26-base - env: - DATA_SOURCE_NAME: 'postgresql://postgres:test@postgres:5432/circle_test?sslmode=disable' - GOOPTS: '-v -tags integration' - steps: - - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - - run: make build - - run: make test - build: - name: Build Prometheus for common architectures - runs-on: ubuntu-latest - if: | - !(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) - && - !(github.event_name == 'pull_request' && startsWith(github.event.pull_request.base.ref, 'release-')) - && - !(github.event_name == 'push' && github.event.ref == 'refs/heads/main') - && - !(github.event_name == 'push' && github.event.ref == 'refs/heads/master') - strategy: - matrix: - thread: [ 0, 1, 2 ] - steps: - - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/build@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 - with: - promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" - parallelism: 3 - thread: ${{ matrix.thread }} - - build_all: - name: Build Prometheus for all architectures + name: Build runs-on: ubuntu-latest - if: | - (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) - || - (github.event_name == 'pull_request' && startsWith(github.event.pull_request.base.ref, 'release-')) - || - (github.event_name == 'push' && github.event.ref == 'refs/heads/main') - || - (github.event_name == 'push' && github.event.ref == 'refs/heads/master') strategy: matrix: - thread: [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 ] - - # Whenever the Go version is updated here, .promu.yml - # should also be updated. + thread: [ 0, 1, 2, 3] steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/build@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 + - uses: prometheus/promci/build@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 with: - parallelism: 12 + parallelism: 4 thread: ${{ matrix.thread }} publish_main: name: Publish main branch artifacts runs-on: ubuntu-latest - needs: [test_go, build_all] + needs: [test_go, build] if: | (github.event_name == 'push' && github.event.ref == 'refs/heads/main') || (github.event_name == 'push' && github.event.ref == 'refs/heads/master') steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_main@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 + - uses: prometheus/promci/publish_main@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 with: docker_hub_organization: prometheuscommunity - docker_hub_login: ${{ secrets.docker_hub_login }} docker_hub_password: ${{ secrets.docker_hub_password }} quay_io_organization: prometheuscommunity - quay_io_login: ${{ secrets.quay_io_login }} quay_io_password: ${{ secrets.quay_io_password }} publish_release: name: Publish release artefacts runs-on: ubuntu-latest - needs: [test_go, build_all] + needs: [test_go, build] if: | (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_release@42c3c84c865e5c1ab78543b929f7341eb2ef6123 # v0.6.1 + - uses: prometheus/promci/publish_release@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 with: docker_hub_organization: prometheuscommunity - docker_hub_login: ${{ secrets.docker_hub_login }} docker_hub_password: ${{ secrets.docker_hub_password }} quay_io_organization: prometheuscommunity - quay_io_login: ${{ secrets.quay_io_login }} quay_io_password: ${{ secrets.quay_io_password }} github_token: ${{ secrets.PROMBOT_GITHUB_TOKEN }} diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml new file mode 100644 index 000000000..8c2c65522 --- /dev/null +++ b/.github/workflows/integration.yml @@ -0,0 +1,49 @@ +--- +name: CI +on: + pull_request: + push: + branches: [main, master, 'release-*'] + tags: ['v*'] + +permissions: + contents: read + +jobs: + integration_tests: + name: Integration tests + runs-on: ubuntu-latest + strategy: + matrix: + # Define Postgres versions to test against. + postgres_version: + - 13.22 + - 14.19 + - 15.14 + - 16.10 + - 17.6 + - 18.1 + services: + postgres: + image: postgres:${{ matrix.postgres_version }} + env: + POSTGRES_DB: circle_test + POSTGRES_USER: postgres + POSTGRES_PASSWORD: test + # options: >- + # --health-cmd="pg_isready -U postgres -d circle_test" + # --health-interval=10s + # --health-timeout=5s + # --health-retries=5 + container: + # Whenever the Go version is updated here, .promu.yml + # should also be updated. + image: quay.io/prometheus/golang-builder:1.26-base + env: + DATA_SOURCE_NAME: 'postgresql://postgres:test@postgres:5432/circle_test?sslmode=disable' + GOOPTS: '-v -tags integration' + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 + - run: make build + - run: make test diff --git a/.promu.yml b/.promu.yml index f3b76a765..0f1be42c7 100644 --- a/.promu.yml +++ b/.promu.yml @@ -1,5 +1,5 @@ go: - # This must match .circle/config.yml. + # This must match .github/workflows/ci.yml. version: 1.26 repository: path: github.com/prometheus-community/postgres_exporter diff --git a/README.md b/README.md index d0cdf5947..36f2f329f 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,10 @@ -[![Build Status](https://circleci.com/gh/prometheus-community/postgres_exporter.svg?style=svg)](https://circleci.com/gh/prometheus-community/postgres_exporter) +# PostgreSQL Server Exporter + +[![Build Status](https://github.com/prometheus-community/postgres_exporter/actions/workflows/ci.yml/badge.svg)](https://github.com/prometheus-community/postgres_exporter/actions/workflows/ci.yml) [![Coverage Status](https://coveralls.io/repos/github/prometheus-community/postgres_exporter/badge.svg?branch=master)](https://coveralls.io/github/prometheus-community/postgres_exporter?branch=master) [![Go Report Card](https://goreportcard.com/badge/github.com/prometheus-community/postgres_exporter)](https://goreportcard.com/report/github.com/prometheus-community/postgres_exporter) [![Docker Pulls](https://img.shields.io/docker/pulls/prometheuscommunity/postgres-exporter.svg)](https://hub.docker.com/r/prometheuscommunity/postgres-exporter/tags) -# PostgreSQL Server Exporter Prometheus exporter for PostgreSQL server metrics. From 3b0317c07ebb3aa21646bc6c318398500a708660 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Fri, 17 Apr 2026 17:06:43 +0200 Subject: [PATCH 100/113] Update common Prometheus files (#1288) Signed-off-by: prombot --- Makefile.common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.common b/Makefile.common index 42dc653d3..6bd1e1526 100644 --- a/Makefile.common +++ b/Makefile.common @@ -92,7 +92,7 @@ endif DOCKER_ARCHS ?= amd64 arm64 armv7 ppc64le riscv64 s390x DOCKERFILE_ARCH_EXCLUSIONS ?= -DOCKER_REGISTRY_ARCH_EXCLUSIONS ?= quay.io:riscv64 +DOCKER_REGISTRY_ARCH_EXCLUSIONS ?= DOCKERFILE_VARIANTS ?= $(wildcard Dockerfile Dockerfile.*) # Function to extract variant from Dockerfile label. From 8879f5a1c6db4ed88bed601f4f5850770c0341c1 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 21 Apr 2026 08:32:52 +0200 Subject: [PATCH 101/113] Update common Prometheus files (#1291) Signed-off-by: prombot --- Makefile.common | 72 ------------------------------------------------- 1 file changed, 72 deletions(-) diff --git a/Makefile.common b/Makefile.common index 6bd1e1526..6cd3320bc 100644 --- a/Makefile.common +++ b/Makefile.common @@ -91,8 +91,6 @@ $(error DOCKERFILE_PATH is deprecated. Use DOCKERFILE_VARIANTS ?= $(DOCKERFILE_P endif DOCKER_ARCHS ?= amd64 arm64 armv7 ppc64le riscv64 s390x -DOCKERFILE_ARCH_EXCLUSIONS ?= -DOCKER_REGISTRY_ARCH_EXCLUSIONS ?= DOCKERFILE_VARIANTS ?= $(wildcard Dockerfile Dockerfile.*) # Function to extract variant from Dockerfile label. @@ -111,24 +109,6 @@ endif # Build variant:dockerfile pairs for shell iteration. DOCKERFILE_VARIANTS_WITH_NAMES := $(foreach df,$(DOCKERFILE_VARIANTS),$(call dockerfile_variant,$(df)):$(df)) -# Shell helper to check whether a dockerfile/arch pair is excluded. -define dockerfile_arch_is_excluded -case " $(DOCKERFILE_ARCH_EXCLUSIONS) " in \ - *" $$dockerfile:$(1) "*) true ;; \ - *) false ;; \ -esac -endef - -# Shell helper to check whether a registry/arch pair is excluded. -# Extracts registry from DOCKER_REPO (e.g., quay.io/prometheus -> quay.io) -define registry_arch_is_excluded -registry=$$(echo "$(DOCKER_REPO)" | cut -d'/' -f1); \ -case " $(DOCKER_REGISTRY_ARCH_EXCLUSIONS) " in \ - *" $$registry:$(1) "*) true ;; \ - *) false ;; \ -esac -endef - BUILD_DOCKER_ARCHS = $(addprefix common-docker-,$(DOCKER_ARCHS)) PUBLISH_DOCKER_ARCHS = $(addprefix common-docker-publish-,$(DOCKER_ARCHS)) TAG_DOCKER_ARCHS = $(addprefix common-docker-tag-latest-,$(DOCKER_ARCHS)) @@ -270,10 +250,6 @@ $(BUILD_DOCKER_ARCHS): common-docker-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ - if $(call dockerfile_arch_is_excluded,$*); then \ - echo "Skipping $$variant_name variant for linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ distroless_arch="$*"; \ if [ "$*" = "armv7" ]; then \ distroless_arch="arm"; \ @@ -308,14 +284,6 @@ $(PUBLISH_DOCKER_ARCHS): common-docker-publish-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ - if $(call dockerfile_arch_is_excluded,$*); then \ - echo "Skipping push for $$variant_name variant on linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$*); then \ - echo "Skipping push for $$variant_name variant on linux-$* to $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Pushing $$variant_name variant for linux-$*"; \ docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ @@ -343,14 +311,6 @@ $(TAG_DOCKER_ARCHS): common-docker-tag-latest-%: @for variant in $(DOCKERFILE_VARIANTS_WITH_NAMES); do \ dockerfile=$${variant#*:}; \ variant_name=$${variant%%:*}; \ - if $(call dockerfile_arch_is_excluded,$*); then \ - echo "Skipping tag for $$variant_name variant on linux-$* (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$*); then \ - echo "Skipping tag for $$variant_name variant on linux-$* for $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ if [ "$$dockerfile" != "Dockerfile" ] || [ "$$variant_name" != "default" ]; then \ echo "Tagging $$variant_name variant for linux-$* as latest"; \ docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest-$$variant_name"; \ @@ -372,14 +332,6 @@ common-docker-manifest: echo "Creating manifest for $$variant_name variant"; \ refs=""; \ for arch in $(DOCKER_ARCHS); do \ - if $(call dockerfile_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for $$variant_name (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for $$variant_name on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:$(SANITIZED_DOCKER_IMAGE_TAG)-$$variant_name"; \ done; \ if [ -z "$$refs" ]; then \ @@ -393,14 +345,6 @@ common-docker-manifest: echo "Creating default variant ($$variant_name) manifest"; \ refs=""; \ for arch in $(DOCKER_ARCHS); do \ - if $(call dockerfile_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for default variant (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for default variant on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:$(SANITIZED_DOCKER_IMAGE_TAG)"; \ done; \ if [ -z "$$refs" ]; then \ @@ -415,14 +359,6 @@ common-docker-manifest: echo "Creating manifest for $$variant_name variant version tag"; \ refs=""; \ for arch in $(DOCKER_ARCHS); do \ - if $(call dockerfile_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for $$variant_name version tag (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for $$variant_name version tag on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:v$(DOCKER_MAJOR_VERSION_TAG)-$$variant_name"; \ done; \ if [ -z "$$refs" ]; then \ @@ -436,14 +372,6 @@ common-docker-manifest: echo "Creating default variant version tag manifest"; \ refs=""; \ for arch in $(DOCKER_ARCHS); do \ - if $(call dockerfile_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for default variant version tag (excluded by DOCKERFILE_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ - if $(call registry_arch_is_excluded,$$arch); then \ - echo " Skipping $$arch for default variant version tag on $(DOCKER_REPO) (excluded by DOCKER_REGISTRY_ARCH_EXCLUSIONS)"; \ - continue; \ - fi; \ refs="$$refs $(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$$arch:v$(DOCKER_MAJOR_VERSION_TAG)"; \ done; \ if [ -z "$$refs" ]; then \ From f9d6723c3123fdda199795aafdbc726ee63bba85 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Thu, 23 Apr 2026 09:39:38 +0200 Subject: [PATCH 102/113] Update common Prometheus files (#1293) Signed-off-by: prombot --- .github/workflows/govulncheck.yml | 21 +++++++++++++++++++++ Makefile.common | 6 ------ 2 files changed, 21 insertions(+), 6 deletions(-) create mode 100644 .github/workflows/govulncheck.yml diff --git a/.github/workflows/govulncheck.yml b/.github/workflows/govulncheck.yml new file mode 100644 index 000000000..df4a014e5 --- /dev/null +++ b/.github/workflows/govulncheck.yml @@ -0,0 +1,21 @@ +--- +name: govulncheck +on: + pull_request: + push: + branches: + - main + - master + schedule: + - cron: '33 2 * * *' + +permissions: + contents: read + +jobs: + govulncheck: + runs-on: ubuntu-latest + name: Run govulncheck + steps: + - id: govulncheck + uses: golang/govulncheck-action@31f7c5463448f83528bd771c2d978d940080c9fd # v1.0.4-unreleased diff --git a/Makefile.common b/Makefile.common index 6cd3320bc..ef05881be 100644 --- a/Makefile.common +++ b/Makefile.common @@ -425,9 +425,3 @@ $(1)_precheck: exit 1; \ fi endef - -govulncheck: install-govulncheck - govulncheck ./... - -install-govulncheck: - command -v govulncheck > /dev/null || go install golang.org/x/vuln/cmd/govulncheck@latest From 8ad13dc6b658237a21acde613a5b6e04de01f4f5 Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 12 May 2026 03:58:19 +0200 Subject: [PATCH 103/113] Update common Prometheus files (#1300) Signed-off-by: prombot --- .github/workflows/govulncheck.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/govulncheck.yml b/.github/workflows/govulncheck.yml index df4a014e5..1e13fbd99 100644 --- a/.github/workflows/govulncheck.yml +++ b/.github/workflows/govulncheck.yml @@ -2,6 +2,9 @@ name: govulncheck on: pull_request: + paths: + - VERSION + - .github/workflows/govulncheck.yml push: branches: - main From 6f2a7a301a5c3d0bcfe50ff908d5a989f688ccec Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 May 2026 22:40:18 -0400 Subject: [PATCH 104/113] Bump github.com/prometheus/exporter-toolkit from 0.15.1 to 0.16.0 (#1295) Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.15.1 to 0.16.0. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.15.1...v0.16.0) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-version: 0.16.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 18 +++++++++--------- go.sum | 36 ++++++++++++++++++------------------ 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/go.mod b/go.mod index 2844bbc62..f50fda431 100644 --- a/go.mod +++ b/go.mod @@ -10,7 +10,7 @@ require ( github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.67.5 - github.com/prometheus/exporter-toolkit v0.15.1 + github.com/prometheus/exporter-toolkit v0.16.0 github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 @@ -21,7 +21,7 @@ require ( github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/coreos/go-systemd/v22 v22.6.0 // indirect + github.com/coreos/go-systemd/v22 v22.7.0 // indirect github.com/golang-jwt/jwt/v5 v5.3.0 // indirect github.com/google/uuid v1.6.0 // indirect github.com/gopherjs/gopherjs v1.17.2 // indirect @@ -37,13 +37,13 @@ require ( github.com/rogpeppe/go-internal v1.10.0 // indirect github.com/smarty/assertions v1.15.0 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - go.yaml.in/yaml/v2 v2.4.3 // indirect - golang.org/x/crypto v0.46.0 // indirect - golang.org/x/net v0.48.0 // indirect + go.yaml.in/yaml/v2 v2.4.4 // indirect + golang.org/x/crypto v0.49.0 // indirect + golang.org/x/net v0.51.0 // indirect golang.org/x/oauth2 v0.34.0 // indirect - golang.org/x/sync v0.19.0 // indirect - golang.org/x/sys v0.39.0 // indirect - golang.org/x/text v0.32.0 // indirect - golang.org/x/time v0.14.0 // indirect + golang.org/x/sync v0.20.0 // indirect + golang.org/x/sys v0.42.0 // indirect + golang.org/x/text v0.35.0 // indirect + golang.org/x/time v0.15.0 // indirect google.golang.org/protobuf v1.36.11 // indirect ) diff --git a/go.sum b/go.sum index a40e54c06..deea6eedf 100644 --- a/go.sum +++ b/go.sum @@ -10,8 +10,8 @@ github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/coreos/go-systemd/v22 v22.6.0 h1:aGVa/v8B7hpb0TKl0MWoAavPDmHvobFe5R5zn0bCJWo= -github.com/coreos/go-systemd/v22 v22.6.0/go.mod h1:iG+pp635Fo7ZmV/j14KUcmEyWF+0X7Lua8rrTWzYgWU= +github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA= +github.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= @@ -59,8 +59,8 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4= github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw= -github.com/prometheus/exporter-toolkit v0.15.1 h1:XrGGr/qWl8Gd+pqJqTkNLww9eG8vR/CoRk0FubOKfLE= -github.com/prometheus/exporter-toolkit v0.15.1/go.mod h1:P/NR9qFRGbCFgpklyhix9F6v6fFr/VQB/CVsrMDGKo4= +github.com/prometheus/exporter-toolkit v0.16.0 h1:xT/j7L2XKF+VJd6B4fpUw6xWabHrSmsUf6mYmFqyu0s= +github.com/prometheus/exporter-toolkit v0.16.0/go.mod h1:d1EL8Z9674xQe/iWhwP2wDyCEoBPbXVeqDbqAUsgJWY= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -84,22 +84,22 @@ github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8 github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= -go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= -golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU= -golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0= -golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU= -golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY= +go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ= +go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ= +golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4= +golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA= +golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo= +golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y= golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= -golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= -golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= -golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= -golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= -golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY= -golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= -golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= +golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= +golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8= +golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA= +golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= +golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= From 7b5d85fe96bac3a2cf9424f4dd1bf0a611ffee2f Mon Sep 17 00:00:00 2001 From: Arthur Silva Sens Date: Tue, 12 May 2026 10:12:46 -0300 Subject: [PATCH 105/113] Remove global auth config reload metrics (#1296) Signed-off-by: Arthur Silva Sens --- cmd/postgres_exporter/main.go | 12 +++-- config/config.go | 96 +++++++++++++++++++++++++---------- config/config_test.go | 53 ++++++++++++++++--- 3 files changed, 125 insertions(+), 36 deletions(-) diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index 0c1bd003e..8f58796ef 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -35,9 +35,7 @@ import ( ) var ( - c = config.Handler{ - Config: &config.Config{}, - } + c = newConfigHandler() configFile = kingpin.Flag("config.file", "Postgres exporter configuration file.").Default("postgres_exporter.yml").String() webConfig = kingpinflag.AddFlags(kingpin.CommandLine, ":9187") @@ -58,6 +56,14 @@ var ( // The name of the exporter. const exporterName = "postgres_exporter" +func newConfigHandler() *config.Handler { + handler, err := config.NewHandler(prometheus.DefaultRegisterer) + if err != nil { + panic(err) + } + return handler +} + func main() { kingpin.Version(version.Print(exporterName)) promslogConfig := &promslog.Config{} diff --git a/config/config.go b/config/config.go index 52c66513a..d2588d327 100644 --- a/config/config.go +++ b/config/config.go @@ -14,30 +14,17 @@ package config import ( + "errors" "fmt" + "io" "log/slog" "os" "sync" "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promauto" "gopkg.in/yaml.v3" ) -var ( - configReloadSuccess = promauto.NewGauge(prometheus.GaugeOpts{ - Namespace: "postgres_exporter", - Name: "config_last_reload_successful", - Help: "Postgres exporter config loaded successfully.", - }) - - configReloadSeconds = promauto.NewGauge(prometheus.GaugeOpts{ - Namespace: "postgres_exporter", - Name: "config_last_reload_success_timestamp_seconds", - Help: "Timestamp of the last successful configuration reload.", - }) -) - type Config struct { AuthModules map[string]AuthModule `yaml:"auth_modules"` } @@ -57,6 +44,31 @@ type UserPass struct { type Handler struct { sync.RWMutex Config *Config + + configReloadSuccess prometheus.Gauge + configReloadSeconds prometheus.Gauge +} + +func NewHandler(registerer prometheus.Registerer) (*Handler, error) { + if registerer == nil { + return nil, errors.New("registerer is required") + } + h := &Handler{ + Config: &Config{}, + configReloadSuccess: prometheus.NewGauge(prometheus.GaugeOpts{ + Namespace: "postgres_exporter", + Name: "config_last_reload_successful", + Help: "Postgres exporter config loaded successfully.", + }), + configReloadSeconds: prometheus.NewGauge(prometheus.GaugeOpts{ + Namespace: "postgres_exporter", + Name: "config_last_reload_success_timestamp_seconds", + Help: "Timestamp of the last successful configuration reload.", + }), + } + registerer.MustRegister(h.configReloadSuccess, h.configReloadSeconds) + + return h, nil } func (ch *Handler) GetConfig() *Config { @@ -66,33 +78,63 @@ func (ch *Handler) GetConfig() *Config { } func (ch *Handler) ReloadConfig(f string, logger *slog.Logger) error { - config := &Config{} var err error defer func() { - if err != nil { - configReloadSuccess.Set(0) - } else { - configReloadSuccess.Set(1) - configReloadSeconds.SetToCurrentTime() - } + ch.observeReload(err) }() + config, err := LoadConfig(f) + if err != nil { + return err + } + + ch.SetConfig(config) + return nil +} + +func (ch *Handler) observeReload(err error) { + if ch.configReloadSuccess == nil { + return + } + if err != nil { + ch.configReloadSuccess.Set(0) + return + } + ch.configReloadSuccess.Set(1) + if ch.configReloadSeconds != nil { + ch.configReloadSeconds.SetToCurrentTime() + } +} + +func LoadConfig(f string) (*Config, error) { yamlReader, err := os.Open(f) if err != nil { - return fmt.Errorf("error opening config file %q: %s", f, err) + return nil, fmt.Errorf("error opening config file %q: %s", f, err) } defer yamlReader.Close() - decoder := yaml.NewDecoder(yamlReader) + + config, err := DecodeConfig(yamlReader) + if err != nil { + return nil, fmt.Errorf("error parsing config file %q: %s", f, err) + } + return config, nil +} + +func DecodeConfig(r io.Reader) (*Config, error) { + config := &Config{} + decoder := yaml.NewDecoder(r) decoder.KnownFields(true) - if err = decoder.Decode(config); err != nil { - return fmt.Errorf("error parsing config file %q: %s", f, err) + if err := decoder.Decode(config); err != nil { + return nil, err } + return config, nil +} +func (ch *Handler) SetConfig(config *Config) { ch.Lock() ch.Config = config ch.Unlock() - return nil } func (m AuthModule) ConfigureTarget(target string) (DSN, error) { diff --git a/config/config_test.go b/config/config_test.go index fa59c9b40..00ded7e93 100644 --- a/config/config_test.go +++ b/config/config_test.go @@ -14,23 +14,64 @@ package config import ( + "strings" "testing" + + "github.com/prometheus/client_golang/prometheus" ) -func TestLoadConfig(t *testing.T) { - ch := &Handler{ - Config: &Config{}, +func TestLoadConfigFile(t *testing.T) { + config, err := LoadConfig("testdata/config-good.yaml") + if err != nil { + t.Fatalf("LoadConfig() error = %v", err) + } + if len(config.AuthModules) == 0 { + t.Fatal("LoadConfig() loaded no auth modules") + } +} + +func TestDecodeConfig(t *testing.T) { + config, err := DecodeConfig(strings.NewReader(` +auth_modules: + module: + type: userpass + userpass: + username: user + password: pass +`)) + if err != nil { + t.Fatalf("DecodeConfig() error = %v", err) + } + if got, want := config.AuthModules["module"].UserPass.Username, "user"; got != want { + t.Fatalf("username = %q, want %q", got, want) } +} - err := ch.ReloadConfig("testdata/config-good.yaml", nil) +func TestLoadConfig(t *testing.T) { + ch, err := NewHandler(prometheus.NewRegistry()) if err != nil { + t.Fatalf("NewHandler() error = %v", err) + } + + if err := ch.ReloadConfig("testdata/config-good.yaml", nil); err != nil { t.Errorf("error loading config: %s", err) } } +func TestNewHandlerRequiresRegisterer(t *testing.T) { + handler, err := NewHandler(nil) + if err == nil { + t.Fatal("NewHandler() error = nil, want error") + } + if handler != nil { + t.Fatalf("NewHandler() handler = %v, want nil", handler) + } +} + func TestLoadBadConfigs(t *testing.T) { - ch := &Handler{ - Config: &Config{}, + ch, err := NewHandler(prometheus.NewRegistry()) + if err != nil { + t.Fatalf("NewHandler() error = %v", err) } tests := []struct { From aea92ea967cc903913b4980ccd8d9474e282ee4b Mon Sep 17 00:00:00 2001 From: Arthur Silva Sens Date: Fri, 15 May 2026 13:58:42 -0300 Subject: [PATCH 106/113] Refactor stat_archiver into standalone collector (#1304) * Refactor stat_archiver into standalone collector Signed-off-by: Arthur Silva Sens * Don't create metric for NULL returns Signed-off-by: Arthur Silva Sens --------- Signed-off-by: Arthur Silva Sens --- README.md | 3 + collector/pg_stat_archiver.go | 103 +++++++++++++++++++++++++ collector/pg_stat_archiver_test.go | 119 +++++++++++++++++++++++++++++ exporter/postgres_exporter.go | 14 ---- exporter/queries.go | 11 --- 5 files changed, 225 insertions(+), 25 deletions(-) create mode 100644 collector/pg_stat_archiver.go create mode 100644 collector/pg_stat_archiver_test.go diff --git a/README.md b/README.md index 36f2f329f..3d296d9b1 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.stat_activity_autovacuum` Enable the `stat_activity_autovacuum` collector (default: disabled). +* `[no-]collector.stat_archiver` + Enable the `stat_archiver` collector (default: enabled). + * `[no-]collector.stat_bgwriter` Enable the `stat_bgwriter` collector (default: enabled). diff --git a/collector/pg_stat_archiver.go b/collector/pg_stat_archiver.go new file mode 100644 index 000000000..abc315741 --- /dev/null +++ b/collector/pg_stat_archiver.go @@ -0,0 +1,103 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" +) + +const statArchiverSubsystem = "stat_archiver" + +func init() { + registerCollector(statArchiverSubsystem, defaultEnabled, NewPGStatArchiverCollector) +} + +type PGStatArchiverCollector struct{} + +func NewPGStatArchiverCollector(collectorConfig) (Collector, error) { + return &PGStatArchiverCollector{}, nil +} + +var ( + statArchiverArchivedCountDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statArchiverSubsystem, "archived_count"), + "Number of WAL files that have been successfully archived", + []string{}, + prometheus.Labels{}, + ) + statArchiverFailedCountDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statArchiverSubsystem, "failed_count"), + "Number of failed attempts for archiving WAL files", + []string{}, + prometheus.Labels{}, + ) + statArchiverLastArchiveAgeDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statArchiverSubsystem, "last_archive_age"), + "Time in seconds since last WAL segment was successfully archived", + []string{}, + prometheus.Labels{}, + ) + + statArchiverQuery = `SELECT + archived_count, + failed_count, + extract(epoch from now() - last_archived_time) AS last_archive_age + FROM pg_stat_archiver;` +) + +func (PGStatArchiverCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + if instance.version.LT(semver.MustParse("9.4.0")) { + return nil + } + + db := instance.getDB() + row := db.QueryRowContext(ctx, statArchiverQuery) + + var archivedCount, failedCount sql.NullInt64 + var lastArchiveAge sql.NullFloat64 + + if err := row.Scan(&archivedCount, &failedCount, &lastArchiveAge); err != nil { + return err + } + + if archivedCount.Valid { + ch <- prometheus.MustNewConstMetric( + statArchiverArchivedCountDesc, + prometheus.CounterValue, + float64(archivedCount.Int64), + ) + } + + if failedCount.Valid { + ch <- prometheus.MustNewConstMetric( + statArchiverFailedCountDesc, + prometheus.CounterValue, + float64(failedCount.Int64), + ) + } + + if lastArchiveAge.Valid { + ch <- prometheus.MustNewConstMetric( + statArchiverLastArchiveAgeDesc, + prometheus.GaugeValue, + lastArchiveAge.Float64, + ) + } + + return nil +} diff --git a/collector/pg_stat_archiver_test.go b/collector/pg_stat_archiver_test.go new file mode 100644 index 000000000..f1b141d91 --- /dev/null +++ b/collector/pg_stat_archiver_test.go @@ -0,0 +1,119 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatArchiverCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{"archived_count", "failed_count", "last_archive_age"}). + AddRow(42, 7, 13.5) + mock.ExpectQuery(sanitizeQuery(statArchiverQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatArchiverCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatArchiverCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 42}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 7}, + {labels: labelMap{}, metricType: dto.MetricType_GAUGE, value: 13.5}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatArchiverCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{"archived_count", "failed_count", "last_archive_age"}). + AddRow(nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statArchiverQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatArchiverCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatArchiverCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for NULL stat_archiver value: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatArchiverCollectorBeforePostgres94(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("9.3.0")} + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatArchiverCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatArchiverCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for PostgreSQL 9.3: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/exporter/postgres_exporter.go b/exporter/postgres_exporter.go index 720a4667f..9f6bcb5f5 100644 --- a/exporter/postgres_exporter.go +++ b/exporter/postgres_exporter.go @@ -245,20 +245,6 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ true, 0, }, - "pg_stat_archiver": { - map[string]ColumnMapping{ - "archived_count": {COUNTER, "Number of WAL files that have been successfully archived", nil, nil}, - "last_archived_wal": {DISCARD, "Name of the last WAL file successfully archived", nil, nil}, - "last_archived_time": {DISCARD, "Time of the last successful archive operation", nil, nil}, - "failed_count": {COUNTER, "Number of failed attempts for archiving WAL files", nil, nil}, - "last_failed_wal": {DISCARD, "Name of the WAL file of the last failed archival operation", nil, nil}, - "last_failed_time": {DISCARD, "Time of the last failed archival operation", nil, nil}, - "stats_reset": {DISCARD, "Time at which these statistics were last reset", nil, nil}, - "last_archive_age": {GAUGE, "Time in seconds since last WAL segment was successfully archived", nil, nil}, - }, - true, - 0, - }, "pg_stat_activity": { map[string]ColumnMapping{ "datname": {LABEL, "Name of this database", nil, nil}, diff --git a/exporter/queries.go b/exporter/queries.go index b0f64f657..b835af136 100644 --- a/exporter/queries.go +++ b/exporter/queries.go @@ -95,17 +95,6 @@ var queryOverrides = map[string][]OverrideQuery{ }, }, - "pg_stat_archiver": { - { - semver.MustParseRange(">=9.4.0"), - ` - SELECT *, - extract(epoch from now() - last_archived_time) AS last_archive_age - FROM pg_stat_archiver - `, - }, - }, - "pg_stat_activity": { // This query only works { From d0bac0b47d06c139a5e7bcc42bd3347f724703c0 Mon Sep 17 00:00:00 2001 From: Arthur Silva Sens Date: Fri, 15 May 2026 14:19:03 -0300 Subject: [PATCH 107/113] Refactor stat_activity into standalone collector (#1305) * Refactor stat_activity into standalone collector Signed-off-by: Arthur Silva Sens * Do not expose metrics for NULL results Signed-off-by: Arthur Silva Sens --------- Signed-off-by: Arthur Silva Sens --- README.md | 3 + collector/pg_stat_activity.go | 181 +++++++++++++++++++++++++++++ collector/pg_stat_activity_test.go | 174 +++++++++++++++++++++++++++ exporter/postgres_exporter.go | 15 --- exporter/queries.go | 59 ---------- 5 files changed, 358 insertions(+), 74 deletions(-) create mode 100644 collector/pg_stat_activity.go create mode 100644 collector/pg_stat_activity_test.go diff --git a/README.md b/README.md index 3d296d9b1..0a58d20fb 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.replication_slot` Enable the `replication_slot` collector (default: enabled). +* `[no-]collector.stat_activity` + Enable the `stat_activity` collector (default: enabled). + * `[no-]collector.stat_activity_autovacuum` Enable the `stat_activity_autovacuum` collector (default: disabled). diff --git a/collector/pg_stat_activity.go b/collector/pg_stat_activity.go new file mode 100644 index 000000000..517fd25f2 --- /dev/null +++ b/collector/pg_stat_activity.go @@ -0,0 +1,181 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" +) + +const statActivitySubsystem = "stat_activity" + +func init() { + registerCollector(statActivitySubsystem, defaultEnabled, NewPGStatActivityCollector) +} + +type PGStatActivityCollector struct{} + +func NewPGStatActivityCollector(collectorConfig) (Collector, error) { + return &PGStatActivityCollector{}, nil +} + +var ( + statActivityLabels = []string{ + "datname", + "state", + "usename", + "application_name", + "backend_type", + "wait_event_type", + "wait_event", + } + statActivityCountDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statActivitySubsystem, "count"), + "number of connections in this state", + statActivityLabels, + prometheus.Labels{}, + ) + statActivityMaxTxDurationDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statActivitySubsystem, "max_tx_duration"), + "max duration in seconds any active transaction has been running", + statActivityLabels, + prometheus.Labels{}, + ) + + statActivityQuery = ` + SELECT + pg_database.datname, + tmp.state, + tmp2.usename, + tmp2.application_name, + tmp2.backend_type, + tmp2.wait_event_type, + tmp2.wait_event, + COALESCE(count,0) as count, + COALESCE(max_tx_duration,0) as max_tx_duration + FROM + ( + VALUES ('active'), + ('idle'), + ('idle in transaction'), + ('idle in transaction (aborted)'), + ('fastpath function call'), + ('disabled') + ) AS tmp(state) CROSS JOIN pg_database + LEFT JOIN + ( + SELECT + datname, + state, + usename, + application_name, + backend_type, + wait_event_type, + wait_event, + count(*) AS count, + MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration + FROM pg_stat_activity + WHERE pid <> pg_backend_pid() + GROUP BY datname,state,usename,application_name,backend_type,wait_event_type,wait_event) AS tmp2 + ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname + ` + + statActivityQueryBefore92 = ` + SELECT + datname, + 'unknown' AS state, + usename, + application_name, + '' AS backend_type, + '' AS wait_event_type, + '' AS wait_event, + COALESCE(count(*),0) AS count, + COALESCE(MAX(EXTRACT(EPOCH FROM now() - xact_start))::float,0) AS max_tx_duration + FROM pg_stat_activity + WHERE procpid <> pg_backend_pid() + GROUP BY datname,usename,application_name + ` +) + +func (PGStatActivityCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + query := statActivityQuery + if instance.version.LT(semver.MustParse("9.2.0")) { + query = statActivityQueryBefore92 + } + + db := instance.getDB() + rows, err := db.QueryContext(ctx, query) + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var datname, state, usename, applicationName, backendType, waitEventType, waitEvent sql.NullString + var count, maxTxDuration sql.NullFloat64 + if err := rows.Scan( + &datname, + &state, + &usename, + &applicationName, + &backendType, + &waitEventType, + &waitEvent, + &count, + &maxTxDuration, + ); err != nil { + return err + } + + labels := []string{ + stringValue(datname), + stringValue(state), + stringValue(usename), + stringValue(applicationName), + stringValue(backendType), + stringValue(waitEventType), + stringValue(waitEvent), + } + + if count.Valid { + ch <- prometheus.MustNewConstMetric( + statActivityCountDesc, + prometheus.GaugeValue, + count.Float64, + labels..., + ) + } + + if maxTxDuration.Valid { + ch <- prometheus.MustNewConstMetric( + statActivityMaxTxDurationDesc, + prometheus.GaugeValue, + maxTxDuration.Float64, + labels..., + ) + } + } + + return rows.Err() +} + +func stringValue(s sql.NullString) string { + if s.Valid { + return s.String + } + return "" +} diff --git a/collector/pg_stat_activity_test.go b/collector/pg_stat_activity_test.go new file mode 100644 index 000000000..0525153c4 --- /dev/null +++ b/collector/pg_stat_activity_test.go @@ -0,0 +1,174 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatActivityCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{ + "datname", + "state", + "usename", + "application_name", + "backend_type", + "wait_event_type", + "wait_event", + "count", + "max_tx_duration", + }).AddRow("postgres", "active", "postgres", "psql", "client backend", "Lock", "relation", 3, 12.5) + mock.ExpectQuery(sanitizeQuery(statActivityQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatActivityCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatActivityCollector.Update: %s", err) + } + }() + + expectedLabels := labelMap{ + "datname": "postgres", + "state": "active", + "usename": "postgres", + "application_name": "psql", + "backend_type": "client backend", + "wait_event_type": "Lock", + "wait_event": "relation", + } + expected := []MetricResult{ + {labels: expectedLabels, value: 3, metricType: dto.MetricType_GAUGE}, + {labels: expectedLabels, value: 12.5, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatActivityCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{ + "datname", + "state", + "usename", + "application_name", + "backend_type", + "wait_event_type", + "wait_event", + "count", + "max_tx_duration", + }).AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statActivityQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatActivityCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatActivityCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for NULL stat_activity value: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatActivityCollectorBefore92(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("9.1.0")} + + rows := sqlmock.NewRows([]string{ + "datname", + "state", + "usename", + "application_name", + "backend_type", + "wait_event_type", + "wait_event", + "count", + "max_tx_duration", + }).AddRow("postgres", "unknown", "postgres", "psql", "", "", "", 2, 7.25) + mock.ExpectQuery(sanitizeQuery(statActivityQueryBefore92)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatActivityCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatActivityCollector.Update: %s", err) + } + }() + + expectedLabels := labelMap{ + "datname": "postgres", + "state": "unknown", + "usename": "postgres", + "application_name": "psql", + "backend_type": "", + "wait_event_type": "", + "wait_event": "", + } + expected := []MetricResult{ + {labels: expectedLabels, value: 2, metricType: dto.MetricType_GAUGE}, + {labels: expectedLabels, value: 7.25, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/exporter/postgres_exporter.go b/exporter/postgres_exporter.go index 9f6bcb5f5..4090d8d81 100644 --- a/exporter/postgres_exporter.go +++ b/exporter/postgres_exporter.go @@ -245,21 +245,6 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ true, 0, }, - "pg_stat_activity": { - map[string]ColumnMapping{ - "datname": {LABEL, "Name of this database", nil, nil}, - "state": {LABEL, "connection state", nil, semver.MustParseRange(">=9.2.0")}, - "usename": {LABEL, "connection usename", nil, nil}, - "application_name": {LABEL, "connection application_name", nil, nil}, - "backend_type": {LABEL, "connection backend_type", nil, nil}, - "wait_event_type": {LABEL, "connection wait_event_type", nil, nil}, - "wait_event": {LABEL, "connection wait_event", nil, nil}, - "count": {GAUGE, "number of connections in this state", nil, nil}, - "max_tx_duration": {GAUGE, "max duration in seconds any active transaction has been running", nil, nil}, - }, - true, - 0, - }, } // Turn the MetricMap column mapping into a prometheus descriptor mapping. diff --git a/exporter/queries.go b/exporter/queries.go index b835af136..16813879e 100644 --- a/exporter/queries.go +++ b/exporter/queries.go @@ -94,65 +94,6 @@ var queryOverrides = map[string][]OverrideQuery{ `, }, }, - - "pg_stat_activity": { - // This query only works - { - semver.MustParseRange(">=9.2.0"), - ` - SELECT - pg_database.datname, - tmp.state, - tmp2.usename, - tmp2.application_name, - tmp2.backend_type, - tmp2.wait_event_type, - tmp2.wait_event, - COALESCE(count,0) as count, - COALESCE(max_tx_duration,0) as max_tx_duration - FROM - ( - VALUES ('active'), - ('idle'), - ('idle in transaction'), - ('idle in transaction (aborted)'), - ('fastpath function call'), - ('disabled') - ) AS tmp(state) CROSS JOIN pg_database - LEFT JOIN - ( - SELECT - datname, - state, - usename, - application_name, - backend_type, - wait_event_type, - wait_event, - count(*) AS count, - MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration - FROM pg_stat_activity - WHERE pid <> pg_backend_pid() - GROUP BY datname,state,usename,application_name,backend_type,wait_event_type,wait_event) AS tmp2 - ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname - `, - }, - { - semver.MustParseRange("<9.2.0"), - ` - SELECT - datname, - 'unknown' AS state, - usename, - application_name, - COALESCE(count(*),0) AS count, - COALESCE(MAX(EXTRACT(EPOCH FROM now() - xact_start))::float,0) AS max_tx_duration - FROM pg_stat_activity - WHERE procpid <> pg_backend_pid() - GROUP BY datname,usename,application_name - `, - }, - }, } // Convert the query override file to the version-specific query override file From c629747bf7d1e46c03cb3c7d8fc6e45fd7dd5a9e Mon Sep 17 00:00:00 2001 From: Arthur Silva Sens Date: Sat, 16 May 2026 14:35:18 -0300 Subject: [PATCH 108/113] Refactor stat_replication into standalone collector (#1306) * Refactor stat_replication into standalone collector Signed-off-by: Arthur Silva Sens * Do not expose metrics for NULL results Signed-off-by: Arthur Silva Sens --------- Signed-off-by: Arthur Silva Sens --- README.md | 3 + collector/pg_stat_replication.go | 170 ++++++++++++++++++++ collector/pg_stat_replication_test.go | 221 ++++++++++++++++++++++++++ exporter/postgres_exporter.go | 46 ------ exporter/queries.go | 30 ---- 5 files changed, 394 insertions(+), 76 deletions(-) create mode 100644 collector/pg_stat_replication.go create mode 100644 collector/pg_stat_replication_test.go diff --git a/README.md b/README.md index 0a58d20fb..dba089a9f 100644 --- a/README.md +++ b/README.md @@ -157,6 +157,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.stat_progress_vacuum` Enable the `stat_progress_vacuum` collector (default: enabled). +* `[no-]collector.stat_replication` + Enable the `stat_replication` collector (default: enabled). + * `[no-]collector.stat_statements` Enable the `stat_statements` collector (default: disabled). diff --git a/collector/pg_stat_replication.go b/collector/pg_stat_replication.go new file mode 100644 index 000000000..816f93f60 --- /dev/null +++ b/collector/pg_stat_replication.go @@ -0,0 +1,170 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" +) + +const statReplicationSubsystem = "stat_replication" + +func init() { + registerCollector(statReplicationSubsystem, defaultEnabled, NewPGStatReplicationCollector) +} + +type PGStatReplicationCollector struct{} + +func NewPGStatReplicationCollector(collectorConfig) (Collector, error) { + return &PGStatReplicationCollector{}, nil +} + +var ( + statReplicationLabels = []string{"application_name", "client_addr", "state", "slot_name"} + + statReplicationCurrentWalLSNBytesDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statReplicationSubsystem, "pg_current_wal_lsn_bytes"), + "WAL position in bytes", + statReplicationLabels, + prometheus.Labels{}, + ) + statReplicationWalLSNDiffDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statReplicationSubsystem, "pg_wal_lsn_diff"), + "Lag in bytes between master and slave", + statReplicationLabels, + prometheus.Labels{}, + ) + statReplicationXlogLocationDiffDesc = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statReplicationSubsystem, "pg_xlog_location_diff"), + "Lag in bytes between master and slave", + statReplicationLabels, + prometheus.Labels{}, + ) + + statReplicationQuery = ` + SELECT + application_name, + client_addr::text, + state, + slot_name, + (case pg_is_in_recovery() when 't' then pg_wal_lsn_diff(pg_last_wal_receive_lsn(), pg_lsn('0/0'))::float else pg_wal_lsn_diff(pg_current_wal_lsn(), pg_lsn('0/0'))::float end) AS pg_current_wal_lsn_bytes, + (case pg_is_in_recovery() when 't' then pg_wal_lsn_diff(pg_last_wal_receive_lsn(), replay_lsn)::float else pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)::float end) AS pg_wal_lsn_diff + FROM pg_stat_replication + ` + + statReplicationQueryBefore10 = ` + SELECT + application_name, + client_addr::text, + state, + slot_name, + (case pg_is_in_recovery() when 't' then pg_xlog_location_diff(pg_last_xlog_receive_location(), replay_location)::float else pg_xlog_location_diff(pg_current_xlog_location(), replay_location)::float end) AS pg_xlog_location_diff + FROM pg_stat_replication + ` +) + +func (PGStatReplicationCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + switch { + case instance.version.GTE(semver.MustParse("10.0.0")): + return updateStatReplication(ctx, instance, ch) + case instance.version.GTE(semver.MustParse("9.2.0")): + return updateStatReplicationBefore10(ctx, instance, ch) + default: + return nil + } +} + +func updateStatReplication(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, statReplicationQuery) + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var applicationName, clientAddr, state, slotName sql.NullString + var currentWalLSNBytes, walLSNDiff sql.NullFloat64 + if err := rows.Scan(&applicationName, &clientAddr, &state, &slotName, ¤tWalLSNBytes, &walLSNDiff); err != nil { + return err + } + labels := statReplicationLabelValues(applicationName, clientAddr, state, slotName) + + if currentWalLSNBytes.Valid { + ch <- prometheus.MustNewConstMetric( + statReplicationCurrentWalLSNBytesDesc, + prometheus.GaugeValue, + currentWalLSNBytes.Float64, + labels..., + ) + } + if walLSNDiff.Valid { + ch <- prometheus.MustNewConstMetric( + statReplicationWalLSNDiffDesc, + prometheus.GaugeValue, + walLSNDiff.Float64, + labels..., + ) + } + } + + return rows.Err() +} + +func updateStatReplicationBefore10(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, statReplicationQueryBefore10) + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var applicationName, clientAddr, state, slotName sql.NullString + var xlogLocationDiff sql.NullFloat64 + if err := rows.Scan(&applicationName, &clientAddr, &state, &slotName, &xlogLocationDiff); err != nil { + return err + } + + if xlogLocationDiff.Valid { + ch <- prometheus.MustNewConstMetric( + statReplicationXlogLocationDiffDesc, + prometheus.GaugeValue, + xlogLocationDiff.Float64, + statReplicationLabelValues(applicationName, clientAddr, state, slotName)..., + ) + } + } + + return rows.Err() +} + +func statReplicationLabelValues(applicationName, clientAddr, state, slotName sql.NullString) []string { + return []string{ + nullStringValue(applicationName), + nullStringValue(clientAddr), + nullStringValue(state), + nullStringValue(slotName), + } +} + +func nullStringValue(s sql.NullString) string { + if s.Valid { + return s.String + } + return "" +} diff --git a/collector/pg_stat_replication_test.go b/collector/pg_stat_replication_test.go new file mode 100644 index 000000000..60917c0a9 --- /dev/null +++ b/collector/pg_stat_replication_test.go @@ -0,0 +1,221 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatReplicationCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{ + "application_name", + "client_addr", + "state", + "slot_name", + "pg_current_wal_lsn_bytes", + "pg_wal_lsn_diff", + }).AddRow("standby", "10.0.0.1", "streaming", "slot_a", 1024.0, 64.0) + mock.ExpectQuery(sanitizeQuery(statReplicationQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatReplicationCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatReplicationCollector.Update: %s", err) + } + }() + + expectedLabels := labelMap{ + "application_name": "standby", + "client_addr": "10.0.0.1", + "state": "streaming", + "slot_name": "slot_a", + } + expected := []MetricResult{ + {labels: expectedLabels, value: 1024, metricType: dto.MetricType_GAUGE}, + {labels: expectedLabels, value: 64, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatReplicationCollectorBefore10(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("9.6.0")} + + rows := sqlmock.NewRows([]string{ + "application_name", + "client_addr", + "state", + "slot_name", + "pg_xlog_location_diff", + }).AddRow("standby", "10.0.0.1", "streaming", "slot_a", 32.0) + mock.ExpectQuery(sanitizeQuery(statReplicationQueryBefore10)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatReplicationCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatReplicationCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + { + labels: labelMap{ + "application_name": "standby", + "client_addr": "10.0.0.1", + "state": "streaming", + "slot_name": "slot_a", + }, + value: 32, + metricType: dto.MetricType_GAUGE, + }, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatReplicationCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("16.0.0")} + + rows := sqlmock.NewRows([]string{ + "application_name", + "client_addr", + "state", + "slot_name", + "pg_current_wal_lsn_bytes", + "pg_wal_lsn_diff", + }).AddRow(nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statReplicationQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatReplicationCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatReplicationCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for NULL stat_replication value: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatReplicationCollectorBefore10NullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("9.6.0")} + + rows := sqlmock.NewRows([]string{ + "application_name", + "client_addr", + "state", + "slot_name", + "pg_xlog_location_diff", + }).AddRow(nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statReplicationQueryBefore10)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatReplicationCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatReplicationCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for NULL stat_replication value: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatReplicationCollectorBefore92(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("9.1.0")} + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatReplicationCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatReplicationCollector.Update: %s", err) + } + }() + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted for PostgreSQL 9.1: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/exporter/postgres_exporter.go b/exporter/postgres_exporter.go index 4090d8d81..ad7b48cd0 100644 --- a/exporter/postgres_exporter.go +++ b/exporter/postgres_exporter.go @@ -189,52 +189,6 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ true, 0, }, - "pg_stat_replication": { - map[string]ColumnMapping{ - "procpid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange("<9.2.0")}, - "pid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange(">=9.2.0")}, - "usesysid": {DISCARD, "OID of the user logged into this WAL sender process", nil, nil}, - "usename": {DISCARD, "Name of the user logged into this WAL sender process", nil, nil}, - "application_name": {LABEL, "Name of the application that is connected to this WAL sender", nil, nil}, - "client_addr": {LABEL, "IP address of the client connected to this WAL sender. If this field is null, it indicates that the client is connected via a Unix socket on the server machine.", nil, nil}, - "client_hostname": {DISCARD, "Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when log_hostname is enabled.", nil, nil}, - "client_port": {DISCARD, "TCP port number that the client is using for communication with this WAL sender, or -1 if a Unix socket is used", nil, nil}, - "backend_start": {DISCARD, "with time zone Time when this process was started, i.e., when the client connected to this WAL sender", nil, nil}, - "backend_xmin": {DISCARD, "The current backend's xmin horizon.", nil, nil}, - "state": {LABEL, "Current WAL sender state", nil, nil}, - "sent_location": {DISCARD, "Last transaction log position sent on this connection", nil, semver.MustParseRange("<10.0.0")}, - "write_location": {DISCARD, "Last transaction log position written to disk by this standby server", nil, semver.MustParseRange("<10.0.0")}, - "flush_location": {DISCARD, "Last transaction log position flushed to disk by this standby server", nil, semver.MustParseRange("<10.0.0")}, - "replay_location": {DISCARD, "Last transaction log position replayed into the database on this standby server", nil, semver.MustParseRange("<10.0.0")}, - "sent_lsn": {DISCARD, "Last transaction log position sent on this connection", nil, semver.MustParseRange(">=10.0.0")}, - "write_lsn": {DISCARD, "Last transaction log position written to disk by this standby server", nil, semver.MustParseRange(">=10.0.0")}, - "flush_lsn": {DISCARD, "Last transaction log position flushed to disk by this standby server", nil, semver.MustParseRange(">=10.0.0")}, - "replay_lsn": {DISCARD, "Last transaction log position replayed into the database on this standby server", nil, semver.MustParseRange(">=10.0.0")}, - "sync_priority": {DISCARD, "Priority of this standby server for being chosen as the synchronous standby", nil, nil}, - "sync_state": {DISCARD, "Synchronous state of this standby server", nil, nil}, - "slot_name": {LABEL, "A unique, cluster-wide identifier for the replication slot", nil, semver.MustParseRange(">=9.2.0")}, - "plugin": {DISCARD, "The base name of the shared object containing the output plugin this logical slot is using, or null for physical slots", nil, nil}, - "slot_type": {DISCARD, "The slot type - physical or logical", nil, nil}, - "datoid": {DISCARD, "The OID of the database this slot is associated with, or null. Only logical slots have an associated database", nil, nil}, - "database": {DISCARD, "The name of the database this slot is associated with, or null. Only logical slots have an associated database", nil, nil}, - "active": {DISCARD, "True if this slot is currently actively being used", nil, nil}, - "active_pid": {DISCARD, "Process ID of a WAL sender process", nil, nil}, - "xmin": {DISCARD, "The oldest transaction that this slot needs the database to retain. VACUUM cannot remove tuples deleted by any later transaction", nil, nil}, - "catalog_xmin": {DISCARD, "The oldest transaction affecting the system catalogs that this slot needs the database to retain. VACUUM cannot remove catalog tuples deleted by any later transaction", nil, nil}, - "restart_lsn": {DISCARD, "The address (LSN) of oldest WAL which still might be required by the consumer of this slot and thus won't be automatically removed during checkpoints", nil, nil}, - "pg_current_xlog_location": {DISCARD, "pg_current_xlog_location", nil, nil}, - "pg_current_wal_lsn": {DISCARD, "pg_current_xlog_location", nil, semver.MustParseRange(">=10.0.0")}, - "pg_current_wal_lsn_bytes": {GAUGE, "WAL position in bytes", nil, semver.MustParseRange(">=10.0.0")}, - "pg_xlog_location_diff": {GAUGE, "Lag in bytes between master and slave", nil, semver.MustParseRange(">=9.2.0 <10.0.0")}, - "pg_wal_lsn_diff": {GAUGE, "Lag in bytes between master and slave", nil, semver.MustParseRange(">=10.0.0")}, - "confirmed_flush_lsn": {DISCARD, "LSN position a consumer of a slot has confirmed flushing the data received", nil, nil}, - "write_lag": {DISCARD, "Time elapsed between flushing recent WAL locally and receiving notification that this standby server has written it (but not yet flushed it or applied it). This can be used to gauge the delay that synchronous_commit level remote_write incurred while committing if this server was configured as a synchronous standby.", nil, semver.MustParseRange(">=10.0.0")}, - "flush_lag": {DISCARD, "Time elapsed between flushing recent WAL locally and receiving notification that this standby server has written and flushed it (but not yet applied it). This can be used to gauge the delay that synchronous_commit level remote_flush incurred while committing if this server was configured as a synchronous standby.", nil, semver.MustParseRange(">=10.0.0")}, - "replay_lag": {DISCARD, "Time elapsed between flushing recent WAL locally and receiving notification that this standby server has written, flushed and applied it. This can be used to gauge the delay that synchronous_commit level remote_apply incurred while committing if this server was configured as a synchronous standby.", nil, semver.MustParseRange(">=10.0.0")}, - }, - true, - 0, - }, "pg_replication_slots": { map[string]ColumnMapping{ "slot_name": {LABEL, "Name of the replication slot", nil, nil}, diff --git a/exporter/queries.go b/exporter/queries.go index 16813879e..cb24f0b16 100644 --- a/exporter/queries.go +++ b/exporter/queries.go @@ -46,36 +46,6 @@ type OverrideQuery struct { // Overriding queries for namespaces above. // TODO: validate this is a closed set in tests, and there are no overlaps var queryOverrides = map[string][]OverrideQuery{ - "pg_stat_replication": { - { - semver.MustParseRange(">=10.0.0"), - ` - SELECT *, - (case pg_is_in_recovery() when 't' then pg_last_wal_receive_lsn() else pg_current_wal_lsn() end) AS pg_current_wal_lsn, - (case pg_is_in_recovery() when 't' then pg_wal_lsn_diff(pg_last_wal_receive_lsn(), pg_lsn('0/0'))::float else pg_wal_lsn_diff(pg_current_wal_lsn(), pg_lsn('0/0'))::float end) AS pg_current_wal_lsn_bytes, - (case pg_is_in_recovery() when 't' then pg_wal_lsn_diff(pg_last_wal_receive_lsn(), replay_lsn)::float else pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)::float end) AS pg_wal_lsn_diff - FROM pg_stat_replication - `, - }, - { - semver.MustParseRange(">=9.2.0 <10.0.0"), - ` - SELECT *, - (case pg_is_in_recovery() when 't' then pg_last_xlog_receive_location() else pg_current_xlog_location() end) AS pg_current_xlog_location, - (case pg_is_in_recovery() when 't' then pg_xlog_location_diff(pg_last_xlog_receive_location(), replay_location)::float else pg_xlog_location_diff(pg_current_xlog_location(), replay_location)::float end) AS pg_xlog_location_diff - FROM pg_stat_replication - `, - }, - { - semver.MustParseRange("<9.2.0"), - ` - SELECT *, - (case pg_is_in_recovery() when 't' then pg_last_xlog_receive_location() else pg_current_xlog_location() end) AS pg_current_xlog_location - FROM pg_stat_replication - `, - }, - }, - "pg_replication_slots": { { semver.MustParseRange(">=9.4.0 <10.0.0"), From 15beb334488e5812de9199928ca1e015ff62278b Mon Sep 17 00:00:00 2001 From: Arthur Silva Sens Date: Sat, 16 May 2026 22:56:25 -0300 Subject: [PATCH 109/113] Refactor pg_settings into a standalone collector (#1303) * Refactor pg_settings into a standalone collector Signed-off-by: Arthur Silva Sens * Fix linter Signed-off-by: Arthur Silva Sens --------- Signed-off-by: Arthur Silva Sens --- README-RDS.md | 4 +- README.md | 9 +- cmd/postgres_exporter/main.go | 28 +- cmd/postgres_exporter/probe.go | 1 - {exporter => collector}/pg_setting.go | 62 ++-- collector/pg_setting_test.go | 200 +++++++++++++ exporter/datasource.go | 2 +- exporter/pg_setting_test.go | 267 ------------------ exporter/postgres_exporter.go | 16 +- .../postgres_exporter_integration_test.go | 6 - exporter/server.go | 9 +- 11 files changed, 257 insertions(+), 347 deletions(-) rename {exporter => collector}/pg_setting.go (72%) create mode 100644 collector/pg_setting_test.go delete mode 100644 exporter/pg_setting_test.go diff --git a/README-RDS.md b/README-RDS.md index 85bf8691f..e0135855f 100644 --- a/README-RDS.md +++ b/README-RDS.md @@ -22,9 +22,9 @@ Running postgres-exporter in a container like so: -e DATA_SOURCE_NAME="postgresql://${PGUSER}:${PGPASS}@${PGHOST}:5432/${DBNAME}?sslmode=disable" \ -e PG_EXPORTER_EXCLUDE_DATABASES=rdsadmin \ -e PG_EXPORTER_DISABLE_DEFAULT_METRICS=true \ - -e PG_EXPORTER_DISABLE_SETTINGS_METRICS=true \ -e PG_EXPORTER_EXTEND_QUERY_PATH='/var/lib/postgresql/queries.yaml' \ - quay.io/prometheuscommunity/postgres-exporter + quay.io/prometheuscommunity/postgres-exporter \ + --no-collector.settings ``` ### Expected changes to RDS: diff --git a/README.md b/README.md index dba089a9f..a3577c15f 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,9 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `[no-]collector.replication_slot` Enable the `replication_slot` collector (default: enabled). +* `[no-]collector.settings` + Enable the `settings` collector (default: enabled). + * `[no-]collector.stat_activity` Enable the `stat_activity` collector (default: enabled). @@ -215,9 +218,6 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `disable-default-metrics` Use only metrics supplied from `queries.yaml` via `--extend.query-path`. Default is `false`. -* `disable-settings-metrics` - Use the flag if you don't want to scrape `pg_settings`. Default is `false`. - * `auto-discover-databases` (DEPRECATED) Whether to discover the databases on a server dynamically. Default is `false`. @@ -291,9 +291,6 @@ The following environment variables configure the exporter: * `PG_EXPORTER_DISABLE_DEFAULT_METRICS` Use only metrics supplied from `queries.yaml`. Value can be `true` or `false`. Default is `false`. -* `PG_EXPORTER_DISABLE_SETTINGS_METRICS` - Use the flag if you don't want to scrape `pg_settings`. Value can be `true` or `false`. Default is `false`. - * `PG_EXPORTER_AUTO_DISCOVER_DATABASES` (DEPRECATED) Whether to discover the databases on a server dynamically. Value can be `true` or `false`. Default is `false`. diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index 8f58796ef..93c56aec4 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -37,20 +37,19 @@ import ( var ( c = newConfigHandler() - configFile = kingpin.Flag("config.file", "Postgres exporter configuration file.").Default("postgres_exporter.yml").String() - webConfig = kingpinflag.AddFlags(kingpin.CommandLine, ":9187") - metricsPath = kingpin.Flag("web.telemetry-path", "Path under which to expose metrics.").Default("/metrics").Envar("PG_EXPORTER_WEB_TELEMETRY_PATH").String() - disableDefaultMetrics = kingpin.Flag("disable-default-metrics", "Do not include default metrics.").Default("false").Envar("PG_EXPORTER_DISABLE_DEFAULT_METRICS").Bool() - disableSettingsMetrics = kingpin.Flag("disable-settings-metrics", "Do not include pg_settings metrics.").Default("false").Envar("PG_EXPORTER_DISABLE_SETTINGS_METRICS").Bool() - autoDiscoverDatabases = kingpin.Flag("auto-discover-databases", "Whether to discover the databases on a server dynamically. (DEPRECATED)").Default("false").Envar("PG_EXPORTER_AUTO_DISCOVER_DATABASES").Bool() - queriesPath = kingpin.Flag("extend.query-path", "Path to custom queries to run. (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXTEND_QUERY_PATH").String() - onlyDumpMaps = kingpin.Flag("dumpmaps", "Do not run, simply dump the maps.").Bool() - constantLabelsList = kingpin.Flag("constantLabels", "A list of label=value separated by comma(,). (DEPRECATED)").Default("").Envar("PG_EXPORTER_CONSTANT_LABELS").String() - excludeDatabases = kingpin.Flag("exclude-databases", "A list of databases to remove when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXCLUDE_DATABASES").String() - includeDatabases = kingpin.Flag("include-databases", "A list of databases to include when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_INCLUDE_DATABASES").String() - metricPrefix = kingpin.Flag("metric-prefix", "A metric prefix can be used to have non-default (not \"pg\") prefixes for each of the metrics").Default("pg").Envar("PG_EXPORTER_METRIC_PREFIX").String() - collectionTimeout = kingpin.Flag("collection-timeout", "Timeout for collecting the statistics when the database is slow").Default("1m").Envar("PG_EXPORTER_COLLECTION_TIMEOUT").String() - logger = promslog.NewNopLogger() + configFile = kingpin.Flag("config.file", "Postgres exporter configuration file.").Default("postgres_exporter.yml").String() + webConfig = kingpinflag.AddFlags(kingpin.CommandLine, ":9187") + metricsPath = kingpin.Flag("web.telemetry-path", "Path under which to expose metrics.").Default("/metrics").Envar("PG_EXPORTER_WEB_TELEMETRY_PATH").String() + disableDefaultMetrics = kingpin.Flag("disable-default-metrics", "Do not include default metrics.").Default("false").Envar("PG_EXPORTER_DISABLE_DEFAULT_METRICS").Bool() + autoDiscoverDatabases = kingpin.Flag("auto-discover-databases", "Whether to discover the databases on a server dynamically. (DEPRECATED)").Default("false").Envar("PG_EXPORTER_AUTO_DISCOVER_DATABASES").Bool() + queriesPath = kingpin.Flag("extend.query-path", "Path to custom queries to run. (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXTEND_QUERY_PATH").String() + onlyDumpMaps = kingpin.Flag("dumpmaps", "Do not run, simply dump the maps.").Bool() + constantLabelsList = kingpin.Flag("constantLabels", "A list of label=value separated by comma(,). (DEPRECATED)").Default("").Envar("PG_EXPORTER_CONSTANT_LABELS").String() + excludeDatabases = kingpin.Flag("exclude-databases", "A list of databases to remove when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXCLUDE_DATABASES").String() + includeDatabases = kingpin.Flag("include-databases", "A list of databases to include when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_INCLUDE_DATABASES").String() + metricPrefix = kingpin.Flag("metric-prefix", "A metric prefix can be used to have non-default (not \"pg\") prefixes for each of the metrics").Default("pg").Envar("PG_EXPORTER_METRIC_PREFIX").String() + collectionTimeout = kingpin.Flag("collection-timeout", "Timeout for collecting the statistics when the database is slow").Default("1m").Envar("PG_EXPORTER_COLLECTION_TIMEOUT").String() + logger = promslog.NewNopLogger() ) // The name of the exporter. @@ -105,7 +104,6 @@ func main() { opts := []exporter.ExporterOpt{ exporter.DisableDefaultMetrics(*disableDefaultMetrics), - exporter.DisableSettingsMetrics(*disableSettingsMetrics), exporter.AutoDiscoverDatabases(*autoDiscoverDatabases), exporter.WithUserQueriesPath(*queriesPath), exporter.WithConstantLabels(*constantLabelsList), diff --git a/cmd/postgres_exporter/probe.go b/cmd/postgres_exporter/probe.go index 66219c07d..043c5a191 100644 --- a/cmd/postgres_exporter/probe.go +++ b/cmd/postgres_exporter/probe.go @@ -67,7 +67,6 @@ func handleProbe(logger *slog.Logger, excludeDatabases []string) http.HandlerFun opts := []exporter.ExporterOpt{ exporter.DisableDefaultMetrics(*disableDefaultMetrics), - exporter.DisableSettingsMetrics(*disableSettingsMetrics), exporter.AutoDiscoverDatabases(*autoDiscoverDatabases), exporter.WithUserQueriesPath(*queriesPath), exporter.WithConstantLabels(*constantLabelsList), diff --git a/exporter/pg_setting.go b/collector/pg_setting.go similarity index 72% rename from exporter/pg_setting.go rename to collector/pg_setting.go index 770536c4b..73ef543fe 100644 --- a/exporter/pg_setting.go +++ b/collector/pg_setting.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -11,10 +11,12 @@ // See the License for the specific language governing permissions and // limitations under the License. -package exporter +package collector import ( + "context" "fmt" + "log/slog" "math" "strconv" "strings" @@ -22,66 +24,74 @@ import ( "github.com/prometheus/client_golang/prometheus" ) +const settingsSubsystem = "settings" + +func init() { + registerCollector(settingsSubsystem, defaultEnabled, NewPGSettingsCollector) +} + +type PGSettingsCollector struct { + log *slog.Logger +} + +func NewPGSettingsCollector(config collectorConfig) (Collector, error) { + return &PGSettingsCollector{log: config.logger}, nil +} + var ( settingUnits = []string{ "ms", "s", "min", "h", "d", "B", "kB", "MB", "GB", "TB", } -) - -// Query the pg_settings view containing runtime variables -func querySettings(ch chan<- prometheus.Metric, server *Server) error { - server.logger.Debug("Querying pg_setting view", "server", server) // pg_settings docs: https://www.postgresql.org/docs/current/static/view-pg-settings.html // // NOTE: If you add more vartypes here, you must update the supported - // types in normaliseUnit() below + // types in normaliseUnit() below. // // Settings intentionally ignored due to invalid format: // - `sync_commit_cancel_wait`, specific to Azure Postgres, see https://github.com/prometheus-community/postgres_exporter/issues/523 // - `google_dataplex.max_messages`, specific to Google Cloud SQL, see https://github.com/prometheus-community/postgres_exporter/issues/1240 - query := "SELECT name, setting, COALESCE(unit, ''), short_desc, vartype FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real') AND name NOT IN ('sync_commit_cancel_wait', 'google_dataplex.max_messages');" + pgSettingsQuery = "SELECT name, setting, COALESCE(unit, ''), short_desc, vartype FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real') AND name NOT IN ('sync_commit_cancel_wait', 'google_dataplex.max_messages');" +) - rows, err := server.db.Query(query) +// Update implements Collector and exposes PostgreSQL runtime settings. +func (c PGSettingsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, pgSettingsQuery) if err != nil { - return fmt.Errorf("Error running query on database %q: %s %v", server, namespace, err) + return err } - defer rows.Close() // nolint: errcheck + defer rows.Close() for rows.Next() { s := &pgSetting{} - err = rows.Scan(&s.name, &s.setting, &s.unit, &s.shortDesc, &s.vartype) - if err != nil { - return fmt.Errorf("Error retrieving rows on %q: %s %v", server, namespace, err) + if err := rows.Scan(&s.name, &s.setting, &s.unit, &s.shortDesc, &s.vartype); err != nil { + return err } - metric, err := s.metric(server.labels) + metric, err := s.metric() if err != nil { - // Log the error and continue - // This could be due to a bad value in the setting - // but we should not fail the entire scrape or panic - server.logger.Warn("Error normalising unit for setting", "setting", s.name, "value", s.setting, "unit", s.unit, "error", err) + c.log.Warn("Error normalising unit for setting", "setting", s.name, "value", s.setting, "unit", s.unit, "error", err) continue } ch <- metric } - return nil + return rows.Err() } -// pgSetting is represents a PostgreSQL runtime variable as returned by the +// pgSetting represents a PostgreSQL runtime variable as returned by the // pg_settings view. type pgSetting struct { name, setting, unit, shortDesc, vartype string } -func (s *pgSetting) metric(labels prometheus.Labels) (prometheus.Metric, error) { +func (s *pgSetting) metric() (prometheus.Metric, error) { var ( err error name = strings.ReplaceAll(strings.ReplaceAll(s.name, ".", "_"), "-", "_") unit = s.unit // nolint: ineffassign shortDesc = fmt.Sprintf("Server Parameter: %s", s.name) - subsystem = "settings" val float64 ) @@ -103,7 +113,7 @@ func (s *pgSetting) metric(labels prometheus.Labels) (prometheus.Metric, error) return nil, fmt.Errorf("pgsetting: unsupported vartype %q", s.vartype) } - desc := newDesc(subsystem, name, shortDesc, labels) + desc := prometheus.NewDesc(prometheus.BuildFQName(namespace, settingsSubsystem, name), shortDesc, nil, nil) return prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, val), nil } @@ -127,7 +137,7 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) { val, err = strconv.ParseFloat(s.setting, 64) if err != nil { - return val, unit, fmt.Errorf("Error converting setting %q value %q to float: %s", s.name, s.setting, err) + return val, unit, fmt.Errorf("error converting setting %q value %q to float: %s", s.name, s.setting, err) } // Units defined in: https://www.postgresql.org/docs/current/static/config-setting.html diff --git a/collector/pg_setting_test.go b/collector/pg_setting_test.go new file mode 100644 index 000000000..ba66980cb --- /dev/null +++ b/collector/pg_setting_test.go @@ -0,0 +1,200 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/prometheus/common/promslog" +) + +func TestPGSettingNormaliseUnit(t *testing.T) { + tests := []struct { + name string + setting pgSetting + wantValue float64 + wantUnit string + wantErr string + }{ + { + name: "seconds", + setting: pgSetting{name: "seconds_fixture_metric", setting: "5", unit: "s", vartype: "integer"}, + wantValue: 5, + wantUnit: "seconds", + }, + { + name: "milliseconds", + setting: pgSetting{name: "milliseconds_fixture_metric", setting: "5000", unit: "ms", vartype: "integer"}, + wantValue: 5, + wantUnit: "seconds", + }, + { + name: "8kB", + setting: pgSetting{name: "eight_kb_fixture_metric", setting: "17", unit: "8kB", vartype: "integer"}, + wantValue: 139264, + wantUnit: "bytes", + }, + { + name: "special minus one", + setting: pgSetting{name: "special_minus_one_value", setting: "-1", unit: "d", vartype: "integer"}, + wantValue: -1, + wantUnit: "seconds", + }, + { + name: "unknown unit", + setting: pgSetting{name: "unknown_unit", setting: "10", unit: "nonexistent", vartype: "integer"}, + wantValue: 10, + wantErr: `unknown unit for runtime variable: "nonexistent"`, + }, + { + name: "value with unit suffix", + setting: pgSetting{name: "aurora_value", setting: "16MB", unit: "MB", vartype: "integer"}, + wantValue: 16777216, + wantUnit: "bytes", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + gotValue, gotUnit, err := tt.setting.normaliseUnit() + if gotValue != tt.wantValue { + t.Fatalf("normaliseUnit() value = %v, want %v", gotValue, tt.wantValue) + } + if gotUnit != tt.wantUnit { + t.Fatalf("normaliseUnit() unit = %q, want %q", gotUnit, tt.wantUnit) + } + if tt.wantErr == "" && err != nil { + t.Fatalf("normaliseUnit() unexpected error: %v", err) + } + if tt.wantErr != "" { + if err == nil { + t.Fatalf("normaliseUnit() expected error %q", tt.wantErr) + } + if err.Error() != tt.wantErr { + t.Fatalf("normaliseUnit() error = %q, want %q", err.Error(), tt.wantErr) + } + } + }) + } +} + +func TestPGSettingMetric(t *testing.T) { + tests := []struct { + name string + setting pgSetting + wantDesc string + wantMetric float64 + }{ + { + name: "integer seconds", + setting: pgSetting{name: "seconds_fixture_metric", setting: "5", unit: "s", vartype: "integer"}, + wantDesc: `Desc{fqName: "pg_settings_seconds_fixture_metric_seconds", help: "Server Parameter: seconds_fixture_metric [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, + wantMetric: 5, + }, + { + name: "bool on", + setting: pgSetting{name: "bool_on_fixture_metric", setting: "on", vartype: "bool"}, + wantDesc: `Desc{fqName: "pg_settings_bool_on_fixture_metric", help: "Server Parameter: bool_on_fixture_metric", constLabels: {}, variableLabels: {}}`, + wantMetric: 1, + }, + { + name: "sanitized name", + setting: pgSetting{name: "rds.rds-superuser-reserved-connections", setting: "2", vartype: "integer"}, + wantDesc: `Desc{fqName: "pg_settings_rds_rds_superuser_reserved_connections", help: "Server Parameter: rds.rds-superuser-reserved-connections", constLabels: {}, variableLabels: {}}`, + wantMetric: 2, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + metric, err := tt.setting.metric() + if err != nil { + t.Fatalf("metric() unexpected error: %v", err) + } + got := &dto.Metric{} + if err := metric.Write(got); err != nil { + t.Fatalf("Write() unexpected error: %v", err) + } + if metric.Desc().String() != tt.wantDesc { + t.Fatalf("metric() desc = %q, want %q", metric.Desc().String(), tt.wantDesc) + } + if got.GetGauge().GetValue() != tt.wantMetric { + t.Fatalf("metric() value = %v, want %v", got.GetGauge().GetValue(), tt.wantMetric) + } + }) + } +} + +func TestPGSettingsCollectorUpdate(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + rows := sqlmock.NewRows([]string{"name", "setting", "unit", "short_desc", "vartype"}). + AddRow("shared_buffers", "128", "8kB", "Sets the number of shared memory buffers used by the server.", "integer"). + AddRow("track_counts", "on", "", "Collects statistics on database activity.", "bool"). + AddRow("bad_setting", "not-a-number", "", "Bad setting.", "integer") + mock.ExpectQuery(sanitizeQuery(pgSettingsQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + collector := PGSettingsCollector{log: promslog.NewNopLogger()} + if err := collector.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGSettingsCollector.Update: %s", err) + } + }() + + tests := []struct { + wantDesc string + wantValue float64 + }{ + { + wantDesc: `Desc{fqName: "pg_settings_shared_buffers_bytes", help: "Server Parameter: shared_buffers [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, + wantValue: 1048576, + }, + { + wantDesc: `Desc{fqName: "pg_settings_track_counts", help: "Server Parameter: track_counts", constLabels: {}, variableLabels: {}}`, + wantValue: 1, + }, + } + + for _, tt := range tests { + metric := <-ch + got := &dto.Metric{} + if err := metric.Write(got); err != nil { + t.Fatalf("Write() unexpected error: %v", err) + } + if metric.Desc().String() != tt.wantDesc { + t.Fatalf("metric desc = %q, want %q", metric.Desc().String(), tt.wantDesc) + } + if got.GetGauge().GetValue() != tt.wantValue { + t.Fatalf("metric value = %v, want %v", got.GetGauge().GetValue(), tt.wantValue) + } + } + + if metric, ok := <-ch; ok { + t.Fatalf("unexpected metric emitted after bad setting was skipped: %s", metric.Desc()) + } + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/exporter/datasource.go b/exporter/datasource.go index 9ae3410fe..7e4f1bd11 100644 --- a/exporter/datasource.go +++ b/exporter/datasource.go @@ -112,7 +112,7 @@ func (e *Exporter) scrapeDSN(ch chan<- prometheus.Metric, dsn string) error { e.logger.Warn("Proceeding with outdated query maps, as the Postgres version could not be determined", "err", err) } - return server.Scrape(ch, e.disableSettingsMetrics) + return server.Scrape(ch) } // try to get the DataSource diff --git a/exporter/pg_setting_test.go b/exporter/pg_setting_test.go deleted file mode 100644 index 20fa65c07..000000000 --- a/exporter/pg_setting_test.go +++ /dev/null @@ -1,267 +0,0 @@ -// Copyright 2021 The Prometheus Authors -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -//go:build !integration - -package exporter - -import ( - "github.com/prometheus/client_golang/prometheus" - dto "github.com/prometheus/client_model/go" - . "gopkg.in/check.v1" -) - -type PgSettingSuite struct{} - -var _ = Suite(&PgSettingSuite{}) - -var fixtures = []fixture{ - { - p: pgSetting{ - name: "seconds_fixture_metric", - setting: "5", - unit: "s", - shortDesc: "Foo foo foo", - vartype: "integer", - }, - n: normalised{ - val: 5, - unit: "seconds", - err: "", - }, - d: `Desc{fqName: "pg_settings_seconds_fixture_metric_seconds", help: "Server Parameter: seconds_fixture_metric [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, - v: 5, - }, - { - p: pgSetting{ - name: "milliseconds_fixture_metric", - setting: "5000", - unit: "ms", - shortDesc: "Foo foo foo", - vartype: "integer", - }, - n: normalised{ - val: 5, - unit: "seconds", - err: "", - }, - d: `Desc{fqName: "pg_settings_milliseconds_fixture_metric_seconds", help: "Server Parameter: milliseconds_fixture_metric [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, - v: 5, - }, - { - p: pgSetting{ - name: "eight_kb_fixture_metric", - setting: "17", - unit: "8kB", - shortDesc: "Foo foo foo", - vartype: "integer", - }, - n: normalised{ - val: 139264, - unit: "bytes", - err: "", - }, - d: `Desc{fqName: "pg_settings_eight_kb_fixture_metric_bytes", help: "Server Parameter: eight_kb_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, - v: 139264, - }, - { - p: pgSetting{ - name: "16_kb_real_fixture_metric", - setting: "3.0", - unit: "16kB", - shortDesc: "Foo foo foo", - vartype: "real", - }, - n: normalised{ - val: 49152, - unit: "bytes", - err: "", - }, - d: `Desc{fqName: "pg_settings_16_kb_real_fixture_metric_bytes", help: "Server Parameter: 16_kb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, - v: 49152, - }, - { - p: pgSetting{ - name: "16_mb_real_fixture_metric", - setting: "3.0", - unit: "16MB", - shortDesc: "Foo foo foo", - vartype: "real", - }, - n: normalised{ - val: 5.0331648e+07, - unit: "bytes", - err: "", - }, - d: `Desc{fqName: "pg_settings_16_mb_real_fixture_metric_bytes", help: "Server Parameter: 16_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, - v: 5.0331648e+07, - }, - { - p: pgSetting{ - name: "32_mb_real_fixture_metric", - setting: "3.0", - unit: "32MB", - shortDesc: "Foo foo foo", - vartype: "real", - }, - n: normalised{ - val: 1.00663296e+08, - unit: "bytes", - err: "", - }, - d: `Desc{fqName: "pg_settings_32_mb_real_fixture_metric_bytes", help: "Server Parameter: 32_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, - v: 1.00663296e+08, - }, - { - p: pgSetting{ - name: "64_mb_real_fixture_metric", - setting: "3.0", - unit: "64MB", - shortDesc: "Foo foo foo", - vartype: "real", - }, - n: normalised{ - val: 2.01326592e+08, - unit: "bytes", - err: "", - }, - d: `Desc{fqName: "pg_settings_64_mb_real_fixture_metric_bytes", help: "Server Parameter: 64_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, - v: 2.01326592e+08, - }, - { - p: pgSetting{ - name: "bool_on_fixture_metric", - setting: "on", - unit: "", - shortDesc: "Foo foo foo", - vartype: "bool", - }, - n: normalised{ - val: 1, - unit: "", - err: "", - }, - d: `Desc{fqName: "pg_settings_bool_on_fixture_metric", help: "Server Parameter: bool_on_fixture_metric", constLabels: {}, variableLabels: {}}`, - v: 1, - }, - { - p: pgSetting{ - name: "bool_off_fixture_metric", - setting: "off", - unit: "", - shortDesc: "Foo foo foo", - vartype: "bool", - }, - n: normalised{ - val: 0, - unit: "", - err: "", - }, - d: `Desc{fqName: "pg_settings_bool_off_fixture_metric", help: "Server Parameter: bool_off_fixture_metric", constLabels: {}, variableLabels: {}}`, - v: 0, - }, - { - p: pgSetting{ - name: "special_minus_one_value", - setting: "-1", - unit: "d", - shortDesc: "foo foo foo", - vartype: "integer", - }, - n: normalised{ - val: -1, - unit: "seconds", - err: "", - }, - d: `Desc{fqName: "pg_settings_special_minus_one_value_seconds", help: "Server Parameter: special_minus_one_value [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, - v: -1, - }, - { - p: pgSetting{ - name: "rds.rds_superuser_reserved_connections", - setting: "2", - unit: "", - shortDesc: "Sets the number of connection slots reserved for rds_superusers.", - vartype: "integer", - }, - n: normalised{ - val: 2, - unit: "", - err: "", - }, - d: `Desc{fqName: "pg_settings_rds_rds_superuser_reserved_connections", help: "Server Parameter: rds.rds_superuser_reserved_connections", constLabels: {}, variableLabels: {}}`, - v: 2, - }, - { - p: pgSetting{ - name: "unknown_unit", - setting: "10", - unit: "nonexistent", - shortDesc: "foo foo foo", - vartype: "integer", - }, - n: normalised{ - val: 10, - unit: "", - err: `unknown unit for runtime variable: "nonexistent"`, - }, - }, -} - -func (s *PgSettingSuite) TestNormaliseUnit(c *C) { - for _, f := range fixtures { - switch f.p.vartype { - case "integer", "real": - val, unit, err := f.p.normaliseUnit() - - c.Check(val, Equals, f.n.val) - c.Check(unit, Equals, f.n.unit) - - if err == nil { - c.Check("", Equals, f.n.err) - } else { - c.Check(err.Error(), Equals, f.n.err) - } - } - } -} - -func (s *PgSettingSuite) TestMetric(c *C) { - for _, f := range fixtures { - if f.n.err != "" { - continue - } - d := &dto.Metric{} - m, err := f.p.metric(prometheus.Labels{}) - if err != nil { - c.Fatalf("Error creating metric: %v", err) - } - m.Write(d) // nolint: errcheck - - c.Check(m.Desc().String(), Equals, f.d) - c.Check(d.GetGauge().GetValue(), Equals, f.v) - } -} - -type normalised struct { - val float64 - unit string - err string -} - -type fixture struct { - p pgSetting - n normalised - d string - v float64 -} diff --git a/exporter/postgres_exporter.go b/exporter/postgres_exporter.go index ad7b48cd0..4c918d828 100644 --- a/exporter/postgres_exporter.go +++ b/exporter/postgres_exporter.go @@ -348,7 +348,7 @@ type Exporter struct { // only, since it just points to the global. builtinMetricMaps map[string]intermediateMetricMap - disableDefaultMetrics, disableSettingsMetrics, autoDiscoverDatabases bool + disableDefaultMetrics, autoDiscoverDatabases bool excludeDatabases []string includeDatabases []string @@ -379,13 +379,6 @@ func DisableDefaultMetrics(b bool) ExporterOpt { } } -// DisableSettingsMetrics configures pg_settings export. -func DisableSettingsMetrics(b bool) ExporterOpt { - return func(e *Exporter) { - e.disableSettingsMetrics = b - } -} - // AutoDiscoverDatabases allows scraping all databases on a database server. func AutoDiscoverDatabases(b bool) ExporterOpt { return func(e *Exporter) { @@ -530,13 +523,6 @@ func (e *Exporter) CloseServers() { e.servers.Close() } -func newDesc(subsystem, name, help string, labels prometheus.Labels) *prometheus.Desc { - return prometheus.NewDesc( - prometheus.BuildFQName(namespace, subsystem, name), - help, nil, labels, - ) -} - func checkPostgresVersion(db *sql.DB, server string, logger *slog.Logger) (semver.Version, string, error) { logger.Debug("Querying PostgreSQL version", "server", server) versionRow := db.QueryRow("SELECT version();") diff --git a/exporter/postgres_exporter_integration_test.go b/exporter/postgres_exporter_integration_test.go index a65dbdaab..21a883bd4 100644 --- a/exporter/postgres_exporter_integration_test.go +++ b/exporter/postgres_exporter_integration_test.go @@ -70,12 +70,6 @@ func (s *IntegrationSuite) TestAllNamespacesReturnResults(c *C) { err = s.e.checkMapVersions(ch, server) c.Assert(err, IsNil) - err = querySettings(ch, server) - if !c.Check(err, Equals, nil) { - fmt.Println("## ERRORS FOUND") - fmt.Println(err) - } - // This should never happen in our test cases. errMap := queryNamespaceMappings(ch, server) if !c.Check(len(errMap), Equals, 0) { diff --git a/exporter/server.go b/exporter/server.go index 2aa2c6d67..25d8b3185 100644 --- a/exporter/server.go +++ b/exporter/server.go @@ -120,19 +120,12 @@ func (s *Server) String() string { } // Scrape loads metrics. -func (s *Server) Scrape(ch chan<- prometheus.Metric, disableSettingsMetrics bool) error { +func (s *Server) Scrape(ch chan<- prometheus.Metric) error { s.mappingMtx.RLock() defer s.mappingMtx.RUnlock() var err error - if !disableSettingsMetrics && s.master { - if err = querySettings(ch, s); err != nil { - err = fmt.Errorf("error retrieving settings: %s", err) - return err - } - } - errMap := queryNamespaceMappings(ch, s) if len(errMap) == 0 { return nil From ac49202029a99c48c2e33175e1dab35fa754a69b Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Tue, 26 May 2026 16:14:48 +0200 Subject: [PATCH 110/113] Update common Prometheus files (#1318) Signed-off-by: prombot --- .github/workflows/govulncheck.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/govulncheck.yml b/.github/workflows/govulncheck.yml index 1e13fbd99..adc84d6c5 100644 --- a/.github/workflows/govulncheck.yml +++ b/.github/workflows/govulncheck.yml @@ -21,4 +21,4 @@ jobs: name: Run govulncheck steps: - id: govulncheck - uses: golang/govulncheck-action@31f7c5463448f83528bd771c2d978d940080c9fd # v1.0.4-unreleased + uses: golang/govulncheck-action@b625fbe08f3bccbe446d94fbf87fcc875a4f50ee # v1.0.4 From 77cca15baaa035cafda7a9ec56fc907b7f28ce7a Mon Sep 17 00:00:00 2001 From: Julien <291750+roidelapluie@users.noreply.github.com> Date: Thu, 28 May 2026 15:12:12 +0200 Subject: [PATCH 111/113] ci: use github.token instead of PROMBOT_GITHUB_TOKEN in publish_release (#1320) Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .github/workflows/ci.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 06ad115f4..5e3b76375 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,6 +55,8 @@ jobs: publish_release: name: Publish release artefacts runs-on: ubuntu-latest + permissions: + contents: write needs: [test_go, build] if: | (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) @@ -66,4 +68,4 @@ jobs: docker_hub_password: ${{ secrets.docker_hub_password }} quay_io_organization: prometheuscommunity quay_io_password: ${{ secrets.quay_io_password }} - github_token: ${{ secrets.PROMBOT_GITHUB_TOKEN }} + github_token: ${{ github.token }} From 3238ccabb2be4b126a05a52f4757ff235496502d Mon Sep 17 00:00:00 2001 From: Julien <291750+roidelapluie@users.noreply.github.com> Date: Sat, 30 May 2026 12:39:37 +0200 Subject: [PATCH 112/113] ci: push to GHCR, harden checkouts, bump promci to v0.8.2 (#1322) - Add packages: write permission and ghcr_io_password to publish_main and publish_release jobs so images are pushed to GHCR - Drop the standalone actions/checkout step before promci build/publish steps (promci v0.8.2 performs its own checkout) - Bump promci composite actions to v0.8.2 (d9d4f5688814f0b77bf003d07fb8c00507390634) - Add persist-credentials: false to kept checkouts in test_go and integration_tests jobs Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .github/workflows/ci.yml | 16 ++++++++++------ .github/workflows/integration.yml | 2 ++ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5e3b76375..791389600 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,6 +19,8 @@ jobs: image: quay.io/prometheus/golang-builder:1.26-base steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1 @@ -29,8 +31,7 @@ jobs: matrix: thread: [ 0, 1, 2, 3] steps: - - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/build@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 + - uses: prometheus/promci/build@d9d4f5688814f0b77bf003d07fb8c00507390634 # v0.8.2 with: parallelism: 4 thread: ${{ matrix.thread }} @@ -38,17 +39,19 @@ jobs: publish_main: name: Publish main branch artifacts runs-on: ubuntu-latest + permissions: + packages: write needs: [test_go, build] if: | (github.event_name == 'push' && github.event.ref == 'refs/heads/main') || (github.event_name == 'push' && github.event.ref == 'refs/heads/master') steps: - - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_main@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 + - uses: prometheus/promci/publish_main@d9d4f5688814f0b77bf003d07fb8c00507390634 # v0.8.2 with: docker_hub_organization: prometheuscommunity docker_hub_password: ${{ secrets.docker_hub_password }} + ghcr_io_password: ${{ github.token }} quay_io_organization: prometheuscommunity quay_io_password: ${{ secrets.quay_io_password }} @@ -57,15 +60,16 @@ jobs: runs-on: ubuntu-latest permissions: contents: write + packages: write needs: [test_go, build] if: | (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) steps: - - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: prometheus/promci/publish_release@9c86752f3395e08c57719af549cc455d8e2c2514 # v0.7.0 + - uses: prometheus/promci/publish_release@d9d4f5688814f0b77bf003d07fb8c00507390634 # v0.8.2 with: docker_hub_organization: prometheuscommunity docker_hub_password: ${{ secrets.docker_hub_password }} + ghcr_io_password: ${{ github.token }} quay_io_organization: prometheuscommunity quay_io_password: ${{ secrets.quay_io_password }} github_token: ${{ github.token }} diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml index 8c2c65522..f864cf853 100644 --- a/.github/workflows/integration.yml +++ b/.github/workflows/integration.yml @@ -44,6 +44,8 @@ jobs: GOOPTS: '-v -tags integration' steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false - uses: prometheus/promci-setup@5af30ba8c199a91d6c04ebdc3c48e630e355f62d # v0.1.0 - run: make build - run: make test From 6c3e586d90834b6554d538694a3ec791706db8cb Mon Sep 17 00:00:00 2001 From: PrometheusBot Date: Sun, 31 May 2026 10:59:55 +0200 Subject: [PATCH 113/113] Update common Prometheus files (#1323) Signed-off-by: prombot --- .github/workflows/govulncheck.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/govulncheck.yml b/.github/workflows/govulncheck.yml index adc84d6c5..4ca92d302 100644 --- a/.github/workflows/govulncheck.yml +++ b/.github/workflows/govulncheck.yml @@ -22,3 +22,5 @@ jobs: steps: - id: govulncheck uses: golang/govulncheck-action@b625fbe08f3bccbe446d94fbf87fcc875a4f50ee # v1.0.4 + env: + GOOS: ${{ contains(github.repository, 'windows_exporter') && 'windows' || '' }}