Skip to content

implement pg_proc table#2902

Open
jennifersp wants to merge 1 commit into
mainfrom
jennifer/proc
Open

implement pg_proc table#2902
jennifersp wants to merge 1 commit into
mainfrom
jennifer/proc

Conversation

@jennifersp

Copy link
Copy Markdown
Contributor

No description provided.

@jennifersp jennifersp requested a review from Hydrocharged July 7, 2026 21:01
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor
Main PR
covering_index_scan_postgres 2447.52/s 2458.46/s +0.4%
groupby_scan_postgres 170.17/s 170.73/s +0.3%
index_join_postgres 871.06/s 868.63/s -0.3%
index_join_scan_postgres 1103.80/s 1093.52/s -1.0%
index_scan_postgres 32.96/s 33.25/s +0.8%
oltp_delete_insert_postgres 717.05/s ${\color{red}170.57/s}$ ${\color{red}-76.3\%}$
oltp_insert 643.58/s ${\color{red}168.63/s}$ ${\color{red}-73.8\%}$
oltp_point_select 3841.96/s 3815.73/s -0.7%
oltp_read_only 3934.87/s 3878.75/s -1.5%
oltp_read_write 2424.51/s 2535.27/s +4.5%
oltp_update_index 669.94/s ${\color{red}502.97/s}$ ${\color{red}-25.0\%}$
oltp_update_non_index 741.61/s ${\color{red}394.80/s}$ ${\color{red}-46.8\%}$
oltp_write_only 1906.10/s 1785.24/s -6.4%
select_random_points 2388.46/s 2478.14/s +3.7%
select_random_ranges 1396.24/s 1419.77/s +1.6%
table_scan_postgres 30.80/s 31.26/s +1.4%
types_delete_insert_postgres 823.36/s ${\color{red}565.15/s}$ ${\color{red}-31.4\%}$
types_table_scan_postgres 13.22/s 13.47/s +1.8%

@itoqa

itoqa Bot commented Jul 7, 2026

Copy link
Copy Markdown

Ito QA test results
Commit: cc4f52e: 10 test cases ran, 2 failed ❌, 8 passed ✅.

Summary

This run exercises database routine metadata and definition lookup behavior across normal catalog introspection, argument-shape edge cases, and function-body merge handling for different quoting styles. Core routine discovery and definition round-trips look healthy, but security-boundary and metadata-consistency checks exposed important gaps.

Not safe to merge yet — this PR has attributable failures that include a high-severity authorization exposure plus an additional medium-severity catalog correctness defect, indicating real user-facing risk beyond minor edge noise. Because these are tied to this branch and affect both access control and metadata reliability, they are merge blockers rather than follow-up-only caveats.

Tests run by Ito

View full run

Result Severity Type Description
High severity Lookup The expected visibility boundary is inconsistent: pg_proc enforces permission checks, but pg_get_functiondef bypasses that boundary and returns full routine definitions to low-privilege users.
Medium severity Catalog proargnames includes duplicated names for named function signatures, so the array cardinality no longer matches the routine argument count.
Catalog The catalog query returned both created routines with the expected kind values: alt_func1 as a function and ptest5 as a procedure.
Catalog The pg_catalog.pg_proc row for procedure ptest5 correctly reports pronargs=3, pronargdefaults=1, proargtypes=23 25 23, and proargnames={a,b,c}, matching the declared signature with one defaulted input parameter.
Iteration Procedure-only verification passed: pg_proc returned ptest5 with prokind=p, OID lookup succeeded, and pg_get_functiondef returned the full procedure definition in the current build.
Lookup pg_get_functiondef resolved procedure OID 1886569565 to the expected CREATE OR REPLACE PROCEDURE definition for ptest5, including the expected SQL body with both INSERT statements.
Lookup The sampled function and procedure OIDs from pg_proc each resolved to their own expected definitions in pg_get_functiondef, with no cross-entity body mismatch.
Parsing Go test TestParsingFunctionDefinitionBody/PARSING-1 passed, confirming single-quoted AS function definitions are merged by inner-body content while preserving the surrounding DDL wrapper.
Parsing Go test TestParsingFunctionDefinitionBody/PARSING-2 passed, confirming $$-quoted function definitions are resolved by inner-body content while preserving the surrounding function wrapper and signature.
Parsing Legacy single-quoted function bodies with embedded doubled-quote patterns were merged at inner-body granularity and preserved the surrounding DDL wrapper.

Tip

Reply with @itoqa to send us feedback on this test run.

if f.Item.ParameterDefaults[i] != "" {
nArgDefs += 1
}
if f.Item.ParameterNames[i] != "" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

View All Evidence

Medium severity pg_proc proargnames duplicates named arguments

What failed: proargnames includes duplicated names for named function signatures, so the array cardinality no longer matches the routine argument count.

Impact · Steps · Stub / mock · Analysis · Why this is likely a bug
  • Severity: Medium Medium severity
  • Impact: Catalog consumers that trust proargnames length can mis-parse routine signatures and display incorrect argument metadata. A workaround exists by truncating to pronargs, but this is incompatible with expected PostgreSQL-style catalog shape.
  • Steps to Reproduce:
    1. Create a named function such as in_only_func(x int, y text).
    2. Query SELECT pronargs, proargnames FROM pg_catalog.pg_proc WHERE proname = 'in_only_func';.
    3. Observe pronargs = 2 while proargnames contains four values ({x,x,y,y}) instead of two.
    4. Compare with unnamed signatures to confirm the issue is specific to named argument handling.
  • Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code Analysis: In cachePgProcs, the function-path loop appends each non-empty parameter name twice: once inside if f.Item.ParameterNames[i] != "" { names = append(names, ...) } and again unconditionally via names = append(names, ...). Because this logic is in the catalog row builder used for pg_proc, named function rows are emitted with duplicated proargnames. The smallest practical fix is to remove the extra append and keep exactly one append per parameter name while preserving the existing null-vs-empty behavior.
  • Why this is likely a bug: The returned metadata violates an expected internal invariant (array_length(proargnames) should align with pronargs for named signatures) and directly reflects a deterministic duplication path in production code. This is user-visible via catalog introspection queries and is not caused by test setup or mocks.
Relevant code

server/tables/pgcatalog/pg_proc.go:119-127

if f.Item.ParameterNames[i] != "" {
    names = append(names, f.Item.ParameterNames[i])
}
types = append(types, typ.AsId())
if f.Item.ParameterNames[i] != "" {
    hasNonEmtpyArgName = true
}
names = append(names, f.Item.ParameterNames[i])
Evidence Package
Copy prompt for an agent
Ito QA identified the following failure during automated PR testing. Please investigate and propose a fix.

**Medium severity — pg_proc proargnames duplicates named arguments**

**What failed:** `proargnames` includes duplicated names for named function signatures, so the array cardinality no longer matches the routine argument count.

- **Impact:** Catalog consumers that trust `proargnames` length can mis-parse routine signatures and display incorrect argument metadata. A workaround exists by truncating to `pronargs`, but this is incompatible with expected PostgreSQL-style catalog shape.
- **Steps to reproduce:**
  1. Create a named function such as `in_only_func(x int, y text)`.
  2. Query `SELECT pronargs, proargnames FROM pg_catalog.pg_proc WHERE proname = 'in_only_func';`.
  3. Observe `pronargs = 2` while `proargnames` contains four values (`{x,x,y,y}`) instead of two.
  4. Compare with unnamed signatures to confirm the issue is specific to named argument handling.
- **Stub / mock content:** No stubs, mocks, or bypasses were applied for this test in the recorded run.
- **Code analysis:** In `cachePgProcs`, the function-path loop appends each non-empty parameter name twice: once inside `if f.Item.ParameterNames[i] != "" { names = append(names, ...) }` and again unconditionally via `names = append(names, ...)`. Because this logic is in the catalog row builder used for `pg_proc`, named function rows are emitted with duplicated `proargnames`. The smallest practical fix is to remove the extra append and keep exactly one append per parameter name while preserving the existing null-vs-empty behavior.
- **Why this is likely a bug:** The returned metadata violates an expected internal invariant (`array_length(proargnames)` should align with `pronargs` for named signatures) and directly reflects a deterministic duplication path in production code. This is user-visible via catalog introspection queries and is not caused by test setup or mocks.

**Relevant code:**

`server/tables/pgcatalog/pg_proc.go:119-127`

~~~go
if f.Item.ParameterNames[i] != "" {
    names = append(names, f.Item.ParameterNames[i])
}
types = append(types, typ.AsId())
if f.Item.ParameterNames[i] != "" {
    hasNonEmtpyArgName = true
}
names = append(names, f.Item.ParameterNames[i])
~~~

@@ -34,7 +36,21 @@ var pg_get_functiondef_oid = framework.Function1{
IsNonDeterministic: true,
Strict: true,
Callable: func(ctx *sql.Context, _ [2]*pgtypes.DoltgresType, val any) (any, error) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

View All Evidence

High severity Low-privilege routine source disclosure

What failed: The expected visibility boundary is inconsistent: pg_proc enforces permission checks, but pg_get_functiondef bypasses that boundary and returns full routine definitions to low-privilege users.

Impact · Steps · Stub / mock · Analysis · Why this is likely a bug
  • Severity: High High severity
  • Impact: Low-privilege users can read full function and procedure definitions they should not be able to access, exposing protected database logic and potentially sensitive implementation details.
  • Steps to Reproduce:
    1. Create a low-privilege login role with CONNECT on the database and USAGE on schema public.
    2. Revoke SELECT on pg_catalog.pg_proc from that role and confirm direct pg_proc reads are denied.
    3. As the same role, call pg_get_functiondef with known routine OIDs for a function and a procedure.
    4. Observe that full CREATE FUNCTION/CREATE PROCEDURE definitions are returned instead of an authorization error or redacted result.
  • Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code Analysis: In server/functions/pg_get_functiondef.go, the PR replaced the previous stub with direct lookup logic that calls RunCallback and copies function/procedure Item.Definition into the response. In server/functions/iterate.go, RunCallback -> runFunction/runProcedure iterates catalog collections and returns matched objects without any privilege check for the caller before invoking callbacks. Together, this creates a direct metadata disclosure path independent of pg_proc table permissions. The smallest practical fix is to add an explicit authorization check in pg_get_functiondef (or inside RunCallback for function/procedure sections) and return an error or empty result when the caller lacks routine-definition visibility.
  • Why this is likely a bug: The runtime evidence shows permission denial on pg_proc but successful routine-definition retrieval through pg_get_functiondef for the same low-privilege role, and the code path confirms no authorization gate before returning definition text. That mismatch indicates a real product authorization defect rather than test setup noise.
Relevant code

server/functions/pg_get_functiondef.go:38-54

Callable: func(ctx *sql.Context, _ [2]*pgtypes.DoltgresType, val any) (any, error) {
	oidVal := val.(id.Id)
	result := ""
	err := RunCallback(ctx, oidVal, Callbacks{
		Function: func(ctx *sql.Context, schema ItemSchema, function ItemFunction) (cont bool, err error) {
			result = function.Item.Definition
			return false, nil
		},
		Procedure: func(ctx *sql.Context, schema ItemSchema, procedure ItemProcedure) (cont bool, err error) {
			result = procedure.Item.Definition
			return false, nil
		},
	})
	if err != nil {
		return "", err
	}
	return result, nil
},

server/functions/iterate.go:824-838

func runProcedure(ctx *sql.Context, internalID id.Id, callbacks Callbacks, itemSchema ItemSchema) error {
	collection, err := core.GetProceduresCollectionFromContext(ctx, itemSchema.Item.Name())
	if err != nil {
		return err
	}
	return collection.IterateProcedures(ctx, func(p procedures.Procedure) (stop bool, err error) {
		if p.ID.AsId() == internalID {
			_, err = callbacks.Procedure(ctx, itemSchema, ItemProcedure{
				OID:  id.Procedure(internalID),
				Item: &p,
			})
			return true, err
		}
		return false, nil
	})
}
Evidence Package
Copy prompt for an agent
Ito QA identified the following failure during automated PR testing. Please investigate and propose a fix.

**High severity — Low-privilege routine source disclosure**

**What failed:** The expected visibility boundary is inconsistent: pg_proc enforces permission checks, but pg_get_functiondef bypasses that boundary and returns full routine definitions to low-privilege users.

- **Impact:** Low-privilege users can read full function and procedure definitions they should not be able to access, exposing protected database logic and potentially sensitive implementation details.
- **Steps to reproduce:**
  1. Create a low-privilege login role with CONNECT on the database and USAGE on schema public.
  2. Revoke SELECT on pg_catalog.pg_proc from that role and confirm direct pg_proc reads are denied.
  3. As the same role, call pg_get_functiondef with known routine OIDs for a function and a procedure.
  4. Observe that full CREATE FUNCTION/CREATE PROCEDURE definitions are returned instead of an authorization error or redacted result.
- **Stub / mock content:** No stubs, mocks, or bypasses were applied for this test in the recorded run.
- **Code analysis:** In server/functions/pg_get_functiondef.go, the PR replaced the previous stub with direct lookup logic that calls RunCallback and copies function/procedure Item.Definition into the response. In server/functions/iterate.go, RunCallback -> runFunction/runProcedure iterates catalog collections and returns matched objects without any privilege check for the caller before invoking callbacks. Together, this creates a direct metadata disclosure path independent of pg_proc table permissions. The smallest practical fix is to add an explicit authorization check in pg_get_functiondef (or inside RunCallback for function/procedure sections) and return an error or empty result when the caller lacks routine-definition visibility.
- **Why this is likely a bug:** The runtime evidence shows permission denial on pg_proc but successful routine-definition retrieval through pg_get_functiondef for the same low-privilege role, and the code path confirms no authorization gate before returning definition text. That mismatch indicates a real product authorization defect rather than test setup noise.

**Relevant code:**

`server/functions/pg_get_functiondef.go:38-54`

~~~go
Callable: func(ctx *sql.Context, _ [2]*pgtypes.DoltgresType, val any) (any, error) {
	oidVal := val.(id.Id)
	result := ""
	err := RunCallback(ctx, oidVal, Callbacks{
		Function: func(ctx *sql.Context, schema ItemSchema, function ItemFunction) (cont bool, err error) {
			result = function.Item.Definition
			return false, nil
		},
		Procedure: func(ctx *sql.Context, schema ItemSchema, procedure ItemProcedure) (cont bool, err error) {
			result = procedure.Item.Definition
			return false, nil
		},
	})
	if err != nil {
		return "", err
	}
	return result, nil
},
~~~

`server/functions/iterate.go:824-838`

~~~go
func runProcedure(ctx *sql.Context, internalID id.Id, callbacks Callbacks, itemSchema ItemSchema) error {
	collection, err := core.GetProceduresCollectionFromContext(ctx, itemSchema.Item.Name())
	if err != nil {
		return err
	}
	return collection.IterateProcedures(ctx, func(p procedures.Procedure) (stop bool, err error) {
		if p.ID.AsId() == internalID {
			_, err = callbacks.Procedure(ctx, itemSchema, ItemProcedure{
				OID:  id.Procedure(internalID),
				Item: &p,
			})
			return true, err
		}
		return false, nil
	})
}
~~~

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor
Main PR
Total 42090 42090
Successful 18275 18262
Failures 23815 23828
Partial Successes1 5335 5335
Main PR
Successful 43.4189% 43.3880%
Failures 56.5811% 56.6120%

${\color{red}Regressions (13)}$

opr_sanity

QUERY:          SELECT p1.oid, p1.proname, p2.oid, p2.proname
FROM pg_proc AS p1, pg_proc AS p2
WHERE p2.oid = p1.prosupport AND
    (p2.prorettype != 'internal'::regtype OR p2.proretset OR p2.pronargs != 1
     OR p2.proargtypes[0] != 'internal'::regtype);
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)

type_sanity

QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typinput = p1.oid AND NOT
    ((p1.pronargs = 1 AND p1.proargtypes[0] = 'cstring'::regtype) OR
     (p1.pronargs = 2 AND p1.proargtypes[0] = 'cstring'::regtype AND
      p1.proargtypes[1] = 'oid'::regtype) OR
     (p1.pronargs = 3 AND p1.proargtypes[0] = 'cstring'::regtype AND
      p1.proargtypes[1] = 'oid'::regtype AND
      p1.proargtypes[2] = 'int4'::regtype));
RECEIVED ERROR: operator does not exist: oidvector = regtype (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typinput = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typoutput = p1.oid AND NOT
    (p1.prorettype = 'cstring'::regtype AND NOT p1.proretset);
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typoutput = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typreceive = p1.oid AND NOT
    ((p1.pronargs = 1 AND p1.proargtypes[0] = 'internal'::regtype) OR
     (p1.pronargs = 2 AND p1.proargtypes[0] = 'internal'::regtype AND
      p1.proargtypes[1] = 'oid'::regtype) OR
     (p1.pronargs = 3 AND p1.proargtypes[0] = 'internal'::regtype AND
      p1.proargtypes[1] = 'oid'::regtype AND
      p1.proargtypes[2] = 'int4'::regtype));
RECEIVED ERROR: operator does not exist: oidvector = regtype (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typreceive = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typsend = p1.oid AND NOT
    (p1.prorettype = 'bytea'::regtype AND NOT p1.proretset);
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typsend = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typmodin = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typmodout = p1.oid AND NOT
    (p1.pronargs = 1 AND
     p1.proargtypes[0] = 'int4'::regtype AND
     p1.prorettype = 'cstring'::regtype AND NOT p1.proretset);
RECEIVED ERROR: operator does not exist: oidvector = regtype (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typmodout = p1.oid AND p1.provolatile NOT IN ('i', 's');
RECEIVED ERROR: incompatible conversion to SQL type: '{Function:["public","table_fail","#�

pg_cataloganyelement"]}'->longtext (errno 1105) (sqlstate HY000)
QUERY:          SELECT t1.oid, t1.typname, p1.oid, p1.proname
FROM pg_type AS t1, pg_proc AS p1
WHERE t1.typanalyze = p1.oid AND NOT
    (p1.pronargs = 1 AND
     p1.proargtypes[0] = 'internal'::regtype AND
     p1.prorettype = 'bool'::regtype AND NOT p1.proretset);
RECEIVED ERROR: operator does not exist: oidvector = regtype (errno 1105) (sqlstate HY000)

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@Hydrocharged Hydrocharged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Strange that we have such a large regression, I'm not seeing anything that would point to that being the case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants