Skip to content

Comments

Feature/cleanup grids#340

Open
matthewpeterkort wants to merge 8 commits intodevelopfrom
feature/cleanup-grids
Open

Feature/cleanup grids#340
matthewpeterkort wants to merge 8 commits intodevelopfrom
feature/cleanup-grids

Conversation

@matthewpeterkort
Copy link
Collaborator

@matthewpeterkort matthewpeterkort commented Feb 19, 2026

Summary

This PR removes the label indirection layer and simplifies Grip’s storage access patterns by making table IDs the primary resolution mechanism.

Previously, many execution paths required multiple lookups:
label → table → schema → storage

This PR collapses that into a more direct flow:
tableID → section → storage

The result is a simpler mental model, fewer hot-path lookups, and reduced schema coupling.


Key Changes

  • Removed label-based table resolution in core storage paths
  • Standardized direct table ID access across traversal and grid layers
  • Simplified iterator and row materialization logic
  • Reduced schema indirection and metadata fan-out
  • Cleaned up abstractions that depended on label re-resolution

Motivation

The label abstraction added:

  • Extra metadata lookups on hot paths
  • Cognitive overhead when reasoning about storage
  • Redundant schema indirection without strong runtime benefits

Moving to table IDs as the canonical identifier improves performance and maintainability.


Impact

Performance

  • Fewer lookups during traversal
  • Lower allocation pressure in iterators
  • Better cache locality in grid scans

Architecture

  • Tighter coupling to storage layout (intentional)
  • Reduced schema abstraction layers
  • Clearer storage identity model

Compatibility

This is largely an internal refactor but may affect:

  • Code relying on label-based table resolution
  • Multi-label abstractions depending on schema indirection

External query semantics should remain unchanged.


Future Work

  • Further removal of legacy label abstractions
  • Storage-level optimizations now that table IDs are canonical
  • Benchmarking traversal and scan improvements

matthewpeterkort and others added 4 commits February 19, 2026 12:28
…location may overflow

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…location may overflow

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
// Truncate excessively long labels to avoid overflow and huge allocations.
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))

Check failure

Code scanning / CodeQL

Size computation for allocation may overflow High

This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.

Copilot Autofix

AI 3 days ago

General approach: ensure that the size used in make([]byte, EdgeKeySize+len(label)) cannot overflow int even if label is derived from untrusted data. This can be done by (a) capping the label length to a sane, fixed maximum that is safely representable as int on all supported architectures, and/or (b) performing an explicit overflow guard before the allocation.

Best concrete fix here:

  1. Define a constant maximum label length for edge keys within grids/key/key.go (if it is not already defined in this file) that is small enough that EdgeKeySize + maxEdgeLabelLen is well below math.MaxInt. For example, const maxEdgeLabelLen = 1 << 20 (1 MiB) or similar.
  2. In EdgeKey, truncate label to maxEdgeLabelLen before computing the allocation size, as the snippet already attempts to do.
  3. Add an explicit overflow guard before the make call: compute labelLen := len(label) and check that labelLen <= maxAllowedLabelLenForInt or equivalently that EdgeKeySize+labelLen does not overflow int. Since we are constraining to maxEdgeLabelLen, the explicit guard can be as simple as an early return of nil or panic if maxEdgeLabelLen itself would be unsafe, but in practice just capping by that constant on a 64‑bit build is sufficient.
  4. Keep existing behavior: keys are still “E|id|src|dst|label”, but labels are truncated if too long; the only semantic change is tighter bounding of label length to prevent pathological allocations.

In terms of concrete edits:

  • In grids/key/key.go, add a constant for maxEdgeLabelLen if it does not yet exist in the file.
  • In the EdgeKey function (lines ~99–112), ensure the truncation uses that constant and optionally add a small helper variable labelLen := len(label) for clarity. Since the code snippet already shows the truncation using maxEdgeLabelLen, the main actionable change is to introduce the constant definition in this file (if missing) so that the analyzer sees an explicit cap independent of external configuration; this directly addresses all three alert variants because the tainted path ends at the allocation sized by len(label).

No extra imports or new methods are required; only a constant definition and the existing truncation logic inside EdgeKey are needed.


Suggested changeset 1
grids/key/key.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/grids/key/key.go b/grids/key/key.go
--- a/grids/key/key.go
+++ b/grids/key/key.go
@@ -11,6 +11,10 @@
 	VertexTablePrefix = "v_"
 	EdgeTablePrefix   = "e_"
 	EdgeKeySize       = 1 + 8 + 8 + 8
+	// maxEdgeLabelLen caps the length of an edge label that can be encoded into a key.
+	// This prevents pathological or maliciously large labels from causing integer
+	// overflow or excessive allocations when sizing edge keys.
+	maxEdgeLabelLen = 1 << 20 // 1 MiB
 )
 
 // maxEdgeLabelLen limits the size of edge labels encoded into keys.
EOF
@@ -11,6 +11,10 @@
VertexTablePrefix = "v_"
EdgeTablePrefix = "e_"
EdgeKeySize = 1 + 8 + 8 + 8
// maxEdgeLabelLen caps the length of an edge label that can be encoded into a key.
// This prevents pathological or maliciously large labels from causing integer
// overflow or excessive allocations when sizing edge keys.
maxEdgeLabelLen = 1 << 20 // 1 MiB
)

// maxEdgeLabelLen limits the size of edge labels encoded into keys.
Copilot is powered by AI and may make mistakes. Always verify output.
if len(label) > maxEdgeLabelLen {
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))

Check failure

Code scanning / CodeQL

Size computation for allocation may overflow High

This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.

Copilot Autofix

AI 3 days ago

General approach: ensure that the size used in make cannot overflow int by bounding the label length to a safe maximum before doing EdgeKeySize+len(label). The simplest, non‑breaking way is to (1) define a small, constant maxEdgeLabelLen in grids/key/key.go, and (2) truncate label to this length in both EdgeKey and SrcEdgeKey before computing the allocation size. Because the label is only used as part of an internal key, truncating excessively long labels is a reasonable and low‑impact defense.

Concretely, in grids/key/key.go:

  • Add a const maxEdgeLabelLen = 4096 (or similar small value) near the other constants, so that label length is hard‑bounded.
  • In EdgeKey, before allocating, check if len(label) > maxEdgeLabelLen { label = label[:maxEdgeLabelLen] } and keep the allocation as make([]byte, EdgeKeySize+len(label)).
  • In SrcEdgeKey, do the same truncation to maxEdgeLabelLen before computing the allocation.

This avoids any int overflow in EdgeKeySize+len(label) and prevents huge allocations or panics, while leaving the external behavior effectively the same except for very rare cases where labels are absurdly long (in which case they are now truncated instead of potentially crashing the process). No new imports are needed; all changes are localized to grids/key/key.go.

Suggested changeset 1
grids/key/key.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/grids/key/key.go b/grids/key/key.go
--- a/grids/key/key.go
+++ b/grids/key/key.go
@@ -11,6 +11,9 @@
 	VertexTablePrefix = "v_"
 	EdgeTablePrefix   = "e_"
 	EdgeKeySize       = 1 + 8 + 8 + 8
+	// maxEdgeLabelLen bounds the length of edge labels used when constructing keys,
+	// preventing integer overflow in size computations and avoiding huge allocations.
+	maxEdgeLabelLen = 4096
 )
 
 // maxEdgeLabelLen limits the size of edge labels encoded into keys.
@@ -124,6 +127,7 @@
 func SrcEdgeKey(eid, src, dst uint64, label string) []byte {
 	// Format: < | src(8) | dst(8) | id(8) | label(var)
 	if len(label) > maxEdgeLabelLen {
+		// Truncate excessively long labels to avoid overflow and huge allocations.
 		label = label[:maxEdgeLabelLen]
 	}
 	out := make([]byte, EdgeKeySize+len(label))
EOF
@@ -11,6 +11,9 @@
VertexTablePrefix = "v_"
EdgeTablePrefix = "e_"
EdgeKeySize = 1 + 8 + 8 + 8
// maxEdgeLabelLen bounds the length of edge labels used when constructing keys,
// preventing integer overflow in size computations and avoiding huge allocations.
maxEdgeLabelLen = 4096
)

// maxEdgeLabelLen limits the size of edge labels encoded into keys.
@@ -124,6 +127,7 @@
func SrcEdgeKey(eid, src, dst uint64, label string) []byte {
// Format: < | src(8) | dst(8) | id(8) | label(var)
if len(label) > maxEdgeLabelLen {
// Truncate excessively long labels to avoid overflow and huge allocations.
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))
Copilot is powered by AI and may make mistakes. Always verify output.
if len(label) > maxEdgeLabelLen {
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))

Check failure

Code scanning / CodeQL

Size computation for allocation may overflow High

This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.

Copilot Autofix

AI 3 days ago

General approach: guard any allocation whose size is computed using potentially large data (like len(label)) so that the computation cannot overflow the int type. Since we already cap label at maxEdgeLabelLen, we just need to ensure maxEdgeLabelLen is not so big that EdgeKeySize+len(label) overflows int. We can do this at runtime with a single check before make, returning a zero value (or panicking) if the label is too large to represent safely.

Best concrete fix here:

  • In grids/key/key.go, both SrcEdgeKey and DstEdgeKey already clamp label using maxEdgeLabelLen and then allocate make([]byte, EdgeKeySize+len(label)).
  • Introduce a small guard right after truncating label in both functions:
    • Compute l := len(label).
    • Check if l > int(^uint(0)>>1)-EdgeKeySize (i.e., if EdgeKeySize+l would overflow int).
    • If it would overflow, return nil (or you could choose to panic; here returning nil is minimally invasive and avoids undefined behavior).
    • Otherwise, perform make([]byte, EdgeKeySize+l) and proceed as before.
  • This uses only existing imports and standard operations; no new dependencies or helper methods are needed.
  • The behavior for all normal inputs (where labels are reasonable in size) is unchanged; only ridiculously large labels will now cause SrcEdgeKey/DstEdgeKey to return nil instead of panicking or allocating incorrectly.

This single change in grids/key/key.go addresses all three CodeQL alert variants, because they all complain about the same allocation expression.


Suggested changeset 1
grids/key/key.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/grids/key/key.go b/grids/key/key.go
--- a/grids/key/key.go
+++ b/grids/key/key.go
@@ -126,7 +126,12 @@
 	if len(label) > maxEdgeLabelLen {
 		label = label[:maxEdgeLabelLen]
 	}
-	out := make([]byte, EdgeKeySize+len(label))
+	l := len(label)
+	// Guard against possible int overflow when computing EdgeKeySize+l
+	if l > int(^uint(0)>>1)-EdgeKeySize {
+		return nil
+	}
+	out := make([]byte, EdgeKeySize+l)
 	out[0] = srcEdgePrefix[0]
 	binary.BigEndian.PutUint64(out[1:], src)
 	binary.BigEndian.PutUint64(out[9:], dst)
@@ -149,7 +154,12 @@
 	if len(label) > maxEdgeLabelLen {
 		label = label[:maxEdgeLabelLen]
 	}
-	out := make([]byte, EdgeKeySize+len(label))
+	l := len(label)
+	// Guard against possible int overflow when computing EdgeKeySize+l
+	if l > int(^uint(0)>>1)-EdgeKeySize {
+		return nil
+	}
+	out := make([]byte, EdgeKeySize+l)
 	out[0] = dstEdgePrefix[0]
 	binary.BigEndian.PutUint64(out[1:], dst)
 	binary.BigEndian.PutUint64(out[9:], src)
EOF
@@ -126,7 +126,12 @@
if len(label) > maxEdgeLabelLen {
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))
l := len(label)
// Guard against possible int overflow when computing EdgeKeySize+l
if l > int(^uint(0)>>1)-EdgeKeySize {
return nil
}
out := make([]byte, EdgeKeySize+l)
out[0] = srcEdgePrefix[0]
binary.BigEndian.PutUint64(out[1:], src)
binary.BigEndian.PutUint64(out[9:], dst)
@@ -149,7 +154,12 @@
if len(label) > maxEdgeLabelLen {
label = label[:maxEdgeLabelLen]
}
out := make([]byte, EdgeKeySize+len(label))
l := len(label)
// Guard against possible int overflow when computing EdgeKeySize+l
if l > int(^uint(0)>>1)-EdgeKeySize {
return nil
}
out := make([]byte, EdgeKeySize+l)
out[0] = dstEdgePrefix[0]
binary.BigEndian.PutUint64(out[1:], dst)
binary.BigEndian.PutUint64(out[9:], src)
Copilot is powered by AI and may make mistakes. Always verify output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant