Does ClickGraph support PB-scale data queries and subgraph matching?

[peterhunter99001-cyber]

Great work!

I’m looking into ClickGraph and have two questions about its capabilities:

Does the graph library support querying PB-scale datasets? If so, are there any notes on performance (e.g., stability, response times) with such large volumes of data?
Does it include subgraph matching functionality? If yes, is there any documentation or examples for how to use it (e.g., query syntax, parameters)?

Any insights or pointers to relevant docs would be really helpful. Thanks!

[genezhang]

Thanks @peterhunter99001-cyber for your great questions.

Does the graph library support querying PB-scale datasets? If so, are there any notes on performance (e.g., stability, response times) with such large volumes of data?

ClickGraph is a stateless query engine that translates Cypher to ClickHouse SQL, so it inherits ClickHouse's scalability characteristics. While we haven't tested at PB-scale yet, ClickHouse with right infrastructure could support PB-scale, with right expectation of various query performance levels.

Does it include subgraph matching functionality? If yes, is there any documentation or examples for how to use it (e.g., query syntax, parameters)?

Not sure exactly what subgraph matching means - I assume Cypher MATCH queries. If not, could you elaborate it with a use case?
Supported Cypher queries are documented under docs/ directory, and draft wiki content has quite comprehensive syntax and examples to be reviewed and published.

[peterhunter99001-cyber]

this is a case for subgraph matching: https://docs.nebula-graph.io/2.6.1/3.ngql-guide/16.subgraph-and-path/1.get-subgraph/

[genezhang]

Thanks @peterhunter99001-cyber. We studied Nebula's GET SUBGRAPH documentation. Its key features are:

• N-hop neighborhood extraction - Get all vertices/edges within N steps from starting vertices
• Returns structured subgraph - Both vertices AND edges in the result
• Direction control - IN/OUT/BOTH for edge traversal
• Property inclusion - WITH PROP shows properties, otherwise hidden
• Completeness guarantee - Makes extra hop to show complete subgraph structure

Example:

-- Get 1-hop subgraph from player101
GET SUBGRAPH 1 STEPS FROM "player101" YIELD VERTICES AS nodes, EDGES AS relationships;

-- Result: Two arrays
nodes: [("player101"), ("team204"), ("player100"), ("player102")]
edges: [[:serve "player101"->"team204"], [:follow "player101"->"player100"], ...]

ClickGraph has partial support already (need verification/extension):

1. Variable-length paths - MATCH (a)-[*1..N]->(b) gets N-hop traversals, you can specify the Node Labels and Edge Types.
2. Multi-hop patterns - Can traverse multiple steps
3. Direction control - ->, <-, - for IN/OUT/BOTH
4. Property filtering - WHERE clauses work

For example:

-- Equivalent query (returns rows, not structured subgraph)
MATCH p = (start)-[*1..2]-(connected)
WHERE start.user_id = 'player101'
RETURN nodes(p), relationships(p)

What We're Missing ❌:

1. Structured subgraph return format - We return rows, not "vertices array + edges array"
2. Completeness guarantee - We don't do the extra hop to show complete subgraph, this can be mitigated by increasing the step by 1 in the query (?)
3. GET SUBGRAPH syntax - Different from Cypher MATCH

So the questions to you are:

1. what return format do you want in your use case?
2. is something like the above Cypher Match query syntax acceptable for you?

[peterhunter99001-cyber]

Thanks!

[peterhunter99001-cyber]

My use case is as follows: having prior knowledge of the labels (and related properties) of multiple nodes as well as the relationships between them, this type of result is highly effective for detecting certain patterns. By querying the graph database with this structure, I expect to retrieve subgraphs that match it, and the returned result should be a list of list of triples (head, relationship, tail). A list of triples (head, relationship, tail) represents a matched subgraph.

[peterhunter99001-cyber]

I think the challenge of this matching lies in the performance issue under massive data volumes, and it’s very likely that no results will be returned.

[genezhang]

Thanks @peterhunter99001-cyber for the more details. If you start with a small list of starting nodes, and expand a few steps for the subgraph, I believe it's quite doable as the generated SQL query will be run against ClickHouse that will efficiently evaluate it just as a normal SQL (we could potentially generate more optimized SQL depending on the pattern).
I'd like to design the syntax for your use case. What properties do you need to return in triples (head, relationship, tail) for each of them. We will have id's for head and tail with their labels, but id is not required for a relationship in our current support, we can have relationship types and maybe additional properties you want.

[peterhunter99001-cyber]

Thank you for your thorough research and feedback. I'd like to clarify that the input is a subgraph pattern (input: abstract subgraph structure), and its output is the actual values matching this subgraph pattern (output: specific matched subgraph structures). Could the properties to be returned be specified at query time? To be more specific, we hope to specify the properties of nodes and edges when executing the query.

[genezhang]

Hi @peterhunter99001-cyber , I assume MATCH can be used to specify your subgraph pattern. The remain design point is the RETURN syntax:

Could the properties to be returned be specified at query time? To be more specific, we hope to specify the properties of nodes and edges when executing the query.

this is what we need to define, and we can specify any properties you want to collect into rows, I believe, using "standard" Cypher syntax as much as possible. I see this case is linked to another issue on the polymorphic relationship table. Right?

[peterhunter99001-cyber]

Yes, this issue is related to the polymorphic relationship table problem.

[genezhang]

@peterhunter99001-cyber with release v0.5.2, we've added support for the polymorphic edge tables, as well as subgraph equivalent Cypher features. Please see polymorphic edge table and subgraph extraction for details. Please try it out and let us know. Thanks.

[peterhunter99001-cyber]

i will test it