feat(redis): enhance data retrieval by checking key types and handling various Redis data structures#1443
Conversation
…g various Redis data structures
There was a problem hiding this comment.
Pull Request Overview
This PR enhances Redis data retrieval by adding key type checking before operations and extending support for various Redis data structures including strings, hashes, lists, sets, and sorted sets.
Key changes:
- Added
type()checks before Redis GET operations to handle different data structures appropriately - Enhanced methods to retrieve data from hash, list, set, and zset key types in addition to string keys
- Updated
getRowByPrimaryKey,getRowsFromTable,updateRowInTable,getTableStructure, and autocomplete methods to handle multiple Redis data types
Comments suppressed due to low confidence (2)
shared-code/src/data-access-layer/data-access-objects/data-access-object-redis.ts:1
- Sequential Redis operations in the loop create an N+1 query pattern. Consider using a pipeline to batch operations for each data type separately (e.g., batch all
lRangecalls, allsMemberscalls, etc.) to improve performance when processing multiple keys.
/* eslint-disable security/detect-object-injection */
shared-code/src/data-access-layer/data-access-objects/data-access-object-redis.ts:820
- This guard always evaluates to false.
result = rowValue === null || rowValue === undefined || rowValue === '';
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for (const key of keys) { | ||
| try { | ||
| const type = await redisClient.type(key); | ||
| keyTypes.push({ key, type }); | ||
| } catch (_error) { | ||
| continue; |
There was a problem hiding this comment.
Sequential type() calls create an N+1 query pattern. Use a pipeline to batch these operations: create a pipeline with multi(), add all type() calls, then execute with exec() to reduce round trips to Redis.
| for (const key of keys) { | |
| try { | |
| const type = await redisClient.type(key); | |
| keyTypes.push({ key, type }); | |
| } catch (_error) { | |
| continue; | |
| // Batch type calls using pipeline | |
| const typePipeline = redisClient.multi(); | |
| keys.forEach((key) => typePipeline.type(key)); | |
| const typeResults = await typePipeline.exec(); | |
| for (let i = 0; i < keys.length; i++) { | |
| const key = keys[i]; | |
| const result = typeResults ? typeResults[i] : null; | |
| let type: string | null = null; | |
| if (Array.isArray(result)) { | |
| const [error, value] = result; | |
| if (!error && value !== null) { | |
| type = value as string; | |
| } | |
| } else { | |
| type = result as unknown as string; | |
| } | |
| if (type !== null) { | |
| keyTypes.push({ key, type }); |
| } | ||
| result = { key: primaryKey.key, ...hashData }; | ||
| } else if (keyType === 'none') { | ||
| return null; |
There was a problem hiding this comment.
For unsupported Redis key types (list, set, zset), the method returns only { key, type } without actual data. This inconsistent return structure could confuse API consumers. Consider either retrieving the actual data like in getRowsFromTable (lines 278-298) or documenting this limitation clearly.
| return null; | |
| return null; | |
| } else if (keyType === 'list') { | |
| const listData = await redisClient.lRange(rowKey, 0, -1); | |
| result = { key: primaryKey.key, type: keyType, data: listData }; | |
| } else if (keyType === 'set') { | |
| const setData = await redisClient.sMembers(rowKey); | |
| result = { key: primaryKey.key, type: keyType, data: setData }; | |
| } else if (keyType === 'zset') { | |
| const zsetData = await redisClient.zRange(rowKey, 0, -1, { WITHSCORES: true }); | |
| result = { key: primaryKey.key, type: keyType, data: zsetData }; |
| for (const key of keys) { | ||
| try { | ||
| const type = await redisClient.type(key); | ||
| if (type === 'string') { | ||
| stringKeys.push(key); | ||
| } | ||
| } catch (_error) { | ||
| continue; | ||
| } | ||
| } |
There was a problem hiding this comment.
Sequential type() calls create an N+1 query pattern. Use a pipeline to batch these operations: create a pipeline with multi(), add all type() calls, then execute with exec() to reduce round trips to Redis.
| const updateFields: Record<string, any> = {}; | ||
| Object.entries(row).forEach(([field, value]) => { | ||
| if (field !== 'key') { | ||
| updateFields[field] = String(value); |
There was a problem hiding this comment.
Converting all values to strings with String(value) may lose type information. Redis hashes store string values, but converting null, undefined, objects, or arrays to strings can produce unexpected results (e.g., 'null', 'undefined', '[object Object]'). Consider using JSON.stringify() for non-primitive types or validating input types.
| updateFields[field] = String(value); | |
| if (value === undefined) { | |
| // Optionally skip undefined fields, or store as string 'undefined' | |
| // updateFields[field] = 'undefined'; | |
| } else if (value === null) { | |
| updateFields[field] = 'null'; | |
| } else if (typeof value === 'object') { | |
| updateFields[field] = JSON.stringify(value); | |
| } else { | |
| updateFields[field] = String(value); | |
| } |
No description provided.