Because it sets highWaterMark to 0 (though documented as 16):
|
createRowStream (sql, opts) { |
|
const pageOpts = Object.assign({}, opts, { |
|
paging: false, |
|
highWaterMark: 0, |
|
}) |
This makes sense for a page stream, because it means it won't make preemptive HTTP requests. But for
a row stream it means it'll push only one row (or "chunk") at a time, possibly hurting throughput:
|
// Push single rows |
|
while (this._buffer !== undefined && this._bufferIndex < this._buffer.length) { |
|
const chunk = this._buffer[this._bufferIndex++] |
|
|
|
if (!this.push(chunk)) { |
|
this._reading = false |
|
return |
|
} |
|
} |
I.e. the while loop above never loops. It might be easier to push everything at once (ignoring the return value of push). It's already occupying memory anyway.
Also, when the next readable-stream is out (with nodejs/node@1e0f331), I can maybe get rid of:
|
// See https://github.com/nodejs/node/issues/3203 |
|
if (this._reading) return |
|
else this._reading = true |
For now, I recommend to use createPageStream instead.
Because it sets
highWaterMarkto 0 (though documented as 16):lento/lib/client.js
Lines 176 to 180 in 8670652
This makes sense for a page stream, because it means it won't make preemptive HTTP requests. But for
a row stream it means it'll push only one row (or "chunk") at a time, possibly hurting throughput:
lento/lib/query-stream.js
Lines 68 to 76 in 8670652
I.e. the
whileloop above never loops. It might be easier to push everything at once (ignoring the return value ofpush). It's already occupying memory anyway.Also, when the next
readable-streamis out (with nodejs/node@1e0f331), I can maybe get rid of:lento/lib/query-stream.js
Lines 46 to 48 in 8670652
For now, I recommend to use
createPageStreaminstead.