Skip to content

bugfix: set correct workspace size parameter in build_engines.sh - #7

Open
hebangwen wants to merge 1 commit into
OpenBMB:mainfrom
hebangwen:patch-1
Open

bugfix: set correct workspace size parameter in build_engines.sh#7
hebangwen wants to merge 1 commit into
OpenBMB:mainfrom
hebangwen:patch-1

Conversation

@hebangwen

Copy link
Copy Markdown

The memPoolSize parameter in trtexec uses M instead of MiB.

Logs from trtexec with different memPoolSize:

memPoolSize log siglip latency
workspace:4096MiB Memory Pools: workspace: 0.00390625 MiB 73.6ms
workspace:4096M Memory Pools: workspace: 4096 MiB 72.1ms

Help message from trtexec --help:

  --memPoolSize=poolspec             Specify the size constraints of the designated memory pool(s)
                                     Supports the following base-2 suffixes: B (Bytes), G (Gibibytes), K (Kibibytes), M (Mebibytes).
                                     If none of suffixes is appended, the defualt unit is in MiB.
                                     Note: Also accepts decimal sizes, e.g. 0.25M. Will be rounded down to the nearest integer bytes.
                                     In particular, for dlaSRAM the bytes will be rounded down to the nearest power of 2.
                                   Pool constraint: poolspec ::= poolfmt[","poolspec]
                                                      poolfmt ::= pool:size
                                                    pool ::= "workspace"|"dlaSRAM"|"dlaLocalDRAM"|"dlaGlobalDRAM"|"tacticSharedMem"

The `memPoolSize` parameter in trtexec uses `M` instead of `MiB`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant