feat: add ViT encoder ACL Graph capture for Qwen3-VL. by sunbaosong · Pull Request #1866 · xLLM-AI/xllm

sunbaosong · 2026-07-01T08:30:09Z

新增配置开关（2 个）

开关	类型	默认	说明
`--enable_encoder_graph`	bool	`false`	总开关：是否对 vision encoder 启用 ACL graph。关闭即全程 eager。
`--encoder_graph_budgets`	string	`"256,512"`	逗号分隔的 token 预算（捕获桶）。按 actual_tokens 选最小够用的桶；exact-budget 无 padding，否则 pad 到桶大小。

512 约束（澄清：非开关，不可配）

capture token 上限硬编码为 512（kEncoderGraphMaxReplayTokens），不是配置开关、不可调：

构造期 filter_budgets() 丢弃任何 >512 的 budget（打 LOG(WARNING)；若全部超限则再告警 "graph effectively disabled"）。
select_budget 对 actual_tokens > 512 自然返回 −1 → 回退 eager。
即无论 --encoder_graph_budgets 怎么配，>512 的图永远不会被捕获/服务。

性能测试数据

Δ% = eager−graph，正=graph 快

actual_tokens	1 卡 TP=1	2 卡 TP=2	4 卡 TP=4	8 卡 TP=8
256	-2.9%	+2.3%	+6.5%	+30.3%
484	-5.8%	+2.0%	+1.3%	+17.0%
512	-4.8%	-1.3%	-4.0%	+9.6%
1024	-4.0%	-1.9%	-4.9%	-3.0%
1764	-3.7%	-0.2%	-3.9%	-1.6%
2500	-3.9%	-1.1%	-2.8%	-0.4%
4096	-4.3%	-3.3%	-4.0%	-4.2%

gemini-code-assist

Code Review

This pull request introduces VisionEncoderAclGraphManager to support ACL graph execution for the vision encoder in Qwen3-VL, along with corresponding configuration options and integration into the VLM executor and model. The review feedback identifies several critical issues, including a redundant dynamic_cast, potential race conditions from incorrect stream synchronization during graph capture, and graph replay bugs caused by changing tensor memory addresses with set_data. Additionally, the reviewer recommends refactoring the hardcoded deepstack outputs to a dynamic vector and addresses multiple style guide violations regarding braces, final classes, relative includes, auto usage, constant argument annotations, and fixed-width integers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-01T08:35:33Z

+    auto* vlm_model = dynamic_cast<CausalVLM*>(model_);
+    if (vlm_model) {
+      vlm_model->init_encoder_graph_manager(args, device);


The member variable model_ is already declared as CausalVLM* in vlm_executor_impl.h. Performing a dynamic_cast<CausalVLM*> on it is redundant and unnecessary. You can directly check if model_ is non-null.

if (model_) { model_->init_encoder_graph_manager(args, device);

gemini-code-assist · 2026-07-01T08:35:33Z

+  bool need_restore = false;
+  if (c10_npu::getCurrentNPUStream(device_index_) ==
+      c10_npu::getDefaultNPUStream(device_index_)) {
+    c10_npu::setCurrentNPUStream(*capture_stream_);
+    aclrtSynchronizeStream(capture_stream_->stream());
+    need_restore = true;
+  }


If need_restore is false (meaning the current stream was already a non-default stream), the capture stream is not switched to capture_stream_. However, at the end of capture, aclrtSynchronizeStream(capture_stream_->stream()) is still called. This synchronizes the wrong stream, which can lead to race conditions or undefined behavior. We should record the stream actually used for capture and synchronize it.

Suggested change

bool need_restore = false;

if (c10_npu::getCurrentNPUStream(device_index_) ==

c10_npu::getDefaultNPUStream(device_index_)) {

c10_npu::setCurrentNPUStream(*capture_stream_);

aclrtSynchronizeStream(capture_stream_->stream());

need_restore = true;

}

bool need_restore = false;

if (c10_npu::getCurrentNPUStream(device_index_) ==

c10_npu::getDefaultNPUStream(device_index_)) {

c10_npu::setCurrentNPUStream(*capture_stream_);

aclrtSynchronizeStream(capture_stream_->stream());

need_restore = true;

}

aclrtStream capture_raw_stream = c10_npu::getCurrentNPUStream(device_index_).stream();

gemini-code-assist · 2026-07-01T08:35:34Z

+  }
+
+  // Verify with test replay
+  aclrtSynchronizeStream(capture_stream_->stream());


Synchronize the stream that was actually used for the capture instead of hardcoding capture_stream_->stream(), which might not have been used if need_restore was false.

Suggested change

aclrtSynchronizeStream(capture_stream_->stream());

aclrtSynchronizeStream(capture_raw_stream);

gemini-code-assist · 2026-07-01T08:35:34Z

+  auto cu_tensor = torch::tensor(
+      param.cu_seqlens_vec,
+      torch::TensorOptions().device(device_).dtype(torch::kInt32));
+  param.cu_seqlens.set_data(cu_tensor);


Using param.cu_seqlens.set_data(cu_tensor) changes the underlying memory address of the tensor. Since the ACL/NPU graph captures the physical memory addresses of tensors at capture time, changing the address during replay means the graph will continue to read from the old address (which may contain stale data or have been deallocated), leading to incorrect results or crashes. Instead, you should copy the data into the existing tensor to preserve its memory address when the shapes match.

auto cu_tensor = torch::tensor( param.cu_seqlens_vec, torch::TensorOptions().device(device_).dtype(torch::kInt32)); if (param.cu_seqlens.sizes() == cu_tensor.sizes()) { param.cu_seqlens.copy_(cu_tensor); } else { param.cu_seqlens.set_data(cu_tensor); }

gemini-code-assist · 2026-07-01T08:35:34Z

+    torch::Tensor deepstack_out_0;  // [budget/merge_sq, d_model]
+    torch::Tensor deepstack_out_1;  // [budget/merge_sq, d_model]
+    torch::Tensor deepstack_out_2;  // [budget/merge_sq, d_model]


Hardcoding deepstack_out_0, deepstack_out_1, and deepstack_out_2 makes the class fragile and limited to exactly 3 deepstacks. If a model has a different number of deepstacks, this will waste memory or fail. Refactor this to use a std::vector<torch::Tensor> to support any number of deepstacks dynamically.

Suggested change

torch::Tensor deepstack_out_0; // [budget/merge_sq, d_model]

torch::Tensor deepstack_out_1; // [budget/merge_sq, d_model]

torch::Tensor deepstack_out_2; // [budget/merge_sq, d_model]

std::vector<torch::Tensor> deepstack_outs;

gemini-code-assist · 2026-07-01T08:35:34Z

+      device_index_(device.index()),
+      dtype_(args.dtype() == "bfloat16" ? torch::kBFloat16 : torch::kFloat16) {
+  max_budget_ = budgets_.empty() ? 0 : budgets_.back();
+  capture_stream_.emplace(c10_npu::getStreamFromPool(true, device_index_));


Annotate constant arguments with a comment indicating the parameter name when calling functions or constructors.

Suggested change

capture_stream_.emplace(c10_npu::getStreamFromPool(true, device_index_));

capture_stream_.emplace(c10_npu::getStreamFromPool(/*isHighPriority=*/true, device_index_));

References

Annotate constant arguments with a comment indicating the parameter name when calling functions or constructors. ^(link)

gemini-code-assist · 2026-07-01T08:35:34Z

+  {
+    LOG(INFO) << "[EncoderGraph] Warm-up run with budget=" << budget;
+    torch::Tensor warm_hidden = param.hidden_states.clone();
+    for (int i = 0; i < static_cast<int>(layers.size()); ++i) {


Use fixed-width integers (int32_t, int64_t) instead of plain int.

Suggested change

for (int i = 0; i < static_cast<int>(layers.size()); ++i) {

for (int32_t i = 0; i < static_cast<int32_t>(layers.size()); ++i) {

References

Use fixed-width integers (int32_t, int64_t) instead of plain int. ^(link)

gemini-code-assist · 2026-07-01T08:35:34Z

+
+  // Run 27-layer encoder loop + deepstack mergers
+  torch::Tensor hidden = param.hidden_states;
+  for (int i = 0; i < static_cast<int>(layers.size()); ++i) {


Use fixed-width integers (int32_t, int64_t) instead of plain int.

Suggested change

for (int i = 0; i < static_cast<int>(layers.size()); ++i) {

for (int32_t i = 0; i < static_cast<int32_t>(layers.size()); ++i) {

References

Use fixed-width integers (int32_t, int64_t) instead of plain int. ^(link)

gemini-code-assist · 2026-07-01T08:35:34Z

+                               ? "true"
+                               : "false")
+                        : "N/A");
+      for (int idx = 0; idx < static_cast<int>(layers_.size()); ++idx) {


Use fixed-width integers (int32_t, int64_t) instead of plain int.

Suggested change

for (int idx = 0; idx < static_cast<int>(layers_.size()); ++idx) {

for (int32_t idx = 0; idx < static_cast<int32_t>(layers_.size()); ++idx) {

References

Use fixed-width integers (int32_t, int64_t) instead of plain int. ^(link)

gemini-code-assist · 2026-07-01T08:35:35Z

+                            deepstack_visual_indexes_.end(),
+                            idx);
+        if (it != deepstack_visual_indexes_.end()) {
+          int index = std::distance(deepstack_visual_indexes_.begin(), it);


Use fixed-width integers (int32_t, int64_t) instead of plain int, and use static_cast for type conversions.

Suggested change

int index = std::distance(deepstack_visual_indexes_.begin(), it);

int32_t index = static_cast<int32_t>(std::distance(deepstack_visual_indexes_.begin(), it));

References

Use fixed-width integers (int32_t, int64_t) instead of plain int. ^(link)

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping. Co-Authored-By: Claude <noreply@anthropic.com>

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping.

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Fix non-NPU backend build: the has_init_encoder_graph_manager SFINAE trait was defined inside #if defined(USE_NPU) but is referenced unconditionally from CausalVLMImpl::init_encoder_graph_manager, so any non-NPU TU compiling causal_vlm.h failed with 'not a member of xllm::detail'. Moved the trait out of the USE_NPU guard (it uses only ModelArgs/torch::Device; on non-NPU no model provides the method, so it evaluates false and the if-constexpr body is discarded — encoder graph stays NPU-only at runtime). - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping.

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Fix non-NPU backend build: the has_init_encoder_graph_manager SFINAE trait was defined inside #if defined(USE_NPU) but is referenced unconditionally from CausalVLMImpl::init_encoder_graph_manager, so any non-NPU TU compiling causal_vlm.h failed with 'not a member of xllm::detail'. Moved the trait out of the USE_NPU guard (it uses only ModelArgs/torch::Device; on non-NPU no model provides the method, so it evaluates false and the if-constexpr body is discarded — encoder graph stays NPU-only at runtime). - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping. Post-build-verification: - Drop the qwen3.h direct #include of rotary_embedding_util.h that this commit had added: a clean build without it succeeds, so layer::rotary::get_concat_rotary_embedding is still resolved transitively (via npu_qwen3_decoder_layer_impl.h / llm_model_base.h). The include was a build side-effect of the CMakeLists change, not required by the feature. Co-Authored-By: Claude <noreply@anthropic.com>

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Fix non-NPU backend build: the has_init_encoder_graph_manager SFINAE trait was defined inside #if defined(USE_NPU) but is referenced unconditionally from CausalVLMImpl::init_encoder_graph_manager, so any non-NPU TU compiling causal_vlm.h failed with 'not a member of xllm::detail'. Moved the trait out of the USE_NPU guard (it uses only ModelArgs/torch::Device; on non-NPU no model provides the method, so it evaluates false and the if-constexpr body is discarded — encoder graph stays NPU-only at runtime). - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping. Post-build-verification: - Drop the qwen3.h direct #include of rotary_embedding_util.h that this commit had added: a clean build without it succeeds, so layer::rotary::get_concat_rotary_embedding is still resolved transitively (via npu_qwen3_decoder_layer_impl.h / llm_model_base.h). The include was a build side-effect of the CMakeLists change, not required by the feature. Production tuning (benefit-driven, folded in): 4-config (1/2/4/8 TP) x 7-token sweep showed graph only helps at small token counts on high-TP (collective-launch-bound, e.g. 8-card 256 +30.3%) and the captured encoder kernel is slower than eager past ~512 tokens (-3..-5%, TP-independent), so capture is capped at 512. - Hard cap: VisionEncoderAclGraphManager drops any budget > kEncoderGraphMaxReplayTokens (512) at construction via filter_budgets() with a WARNING; select_budget then returns -1 for actual_tokens > 512 so larger images fall back to eager. No config knob (cap is fixed). - Default --encoder_graph_budgets "1024,2048,4096,8192" -> "256,512" so the feature is useful out-of-box (small exact/near-exact buckets). - copy_inputs_to_persistent: drop the redundant full-budget zero_() on the head [0, actual) (immediately overwritten by copy_); zero only the padding tail [actual, bucket) when actual < bucket, since padding rows join the last segment's attention. Exact-budget does no zeroing. - Drop uncommitted ViT-forward timing diagnostics (Timer / [ViTForward] LOG / aclrtSynchronizeDevice) and the now-unused timer.h / acl_rt.h includes from qwen3_vl.h. - Trim verbose manager comments to concise one-liners. Verified: build_fia.sh incremental build succeeds; pre-commit clang-format passes. Co-Authored-By: Claude <noreply@anthropic.com>

Capture the vision encoder loop as an ACL Graph (NPUGraph) to cut operator dispatch latency and lower TTFT for VLM inference. The graph manager (core/runtime/vision_encoder_acl_graph_manager) is model-agnostic: it captures/replays purely through the VisionEncoderGraphAdapter interface in xllm::npu, so core/runtime never depends on a concrete model type. Qwen3_VisionTransformerImpl implements the adapter and registers itself via set_adapter(this). - Bucket-based lazy capture/replay with persistent input/output buffers - Stable-storage cu_seqlens: in-place copy_ into a pre-allocated device tensor (no set_data storage swap) so the captured graph reads a valid address on every replay - Segment-count guard: a request whose image count differs from the captured graph falls back to eager instead of corrupting attention - capture() wrapped in try/catch with stream restore + eager fallback on failure - head_dim sourced from args.mm_head_dim() (not derived) to support GQA - Config: --enable_encoder_graph, --encoder_graph_budgets Address PR xLLM-AI#1866 review feedback: - Fix capture-stream sync race: when the current stream was already non-default, capture ran on it but we synced capture_stream_ (wrong/idle stream). Record the raw stream actually used and synchronize that. - Remove redundant dynamic_cast<CausalVLM*>(model_) in VlmExecutorImpl (model_ is already CausalVLM*); null-check is sufficient. - Fix non-NPU backend build: the has_init_encoder_graph_manager SFINAE trait was defined inside #if defined(USE_NPU) but is referenced unconditionally from CausalVLMImpl::init_encoder_graph_manager, so any non-NPU TU compiling causal_vlm.h failed with 'not a member of xllm::detail'. Moved the trait out of the USE_NPU guard (it uses only ModelArgs/torch::Device; on non-NPU no model provides the method, so it evaluates false and the if-constexpr body is discarded — encoder graph stays NPU-only at runtime). - Style (per .agents/skills/code-review/references/custom-code-style.md): project-root-relative #include, class final, braces on single-line if, int32_t over plain int with static_cast, no auto for std::string, annotate constant args, clang-format line wrapping. Post-build-verification: - Drop the qwen3.h direct #include of rotary_embedding_util.h that this commit had added: a clean build without it succeeds, so layer::rotary::get_concat_rotary_embedding is still resolved transitively (via npu_qwen3_decoder_layer_impl.h / llm_model_base.h). The include was a build side-effect of the CMakeLists change, not required by the feature. Production tuning (benefit-driven, folded in): 4-config (1/2/4/8 TP) x 7-token sweep showed graph only helps at small token counts on high-TP (collective-launch-bound, e.g. 8-card 256 +30.3%) and the captured encoder kernel is slower than eager past ~512 tokens (-3..-5%, TP-independent), so capture is capped at 512. - Hard cap: VisionEncoderAclGraphManager drops any budget > kEncoderGraphMaxReplayTokens (512) at construction via filter_budgets() with a WARNING; select_budget then returns -1 for actual_tokens > 512 so larger images fall back to eager. No config knob (cap is fixed). - Default --encoder_graph_budgets "1024,2048,4096,8192" -> "256,512" so the feature is useful out-of-box (small exact/near-exact buckets). - copy_inputs_to_persistent: drop the redundant full-budget zero_() on the head [0, actual) (immediately overwritten by copy_); zero only the padding tail [actual, bucket) when actual < bucket, since padding rows join the last segment's attention. Exact-budget does no zeroing. - Drop uncommitted ViT-forward timing diagnostics (Timer / [ViTForward] LOG / aclrtSynchronizeDevice) and the now-unused timer.h / acl_rt.h includes from qwen3_vl.h. - Strip non-essential comments; keep only 2 (the 512-cap rationale and the padding-tail zero invariant) plus mandatory license headers. Verified: build_fia.sh incremental build succeeds; pre-commit clang-format passes. Co-Authored-By: Claude <noreply@anthropic.com>

sunbaosong requested review from Clement-Wang26, DongheJin, DragonFive, JimHsiung, Kang-Meng, RobbieLeung, XuZhang99, liujinguang0125, liutongxuan, walsonyang, xiao-yu-chen, yingxudeng, yq33victor and zhang-minchao as code owners July 1, 2026 08:30

gemini-code-assist Bot reviewed Jul 1, 2026

View reviewed changes

XuZhang99 changed the title ~~feat: add ViT encoder ACL Graph capture for Qwen3-VL~~ feat: add ViT encoder ACL Graph capture for Qwen3-VL. Jul 1, 2026

sunbaosong force-pushed the vit-graph branch from 98ec508 to a5a8da4 Compare July 1, 2026 09:39

sunbaosong force-pushed the vit-graph branch from a5a8da4 to 9642d3d Compare July 2, 2026 01:30

sunbaosong force-pushed the vit-graph branch from 9642d3d to 43c8864 Compare July 2, 2026 01:49

sunbaosong force-pushed the vit-graph branch from 43c8864 to 3d7f149 Compare July 2, 2026 02:05

sunbaosong force-pushed the vit-graph branch from 3d7f149 to f844654 Compare July 3, 2026 07:01

sunbaosong force-pushed the vit-graph branch from f844654 to 8f4efbd Compare July 7, 2026 02:51

sunbaosong force-pushed the vit-graph branch from 8f4efbd to f18e4b5 Compare July 10, 2026 02:18

sunbaosong force-pushed the vit-graph branch from f18e4b5 to 02ae0f2 Compare July 10, 2026 03:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add ViT encoder ACL Graph capture for Qwen3-VL.#1866

feat: add ViT encoder ACL Graph capture for Qwen3-VL.#1866
sunbaosong wants to merge 1 commit into
xLLM-AI:mainfrom
sunbaosong:vit-graph

sunbaosong commented Jul 1, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	aclrtSynchronizeStream(capture_stream_->stream());
	aclrtSynchronizeStream(capture_raw_stream);

	capture_stream_.emplace(c10_npu::getStreamFromPool(true, device_index_));
	capture_stream_.emplace(c10_npu::getStreamFromPool(/isHighPriority=/true, device_index_));

	for (int i = 0; i < static_cast<int>(layers.size()); ++i) {
	for (int32_t i = 0; i < static_cast<int32_t>(layers.size()); ++i) {

	for (int idx = 0; idx < static_cast<int>(layers_.size()); ++idx) {
	for (int32_t idx = 0; idx < static_cast<int32_t>(layers_.size()); ++idx) {

	int index = std::distance(deepstack_visual_indexes_.begin(), it);
	int32_t index = static_cast<int32_t>(std::distance(deepstack_visual_indexes_.begin(), it));

Uh oh!

Conversation

sunbaosong commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

新增配置开关（2 个）

512 约束（澄清：非开关，不可配）

性能测试数据

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sunbaosong commented Jul 1, 2026 •

edited

Loading