Skip to content

[VL] Flaky test from Spark-40 and Spark-41 #12649

Description

@zhouyuan

Backend

VL (Velox)

Bug description

- SPARK-32388: TRANSFORM should handle schema less correctly (no serde)
17:20:17.096 WARN org.apache.spark.sql.execution.GlutenFallbackReporter: Validation failed for plan: LocalTableScan, due to: Gluten does not touch it or does not support it

17:20:17.096 WARN org.apache.spark.sql.execution.GlutenFallbackReporter: Validation failed for plan: SparkScriptTransformation, due to: Gluten does not touch it or does not support it

E20260727 17:20:17.165253  7538 VeloxMemoryManager.cc:446] Failed to release Velox memory manager as there are still outstanding memory resources. 
E20260727 17:20:17.165344  7538 Exceptions.h:55] Line: /work/ep/build-velox/build/velox_ep/velox/common/memory/Memory.cpp:196, Function:~MemoryManager, Expression:  pools_.size() != 0 (1 vs 0). There are unexpected alive memory pools allocated by user on memory manager destruction:
Memory Manager[capacity UNLIMITED alignment 64B usedBytes 0B number of pools 4
List of root pools:
__sys_root__ usage 0B reserved 0B peak 0B
    __sys_shared_leaf__0 usage 0B reserved 0B peak 0B
    __sys_tracing__ usage 0B reserved 0B peak 0B
    __sys_caching__ usage 0B reserved 0B peak 0B
    __sys_spilling__ usage 0B reserved 0B peak 0B

root usage 0B reserved 0B peak 0B
    default_leaf usage 0B reserved 0B peak 0B

	refcount 2
Memory Allocator[MALLOC capacity UNLIMITED allocated bytes 0 allocated pages 0 mapped pages 0]
ARBITRATOR[GLUTEN] CAPACITY 8388608.00TB numRequests 0 numRunning 0 numSucceded 0 numAborted 0 numFailures 0 numNonReclaimableAttempts 0 reclaimedFreeCapacity 0B reclaimedUsedCapacity 0B maxCapacity 0B freeCapacity 0B freeReservedCapacity 0B], Source: RUNTIME, ErrorCode: INVALID_STATE
terminate called after throwing an instance of 'facebook::velox::VeloxRuntimeError'
  what():  Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: pools_.size() != 0 (1 vs 0). There are unexpected alive memory pools allocated by user on memory manager destruction:
Memory Manager[capacity UNLIMITED alignment 64B usedBytes 0B number of pools 4
List of root pools:
__sys_root__ usage 0B reserved 0B peak 0B
    __sys_shared_leaf__0 usage 0B reserved 0B peak 0B
    __sys_tracing__ usage 0B reserved 0B peak 0B
    __sys_caching__ usage 0B reserved 0B peak 0B
    __sys_spilling__ usage 0B reserved 0B peak 0B
17:20:17.163 ERROR org.apache.spark.sql.execution.SparkScriptTransformationExec: /bin/bash: some_non_existent_command: command not found

Gluten version

No response

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions