Skip to content

【BUG】Connect runtime out of memory  #542

@VictoryAnn

Description

@VictoryAnn

BUG REPORT

  1. Please describe the issue you observed:
  • What did you do (The steps to reproduce)?

  • What is expected to see?
    task status is RUNNING

  • What did you see instead?
    task status is FAILED, and reason is OutOfMemory.

  1. Please tell us about your environment:

  2. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
    a. run jmap -heap 16 found memory top 1 is org.apache.rocketmq.connect.runtime.stats.ConnectStatsService.
    image
    b. use arthas watch sourceTaskTimesList and sinkTaskTimesList
    image
    image

SOLUTION:

    private void sampling() {
        this.lockSampling.lock();
        try {
            this.sourceTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sourceTaskTimesTotal()));
            if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
                this.sourceTaskTimesList.removeFirst();
            }
            // !!! sourceTaskTimesList is wrong !!!
           // !!! should be sinkTaskTimesList !!!
            this.sinkTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sinkTaskTimesTotal()));
            if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
                this.sourceTaskTimesList.removeFirst();
            }
        } finally {
            this.lockSampling.unlock();
        }
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions