Breaks intermittently due to Thread::create failed error

[Tanmay3610]

I have created a custom docker image which is using shelf/lambda-libreoffice-base:7.4-node16-x86_64 as a base image by following the instructions in README file. It is working fine for most of the time but it intermittently breaks.

Here's the code of my docker image:

FROM public.ecr.aws/shelf/lambda-libreoffice-base:7.4-node16-x86_64

COPY ./fonts/* /usr/local/share/fonts/
COPY ./ ./
RUN yum install java-1.8.0-openjdk-devel -y
CMD [ "app.handler" ]

And here's my handler function:

const handler = async (event) => {
  try {
    const {fileUrl, isExcel} = event;

    const inputFileExtension = ".docx";
    const fileName = `template-${uuidv4()}${inputFileExtension}`;
    await download(fileUrl, `/tmp/${fileName}`);
    const pdfFilePath = convertTo(fileName, 'pdf');
    const stats = fs.statSync(pdfFilePath);
    const pdfFileData = fs.readFileSync(pdfFilePath);
  
    await clearTmpDirectory();
    const response = {
      statusCode: 200,
      body: pdfFileData
    };
    return response;
  } catch (err) {
    console.error(err);
    throw err;
  }
}

Here download function is simply downloading my file from url that has been passed to it and clearTmpDirectory function is simply clearing all the files from tmp directory.

Here's the error that I am getting intermittently:

javaldx: Could not find a Java Runtime Environment!

what():  osl::Thread::create failed

terminate called after throwing an instance of 'std::runtime_error'

Error: Command failed: cd /tmp && libreoffice7.4 --headless --invisible --nodefault --view --nolockcheck --nologo --norestore --convert-to pdf --outdir /tmp /tmp/template-bf853fb4-95d9-4fef-87b1-59c54cf13c58.docx
javaldx: Could not find a Java Runtime Environment!
Warning: failed to read path from javaldx
terminate called after throwing an instance of 'std::runtime_error'
  what():  osl::Thread::create failed

    at checkExecSyncError (node:child_process:861:11)
    at execSync (node:child_process:932:15)
    at convertTo (/var/task/node_modules/@shelf/aws-lambda-libreoffice/lib/convert.js:29:40)
    at Runtime.handler (/var/task/app.js:66:25) {
  status: 134,
  signal: null,
  output: [
    null,
    <Buffer >,
    <Buffer 6a 61 76 61 6c 64 78 3a 20 43 6f 75 6c 64 20 6e 6f 74 20 66 69 6e 64 20 61 20 4a 61 76 61 20 52 75 6e 74 69 6d 65 20 45 6e 76 69 72 6f 6e 6d 65 6e 74 ... 150 more bytes>
  ],
  pid: 12370,
  stdout: <Buffer >,
  stderr: <Buffer 6a 61 76 61 6c 64 78 3a 20 43 6f 75 6c 64 20 6e 6f 74 20 66 69 6e 64 20 61 20 4a 61 76 61 20 52 75 6e 74 69 6d 65 20 45 6e 76 69 72 6f 6e 6d 65 6e 74 ... 150 more bytes>
}

Note: This works perfectly fine for a while if I redeploy my same image on lambda, but after that it again starts to fail with above error intermittently.

[dallanmc]

@Tanmay3610 did you ever resolve this issue? I'm seeing something similar when I use the lambda at scale. Basically a warm instance of the lambda will successfully convert 200 documents and then just stop with the error you have above.

Obviously if the lambda goes cold and restarts then things work again (because the container is restarted).

[Tanmay3610]

Nope, I was not able to resolve the issue, switched to EC2 instead. But issue was due to memory leak as confirmed from AWS Lambda team.

[dallanmc]

I think the issue is because the Aws container only allows a maximum of 1000 new threads created.

I switched to start up libreoffice as a service (using jodconverter). This means that each new conversion no longer requires a new thread.