Unable to delete records using LogBatchPurger

[abhinavk45-debug]

Package Edition of Nebula Logger

Managed Package

Package Version of Nebula Logger

4.16

New Bug Summary

Hi Team,

We have installed the Nebula logger package in our prod org, but we didnt start the batch to purge the record.
Now , Log, LogEntry and LogEntryTag is holding 100M records. When we started the batch, we are getting error that request ran for too long and batch job is failing. Do we have any other way to delete the records.

When we are trying to delete manually, we are getting unable to lock row error as it is trying to modify the totalLogEntries field on loggertag object and as multiple log entries are tied with same LoggerTag, we are getting this error.

Can you help me understand how LoggerTag record get created? Can we delete all the loggerTag first, and then try to delete the other records? Will that work.

-Abhinav

[jongpie]

Hi @abhinavk45-debug - I have a couple of follow up questions for you:

1. Could you clarify how many records you have in each object? I'm not sure if the 100M records you mentioned is the total across all the objects, or 100M in each object.
2. When you tried to manually delete them, what order were you trying to delete the objects? Did you sort the data in any way when doing the deletion?

As far as trying to solve this in Nebula Logger, I think Salesforce hasa query limit of 50 million records in the start method of batchable Apex (which would explain why it's failing with your data volume) - I might be able to simply add a LIMIT statement to the SOQL queries in LogBatchPurger. My hope is that the job would then be able to successfully delete up to X number records in 1 run - in large data volume situations like yours, it may then take a few runs/days to "catch up" on the records to delete, but I think would avoid the errors you're seeing.

I don't think I have an org where I could generate that much data to test this out - I'll have to see if I can figure out a way to somehow recreate this issue & test a fix.

[abhinavk45-debug]

Hi @jongpie ,

1. 100M is across the objects, like logEntryTag has 55M, LogEntry has 30 M, and Log has around 17M records.
2. i was trying to delete Log object record directly as the volume is so high, i wanted to get rid of them quickly.

Can you help me with details on LoggerTag ? it seems i am getting error due to linage of Log Entry and LogEntryTag with Logger tag and it fails . If i delete all the LoggerTag, and then i try to delete Log record, directly, i think that should work.

My question is, if i delete LoggerTag and then any new Log record comes with some LoggerTag, will that create new LoggerTag record if it dont find the existing one?

what do you think about this approach? i will delete all the loggerTag, then i will delete the Log record which should delete all the logEntry and LogEntryTag records as well. will this approach work?