Detailled digitizer

[jessy-daniel]

BEGINRELEASENOTES

Add the possibility to use the new detailed tracking digitizer as option, with the option --detailedDigitization. By default, we keep using the previous version using DDPlanarDigiprocessor from Marlin software.
The new digitizer is implement in Key4Hep within the k4RecTracker repository and has been merged in this merge request : key4hep/k4RecTracker#64

ENDRELEASENOTES

This release goal is to add the possibility to use the new detailed tracking digitizer as option, with the option --detailedDigitization. By default, we keep using the previous version using DDPlanarDigiprocessor from Marlin software.

The new digitizer is implement in Key4Hep within the k4RecTracker repository and has been merged in this merge request : key4hep/k4RecTracker#64

Some details on the implementation and results may be found in this talk : https://indico.cern.ch/event/1588696/contributions/6869431/
Note that the results from this talk have been derived using a modiied geometry where Vertex MAPS sensors have been adapted to 10mum sensitive thickness for more realistic geometry.

Concerning CPU efficiency, we ran CLDReconstruction with both original digitizer and new detailled one, using particle gun with 5000 muons in lxplus. The CPU time was :
Original version : 13m41.067s
New detailed version : 12m12.455s
We suppose that this gain of time (which is counterintuitive as more calculation is made in the detailed one) comes from the fact that the new digitizer is directly implemented within key4HEP and does not need MarlinWrapper.

[andresailer]

Thanks for your contribution!

Are you getting comparable results in terms of the efficiency and performance for track reconstruction?

[jessy-daniel]

Thanks for your contribution!

Are you getting comparable results in terms of the efficiency and performance for track reconstruction?

Hi @andresailer , what exactly would you want me to check ? Which plots ?

[andresailer]

Something like the track parameter pull distributions and track reconstruction efficiency.
The pull distributions are created by the TrackChecker algorithm and you can find them in the "Aida" output file from the standard reconstruction.

[jessy-daniel]

Hi, i just runned precommit to correct the format and pushed

[andresailer]

And some efficiency plot is in the same file in the MyCLicEfficiencyCalculator.

[jessy-daniel]

Something like the track parameter pull distributions and track reconstruction efficiency. The pull distributions are created by the TrackChecker algorithm and you can find them in the "Aida" output file from the standard reconstruction.

Dear @andresailer , you can find attached the tracking pulls for both original and new digitizer. This looks really similar except for D0.
However, I do not understand why D0 pull is worst for new digitizer. The D0 true (D0MCparticle) is at 0 (as expected for particle gun) and the reconstructed D0 (D0Track) is at -0.0014 for new digitizer and 0.0053 for the original one. So, new one pull should even be better, isn't it ?

Pulls Original :

Pulls new :

D0MCParticle :

D0Track Original :

D0track new :

[jessy-daniel]

And some efficiency plot is in the same file in the MyCLicEfficiencyCalculator.

All the plots in MyCLicEfficiencyCalculator look very compatible between both methods, or even better in the new version (for instance looking at recoChi2OverNDF plot in perfTree).

You can fin the root files there if you want to double-check : /afs/cern.ch/work/j/jessy/public/FCC/CLDConfig_tests

Thank you

[andresailer]

Thanks this looks reasonable! I was wondering whether the speed up came from just not having any reconstruction running after, but the number of reconstructed particles is the same, and as you write the pulls are similar or better. I don't know why the d0 pull bias gets bigger. More detailed studies would be needed there, I guess.

[andresailer]

@jmcarcell Do you have any comment in view of #70?

[jessy-daniel]

Thanks this looks reasonable! I was wondering whether the speed up came from just not having any reconstruction running after, but the number of reconstructed particles is the same, and as you write the pulls are similar or better. I don't know why the d0 pull bias gets bigger. More detailed studies would be needed there, I guess.

Something that can explain the D0 pull would be that in this nex digitiser, I consider the small drift of the electrns indide the sensor due to magnetic field . For 50mum thickness like in the geo there, this induce a 5mum deviation of the hit orthogonal to the magnetic field. What stays strange is that the reconstructed D0 is OK while the pull is worst....

[Zehvogel]

What stays strange is that the reconstructed D0 is OK while the pull is worst....

How do you define the reconstructed D0 being ok? The plot you showed above of the D0 distribution has such a large binning that one can't see anything. The D0 residual in your files shows the same bias introduced by the digitizer:

Do you have any more plots of the residuals of the hit positions? I looked at you FCC workshop talk and on the ones there in the backup it looks like there is a strong bias in the hit position reconstruction e.g. here:

If the hit position along phi is biased in the outer tracker the d0 reconstruction will be biased in the opposite direction (imagine the vertex detector as fulcrum for the track), is this the case here?

[jmcarcell]

This should run in CI if you want to make sure it keeps running.

Original version : 13m41.067s
New detailed version : 12m12.455s
We suppose that this gain of time (which is counterintuitive as more calculation is made in the detailed one) comes from the fact that the new digitizer is directly implemented within key4HEP and does not need MarlinWrapper.

No need to suppose anything, it can be measured. Measuring on lxplus is hard since they are shared machines, and one has to take into account interactions with CVMFS too, since the first time if files are not cached it will be slower. The digitizers take very little time in the CLD reconstruction, so the numbers above do not mean much.

@jmcarcell Do you have any comment in view of #70?

No problem, I'll incorporate once I have a look again at #70 if this is merged first...

[jessy-daniel]

What stays strange is that the reconstructed D0 is OK while the pull is worst....

How do you define the reconstructed D0 being ok? The plot you showed above of the D0 distribution has such a large binning that one can't see anything. The D0 residual in your files shows the same bias introduced by the digitizer:

Do you have any more plots of the residuals of the hit positions? I looked at you FCC workshop talk and on the ones there in the backup it looks like there is a strong bias in the hit position reconstruction e.g. here:

If the hit position along phi is biased in the outer tracker the d0 reconstruction will be biased in the opposite direction (imagine the vertex detector as fulcrum for the track), is this the case here?

Hi @Zehvogel , you are right for the binning, i did not touched to the Reconstruction code so this is the default one.

For the plot you are showing from my presentation, this is not a bias, this is accounting for a real physics effect, the magnetic field that is drifting the charges inside the sensor. As the sensitive thickness is 100mum in the OT geometry, this explains large magnetic drift, which were not included in previous simple digitizer.

However, probably at the end, at data taking, this drift effect will be somehow calibrated, so maybe I should add a fix bias to compensate this effect, no ? I think that usually this kind of correction is directly made in the reconstruction for both MC and Data, no ?

[Zehvogel]

For the plot you are showing from my presentation, this is not a bias, this is accounting for a real physics effect, the magnetic field that is drifting the charges inside the sensor. As the sensitive thickness is 100mum in the OT geometry, this explains large magnetic drift, which were not included in previous simple digitizer.

Ah now I understand, thanks for the explanation! :) I would still say it is a bias even if it is caused by a physical effect. The tracking code running after it does not expect the hits to be systematically shifted for any reason.

However, probably at the end, at data taking, this drift effect will be somehow calibrated, so maybe I should add a fix bias to compensate this effect, no ? I think that usually this kind of correction is directly made in the reconstruction for both MC and Data, no ?

Good question :/ I have no experience with real detectors so I am not familiar at which stage this would be calibrated. It looks like it should just need a fixed bias and I guess you can calculate it directly from the sensor thickness and magnetic field? It would be nice if you could apply such a correction directly in your digitiser or in a second algorithm that runs directly after (I would maybe slightly prefer the second option). Then the algorithms running after the digitisation have a better chance to succeed. :)

Otherwise I am all in favor of merging this. If the addition of the bias is too much work for the moment, I am also fine if you just add a big warning that gets printed when someone uses the new digitiser option. I would just like prevent the scenario that people use this to reconstruct events for an analysis and then complain that stuff looks weird...

[jessy-daniel]

Good question :/ I have no experience with real detectors so I am not familiar at which stage this would be calibrated. It looks like it should just need a fixed bias and I guess you can calculate it directly from the sensor thickness and magnetic field? It would be nice if you could apply such a correction directly in your digitiser or in a second algorithm that runs directly after (I would maybe slightly prefer the second option). Then the algorithms running after the digitisation have a better chance to succeed. :)

Indeed, it can be simply calculated by sensor thickness and magnetic field. However, we are not sure that putting such a "correction" inside the digitizer would be a good idea, at some point people may forget that there is this "correction" there while, at the end, it should be in reconstruction. The goal of the digitizer is to reproduce what is made in the sensor by physics and then the reconstruction should be similar in MC and data so taking this magnetic bias into account.
We will think about it and come back with a proposition, maybe choosing a simple warning as you propose.

[Zehvogel]

we are not sure that putting such a "correction" inside the digitizer would be a good idea, at some point people may forget that there is this "correction" there while, at the end, it should be in reconstruction.

I do agree :) my favourite solution would be to add a "calibration" algorithm that is run after your digitisation algorithm as part of the reconstruction when the --detailedDigitization flag is set.