Factor out amplitude in the Correlation (via phase) filter reporting.

[Soundmatters]

Factor out amplitude in the Correlation (via phase) filter reporting. As it stands now, level offsets between channels impact the reporting of ffmpeg’s phase filter. A test file is available here: https://drive.google.com/drive/folders/1PieIpN5w_IvTzfaRYoiPmJJCXlHyZXx8?usp=share_link

[dericed]

didn't this happen in 4bc596d?

[Soundmatters]

didn't this happen in 4bc596d?

No. That solved the problem of DC offset in the signal skewing the phase filter data. This issue is that amplitude offsets between channels skew the phase filter data.

[Soundmatters]

In the folder of test files (linked above) there are two pngs of two 2-channel mono recordings that illustrate the level/correlation issue. They are the same program, but one version was made with a channel offset of .6 dB and the other has a channel offset of 5 dB:
•REC0078_T2869 WITH ONLY point06 DB LEVEL OFFSET.wav.astatsaudit ; OVERALL CORRELATION VALUE = + .96
•REC0078_T2869 WITH 5 DB LEVEL OFFSET.wav.astatsaudit ; OVERALL CORRELATION VALUE = + .82

[Soundmatters]

Here is an additional file which may illustrate the issue better: https://drive.google.com/drive/folders/1j0sdy_byuBzaHmrtsrKV7XdUZkm1TY0k?usp=sharing

[dericed]

Hi @Soundmatters, I've tested this a few ways with having loudnorm and ebur128 put a rolling normalization before the phasemeter test, but the results are messy compared to the axcorrelation graph which we had removed. With the sample you shared, it seems well correlated but has amplitude differences, so I took one channel and offset some of the samples for a few minutes so I could force a loss of correlation.
I eventually I added the axcorrelation back in to test that against other methods.
From there I wanted to check back to the version of astataudit before axcorrelation was removed and found it was different than my current work in progress.

Here's the graph just before axcorrelate was removed in eaf656a. This is on your sample with a section of audio in a single channel offset to force a correlation issue.

Perhaps this was due to the work of the preceding commits, but I see the axcorrelation graph in this commit was problematic. It shows the correlation but the x-axis is halved.

In my work in progress it looks like:

So here the axcorrelation aligns well with the phase graph. They both show the issue but the phase graph factors in amplitude whereas the axcorrelation one doesn't.

So this all has me trying to remember why the axcorrelation graph was dropped as I haven't found anything better than it for plotting phase correlation without factoring in amplitude. Beyond being more accurate it also is much faster than adding in a rolling normalization step before the phasemeter analysis.

[dericed]

Hi @Soundmatters, I think you replied via email rather than at #10, so the image attachments didn't come through.

[Soundmatters]

So the way I remember it, we dropped axcorrelation in the reporting because
--aside from analysis of audio files with an amplitude offset between
channels-- aphasemeter was providing more accurate information; the x-axis
issue that you note may very well have been part of that problem. Here are
a few examples of unreliable reporting that seem unrelated to x-axis issue
though:

1. axcorrelation was reporting values greater than 1; many instances of
   this in the first 5 minutes of this example
2. in the opening minute in this example, there is a 30-second Dolby A
   tone (uncorrelated by design) followed by a 30-second 1 k sine wave (which
   is well-aligned/in-phase in this example); aphasemeter reports accurately,
   axcorrelation does not

And here is another example of very different reporting. The aphasemeter
data here tracks very closely with some commercial software that I use

[Soundmatters]

Please note that the first image that I posted yesterday was not correct; it has been replaced with the correct image.

[dericed]

Hey @Soundmatters, with my latest branch here is a graph of the output, including a revised contextualization of axcorrelate so you can compare the before and after.

1 axcorrelation was reporting values greater than 1; many instances of
this in the first 5 minutes of this example

In the new one there's no values over 1. From a skim, I see 0.999989 but no 1's.

in the opening minute in this example, there is a 30-second Dolby A
tone (uncorrelated by design) followed by a 30-second 1 k sine wave (which
is well-aligned/in-phase in this example); aphasemeter reports accurately,
axcorrelation does not

I reran this process with a -m 2 to get the first 2 mins in their own graph. Here's that:

In the new version the aphasemeter values and axcorrelate values are roughly similar. Let me know what you think, if it seems okay, I can merge into a new release for your testing.

[Soundmatters]

Fantastic!

If you don’t think that it would slow down astataudit’s processing time too much, could we leave both the “Correlation” graphic and this updated “Normalized Cross Correlation” in for the time being? It would give us the chance to compare their analysis over large sets of data. And if you could add the same color scale to the “Normalized Cross Correlation” graphic, that might help with the comparison, though not at all essential if you are out of time. Thanks.

[dericed]

Hi @Soundmatters, here's the 2 minute sample with the color patterns matched.

But you're right, there is a notable speed difference. I ran this on WNYC-NSDS-1987-05-22-32942.6 0027 Show Music From France.wav and compared the last release to my current draft and it's 5:04 for the current draft, and 0:33 for the last release. The axcorrelate does manage a fast and slow algorithm and I was using the slow one, with the fast it takes 1:12 and the graph looks like this. The difference between fast and slow is detailed at https://ffmpeg.org/ffmpeg-filters.html#axcorrelate.

I should note that between the last release and the current draft, the axcorrelation isn't the only addition but there's the spectrum as well.

[Soundmatters]

The default/slow setting certainly seems more accurate.

I’m not sure how helpful this would be, but if dropping some of the other filter graphics would lighten the processing load, I’d say: 1) keep axcorrelate in the default/slow mode, 2) drop zero crossings for now, 3) drop the spectral analysis for now.

The correlation reporting is so important and useful that its accuracy is a primary concern for us.

Maybe, down the road, zero crossings and spectral analysis could be added as options.

[dericed]

Hey @Soundmatters, this is a bit of an investment for future work, but I refactored the way the graph is constructed to separate each analyzer (they were all tangled together before). This should make it a lot easier for me to scale it to add future analyzers.

So with this I can turn them on/off and benchmark. So if I only use one at a time:

astats (without reset) which is for dcoffset
28 seconds

astats (with a reset every frame)
30 seconds

aphasemeter
52 seconds

axcorrelate (slow)
9:42

axcorrelate (fast)
29 seconds

showspectrumpic
34 seconds

all (with fast axcorrelation)
3:13

all (with slow axcorrelation)
9:07

These were just quick single run tests but obviously something was off as the solo run of slow axcorrelation was slower than that with all the others.

[Soundmatters]

I'm wondering about the last graphic in the previous post "all (with slow axcorrelation) 9:07" Is that correct? The Normalized Cross Correlation data looks inaccurate (see the uncorrelated Dolby Tone in the first 30 seconds displaying as almost +1; also other data looks like it is displayed at half its value).