Clipping detection

[claudiodsf]

Hi @krisvanneste,
I moved the discussion on clipping detection here.

Yes, it is hard! So I improved the strategy and added a parameter (commit c71cc5f).

• The improved strategy is to add a distance weighting for the kernel density: peaks that are far away from the trace average will be weighted more (up to 5 times more).
• The new parameter is a sensitivity parameter (between 1 and 5, or 0 for no clipping check) which changes the prominence threshold for detecting peaks. I decided to put pre-defined thresholds in order to avoid too much complexity for the end user. Let me know if you think that those threshold are adequate.

I moved the clipping detector to a standalone module, which can be called from the command line program clipping_detection, in case one wants to do some test (but bear in mind that traces are not cut, when calling from the command line).

Let me know what do you think, as soon as you have time (as usual, no rush!), thanks!

P.S. I tried GitHub Copilot to write the docstring of is_clipped() as well as the main function with argparse: it is pretty impressive!

[krisvanneste]

Claudio,

I will run a test in the coming days!
Will have a look at this GitHub Copilot as well.

[claudiodsf]

Thanks!

The only downside of Copilot is that it's 10$/month... (after 60 days trials)

I'm trying and see if my department can pay for it 😉

[krisvanneste]

Oh I see, that could be a bummer...

[krisvanneste]

Claudio,

I tested your new clip detection with my data, with variable success (both false positives and false negatives).
In the end I modified the code a bit. I think just testing for the presence of more than 1 peak is too simple. Instead I propose to check for the presence of peaks in the first and last few bins to identify clipping.

I made the following changes:

• decreased smoothing for Gaussian kernel density
• do not estimate kernel density beyond the max. amplitude
• add 1 bin to kernel density at start and end before peak finding, and remove it again afterwards
• only consider the presence of peaks in the first or last few (5) bins to indicate clipping
• require a minimum of 11 bins in the plotted histogram, and fixed its shift with respect to the kernel density

With these changes, I obtain 3 correct detections (BE.BEBN, BE.MEMB BE.STI, see attached plots), 1 false positive (station BE.STWA) and no false negatives for the P phase on the Z component with my test case, and 4 correct detections (BE.BEBN, BE.LCH, BE.MEMB, BE.RCHB), 1 possible false positive (BE.MEMS, limit case) and no false negatives for the S phase on the N component. In all cases, this is using the lowest sensitivity (which I think should be the default with the current implementation). With higher sensitivities, more false positives are coming up. For the P phase, only 2 stations never trigger a clip detection, and for the S phase only 1.

Debug plots:
clipping_detection.zip

Further improvements that could be considered:

• increase the prominence thresholds for the different sensitivities or allow full control of the peak prominence parameter
• do not apply weighting to the kernel density
• set default sensitivity to lower value (depending on the previous changes)
• improve plot by only showing peaks corresponding to clipping or use a different symbol/color

I made a new branch and will create a pull request.

[claudiodsf]

Hi Kris, thanks for this work!!!

I'll review it in the following days or next week.

[krisvanneste]

Claudio, no hurry. I'm quite busy as well these days.