Decide fate of NSSP secondary signals

[melange396]

New signals were added to NSSP in issue #2073 / PR #2074 in the hope of getting more frequent updates, but that new source data has not been appended to or revised since Oct 18, 2024 (currently ~100 days out of date).

PR #2089 should cause Sir Complains-a-Lot to ignore those stale signals, thus preventing alerting on their age, and in turn allow us to get better alerting on the age of the "main" NSSP signals.

We need to decide on whether we should continue to attempt to collect these signals, and even if we should permanently retain any of that data in the DB. We essentially have only one real "issue" of those 4 signals, because we started acquiring them Dec 16, 2024 (after the latest revision of the source data) -- however, we continue to import the same data every week and tag it with a new "issue" date.

Some more discussion is available in a slack thread: https://delphi-org.slack.com/archives/C01LZ3A2UMU/p1736451710690869?thread_ts=1736404465.936839&cid=C01LZ3A2UMU

[minhkhul]

Let me write some code to make nssp not import the same data every week. Not importing same data is helpful for other purposes anyway now that we have source data stash for all the soda api source indicators.

[melange396]

Roni's contacts at the CDC told us that they do not expect more updates to that data set, so we should cease its collection.

[minhkhul]

I see. I'll remove it from the codebase then.

[minhkhul]

TODO:

• release the code delete to stop gathering stale data
• adjust docs on epidata
• remove from signal sheets and related (ask Carlyn)
• delete data from db

[melange396]

reopening so the checklist above stays on our radar

[melange396]

all records for these have been removed from the database:

MySQL [covid]> select * from signal_dim where `signal` like '%_2023rvr';
+---------------+--------+---------------------------------+
| signal_key_id | source | signal                          |
+---------------+--------+---------------------------------+
|           843 | nssp   | pct_ed_visits_combined_2023rvr  |
|           844 | nssp   | pct_ed_visits_covid_2023rvr     |
|           845 | nssp   | pct_ed_visits_influenza_2023rvr |
|           846 | nssp   | pct_ed_visits_rsv_2023rvr       |
+---------------+--------+---------------------------------+
4 rows in set (0.01 sec)

MySQL [covid]> delete from epimetric_latest where signal_key_id in (select signal_key_id from signal_dim where `signal` like '%_2023rvr');         
Query OK, 25204 rows affected (1.23 sec)

MySQL [covid]> delete from epimetric_full where signal_key_id in (select signal_key_id from signal_dim where `signal` like '%_2023rvr');  
Query OK, 126020 rows affected (26.57 sec)

MySQL [covid]> delete from signal_dim where `signal` like '%_2023rvr';                                                              
Query OK, 4 rows affected (0.00 sec)

[melange396]

AFAICT, to remove the last traces of these signals, we need to do #2109 and remove their lines from the signals sheet (theyve not made their way to the production environment yet, see cmu-delphi/delphi-epidata#1596).