Skip to content

Commit 9e21bfb

Browse files
authored
Merge pull request #1529 from cmu-delphi/release/delphi-epidata-4.1.26
Release Delphi Epidata 4.1.26
2 parents f431baf + 3ca502f commit 9e21bfb

File tree

10 files changed

+189
-12
lines changed

10 files changed

+189
-12
lines changed

.bumpversion.cfg

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[bumpversion]
2-
current_version = 4.1.25
2+
current_version = 4.1.26
33
commit = False
44
tag = False
55

dev/local/setup.cfg

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[metadata]
22
name = Delphi Development
3-
version = 4.1.25
3+
version = 4.1.26
44

55
[options]
66
packages =

docs/api/README.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,12 @@ nav_order: 3
44
has_children: true
55
---
66

7-
# Epidata API (Other Diseases)
7+
# Other Endpoints (COVID-19 and Other Diseases)
88

99
This is the home of [Delphi](https://delphi.cmu.edu/)'s epidemiological data
1010
API for tracking epidemics such as influenza, dengue, and norovirus. Note that
11-
our work on COVID-19 is described in the [COVIDcast Epidata API documentation](covidcast.md).
11+
additional data, including most COVID-19 signals, is available in the
12+
[main Epidata API (formerly known as COVIDcast)](covidcast.md).
1213

1314
## Table of Contents
1415
{: .no_toc .text-delta}

docs/api/covidcast-signals/fb-survey.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,8 @@ our survey, this means a and (b or c).
126126

127127
COVID-like illness or CLI is not a standard indicator. Through our discussions
128128
with the CDC, we chose to define it as: fever along with cough or shortness of
129-
breath or difficulty breathing.
129+
breath or difficulty breathing. From the list of symptoms from Q1 on
130+
our survey, this means a and (c or d or e).
130131

131132
Symptoms alone are not sufficient to diagnose influenza or coronavirus
132133
infections, and so these ILI and CLI indicators are *not* expected to be
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
---
2+
title: Youtube Survey
3+
parent: Inactive Signals
4+
grand_parent: COVIDcast Main Endpoint
5+
---
6+
7+
[//]: # (code at https://github.com/cmu-delphi/covid-19/tree/deeb4dc1e9a30622b415361ef6b99198e77d2a94/youtube)
8+
9+
# Youtube Survey
10+
{: .no_toc}
11+
12+
* **Source name:** `youtube-survey`
13+
* **Earliest issue available:** May 01, 2020
14+
* **Number of data revisions since May 19, 2020:** 0
15+
* **Date of last change:** Never
16+
* **Available for:** state (see [geography coding docs](../covidcast_geography.md))
17+
* **Time type:** day (see [date format docs](../covidcast_times.md))
18+
* **License:** [CC BY-NC](../covidcast_licensing.md#creative-commons-attribution-noncommercial)
19+
20+
## Overview
21+
22+
This data source is based on a short survey about COVID-19-like illness
23+
run by the Delphi group at Carnegie Mellon.
24+
[Youtube directed](https://9to5google.com/2020/04/29/google-covid-19-cmu-research-survey/)
25+
a random sample of its users to these surveys, which were
26+
voluntary. Users age 18 or older were eligible to complete the surveys, and
27+
their survey responses are held by CMU. No individual survey responses are
28+
shared back to Youtube.
29+
30+
This survey was a pared-down version of the
31+
[COVID-19 Trends and Impact Survey (CTIS)](../../symptom-survey/),
32+
collecting data only about COVID-19 symptoms. CTIS is much longer-running
33+
and more detailed, also collecting belief and behavior data. CTIS also reports
34+
demographic-corrected versions of some metrics. See our
35+
[surveys page](https://delphi.cmu.edu/covid19/ctis/) for more detail
36+
about how CTIS works.
37+
38+
The two surveys report some of the same metrics. While nominally the same,
39+
note that values from the same dates differ between the two surveys for
40+
[unknown reasons](#limitations).
41+
42+
As of late April 2020, the number of Youtube survey responses we received each
43+
day was 4-7 thousand. This was not enough coverage to report at finer
44+
geographic levels, so this indicator only reports at the state level. The
45+
survey ran from April 21, 2020 to June 17, 2020, collecting about 159
46+
thousand responses in the United States in that time.
47+
48+
We produce [influenza-like and COVID-like illness indicators](#ili-and-cli-indicators)
49+
based on the survey data.
50+
51+
## Table of Contents
52+
{: .no_toc .text-delta}
53+
54+
1. TOC
55+
{:toc}
56+
57+
## Survey Text and Questions
58+
59+
The survey contains the following 5 questions:
60+
61+
1. In the past 24 hours, have you or anyone in your household experienced any of the following:
62+
- (a) Fever (100 °F or higher)
63+
- (b) Sore throat
64+
- (c) Cough
65+
- (d) Shortness of breath
66+
- (e) Difficulty breathing
67+
2. How many people in your household (including yourself) are sick (fever, along with at least one other symptom from the above list)?
68+
3. How many people are there in your household (including yourself)?
69+
4. What is your current ZIP code?
70+
5. How many additional people in your local community that you know personally are sick (fever, along with at least one other symptom from the above list)?
71+
72+
73+
## ILI and CLI Indicators
74+
75+
We define COVID-like illness (fever, along with cough, or shortness of breath,
76+
or difficulty breathing) or influenza-like illness (fever, along with cough or
77+
sore throat) for use in forecasting and modeling. Using this survey data, we
78+
estimate the percentage of people (age 18 or older) who have a COVID-like
79+
illness, or influenza-like illness, in a given location, on a given day.
80+
81+
| Signals | Description |
82+
| --- | --- |
83+
| `raw_cli` and `smoothed_cli` | Estimated percentage of people with COVID-like illness <br/> **Earliest date available:** 2020-04-21 |
84+
| `raw_ili` and `smoothed_ili` | Estimated percentage of people with influenza-like illness <br/> **Earliest date available:** 2020-04-21 |
85+
86+
Influenza-like illness or ILI is a standard indicator, and is defined by the CDC
87+
as: fever along with sore throat or cough. From the list of symptoms from Q1 on
88+
our survey, this means a and (b or c).
89+
90+
COVID-like illness or CLI is not a standard indicator. Through our discussions
91+
with the CDC, we chose to define it as: fever along with cough or shortness of
92+
breath or difficulty breathing. From the list of symptoms from Q1 on
93+
our survey, this means a and (c or d or e).
94+
95+
Symptoms alone are not sufficient to diagnose influenza or coronavirus
96+
infections, and so these ILI and CLI indicators are *not* expected to be
97+
unbiased estimates of the true rate of influenza or coronavirus infections.
98+
These symptoms can be caused by many other conditions, and many true infections
99+
can be asymptomatic. Instead, we expect these indicators to be useful for
100+
comparison across the United States and across time, to determine where symptoms
101+
appear to be increasing.
102+
103+
104+
## Estimation
105+
106+
### Estimating Percent ILI and CLI
107+
108+
Estimates are calculated using the
109+
[same method as CTIS](./fb-survey#estimating-percent-ili-and-cli).
110+
However, the Youtube survey does not do weighting.
111+
112+
### Smoothing
113+
114+
The smoothed versions of all `youtube-survey` signals (with `smoothed` prefix) are
115+
calculated using seven day pooling. For example, the estimate reported for June
116+
7 in a specific geographical area is formed by
117+
collecting all surveys completed between June 1 and 7 (inclusive) and using that
118+
data in the estimation procedures described above. Because the smoothed signals combine
119+
information across multiple days, they have larger sample sizes and hence are
120+
available for more locations than the raw signals.
121+
122+
## Lag and Backfill
123+
124+
This indicator has a lag of 2 days. Reported values can be revised for one
125+
day (corresponding to a lag of 3 days), due to how we receive survey
126+
responses. However, these tend to be associated with minimal changes in
127+
value.
128+
129+
130+
## Limitations
131+
132+
When interpreting the signals above, it is important to keep in mind several
133+
limitations of this survey data.
134+
135+
* **Survey population.** People are eligible to participate in the survey if
136+
they are age 18 or older, they are currently located in the USA, and they are
137+
an active user of Youtube. The survey data does not report on children under
138+
age 18, and the Youtube adult user population may differ from the United
139+
States population generally in important ways. We don't adjust for any
140+
demographic biases.
141+
* **Non-response bias.** The survey is voluntary, and people who accept the
142+
invitation when it is presented to them on Youtube may be different from
143+
those who do not.
144+
* **Social desirability.** Previous survey research has shown that people's
145+
responses to surveys are often biased by what responses they believe are
146+
socially desirable or acceptable. For example, if it there is widespread
147+
pressure to wear masks, respondents who do *not* wear masks may feel pressured
148+
to answer that they *do*. This survey is anonymous and online, meaning we
149+
expect the social desirability effect to be smaller, but it may still be
150+
present.
151+
152+
Whenever possible, you should compare this data to other independent sources. We
153+
believe that while these biases may affect point estimates -- that is, they may
154+
bias estimates on a specific day up or down -- the biases should not change
155+
strongly over time. This means that *changes* in signals, such as increases or
156+
decreases, are likely to represent true changes in the underlying population,
157+
even if point estimates are biased.
158+
159+
### Privacy Restrictions
160+
161+
To protect respondent privacy, we discard any estimate that is based on fewer than 100 survey responses. For
162+
signals reported using a 7-day average (those beginning with `smoothed_`), this
163+
means a geographic area must have at least 100 responses in 7 days to be
164+
reported.
165+
166+
This affects some items more than others. It affects some geographic areas
167+
more than others, particularly areas with smaller populations. This affect is
168+
less pronounced with smoothed signals, since responses are pooled across a
169+
longer time period.
170+
171+
172+
## Source and Licensing
173+
174+
This indicator aggregates responses from a Delphi-run survey that is hosted on the Youtube platform.
175+
The data is licensed as [CC BY-NC](../covidcast_licensing.md#creative-commons-attribution-noncommercial).

src/client/delphi_epidata.R

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Epidata <- (function() {
1515
# API base url
1616
BASE_URL <- getOption('epidata.url', default = 'https://api.delphi.cmu.edu/epidata/')
1717

18-
client_version <- '4.1.25'
18+
client_version <- '4.1.26'
1919

2020
auth <- getOption("epidata.auth", default = NA)
2121

src/client/delphi_epidata.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
}
2323
})(this, function (exports, fetchImpl, jQuery) {
2424
const BASE_URL = "https://api.delphi.cmu.edu/epidata/";
25-
const client_version = "4.1.25";
25+
const client_version = "4.1.26";
2626

2727
// Helper function to cast values and/or ranges to strings
2828
function _listitem(value) {

src/client/packaging/npm/package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "delphi_epidata",
33
"description": "Delphi Epidata API Client",
44
"authors": "Delphi Group",
5-
"version": "4.1.25",
5+
"version": "4.1.26",
66
"license": "MIT",
77
"homepage": "https://github.com/cmu-delphi/delphi-epidata",
88
"bugs": {

src/server/_config.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
load_dotenv()
99

10-
VERSION = "4.1.25"
10+
VERSION = "4.1.26"
1111

1212
MAX_RESULTS = int(10e6)
1313
MAX_COMPATIBILITY_RESULTS = int(3650)

src/server/endpoints/covidcast_utils/db_signals.csv

+3-3
Original file line numberDiff line numberDiff line change
@@ -1575,7 +1575,7 @@ NSSP does not report county-level data for all counties with reporting EDs; some
15751575
The following states report no data through NSSP at the county level: CA, WA, AK, AZ, AL, CO, SD, ND, MO, AR, FL, OH, NH, CT, NJ.
15761576

15771577
South Dakota, Missouri, and territories report no data through NSSP at the state level.",Percentage,percent,other,bad,FALSE,FALSE,FALSE,FALSE,FALSE,,,,,,
1578-
nssp,pct_ed_visits_rsv,FALSE,pct_ed_visits_rsv,FALSE,COVID Emergency Department Visits (Percent of total ED visits),TRUE,Percent of ED visits that had a discharge diagnosis code of rsv,Percent of ED visits that had a discharge diagnosis code of rsv,National Syndromic Surveillance Program,rsv,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
1578+
nssp,pct_ed_visits_rsv,FALSE,pct_ed_visits_rsv,FALSE,RSV Emergency Department Visits (Percent of total ED visits),TRUE,Percent of ED visits that had a discharge diagnosis code of rsv,Percent of ED visits that had a discharge diagnosis code of rsv,National Syndromic Surveillance Program,rsv,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
15791579

15801580
NSSP does not report county-level data for all counties with reporting EDs; some reporting EDs are only included in state-level values.
15811581

@@ -1596,14 +1596,14 @@ NSSP does not report county-level data for all counties with reporting EDs; some
15961596
The following states report no data through NSSP at the county level: CA, WA, AK, AZ, AL, CO, SD, ND, MO, AR, FL, OH, NH, CT, NJ.
15971597

15981598
South Dakota, Missouri, and territories report no data through NSSP at the state level.",Percentage,percent,other,bad,TRUE,FALSE,FALSE,FALSE,FALSE,,,,,,
1599-
nssp,pct_ed_visits_influenza,TRUE,smoothed_pct_ed_visits_influenza,FALSE,Influenza Emergency Department Visits (Percent of total ED visits),TRUE,3-week moving average of percent of ED visits that had a discharge diagnosis code of influenza,3-week moving average of percent of ED visits that had a discharge diagnosis code of influenza,National Syndromic Surveillance Program,flu,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
1599+
nssp,pct_ed_visits_influenza,TRUE,smoothed_pct_ed_visits_influenza,FALSE,"Influenza Emergency Department Visits (Percent of total ED visits, 3-week average)",TRUE,3-week moving average of percent of ED visits that had a discharge diagnosis code of influenza,3-week moving average of percent of ED visits that had a discharge diagnosis code of influenza,National Syndromic Surveillance Program,flu,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
16001600

16011601
NSSP does not report county-level data for all counties with reporting EDs; some reporting EDs are only included in state-level values.
16021602

16031603
The following states report no data through NSSP at the county level: CA, WA, AK, AZ, AL, CO, SD, ND, MO, AR, FL, OH, NH, CT, NJ.
16041604

16051605
South Dakota, Missouri, and territories report no data through NSSP at the state level.",Percentage,percent,other,bad,TRUE,FALSE,FALSE,FALSE,FALSE,,,,,,
1606-
nssp,pct_ed_visits_rsv,TRUE,smoothed_pct_ed_visits_rsv,FALSE,COVID Emergency Department Visits (Percent of total ED visits),TRUE,3-week moving average of percent of ED visits that had a discharge diagnosis code of rsv,3-week moving average of percent of ED visits that had a discharge diagnosis code of rsv,National Syndromic Surveillance Program,rsv,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
1606+
nssp,pct_ed_visits_rsv,TRUE,smoothed_pct_ed_visits_rsv,FALSE,"RSV Emergency Department Visits (Percent of total ED visits, 3-week average)",TRUE,3-week moving average of percent of ED visits that had a discharge diagnosis code of rsv,3-week moving average of percent of ED visits that had a discharge diagnosis code of rsv,National Syndromic Surveillance Program,rsv,Hospitalizations,USA,"county,state,hrr,msa","hrr,msa",2022-10-01,,ongoing,,week,Week,weekly,,,All,None,hospitalized,,"Data is available for 78% of US emergency departments. California, Colorado, Missouri, Oklahoma, and Virginia have the most noticeable gaps in coverage, with many counties in those states having either no eligible EDs or having no recently reported data in NSSP. However, most states have some counties that do not contain any reporting EDs.
16071607

16081608
NSSP does not report county-level data for all counties with reporting EDs; some reporting EDs are only included in state-level values.
16091609

0 commit comments

Comments
 (0)