Skip to content

Update flusurv schema and docs with new age, sex, and race groups #1287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 8, 2024
Merged

Update flusurv schema and docs with new age, sex, and race groups #1287

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ed4dcf9
add new columns to flusurv endpoint spec
nmdefries Sep 15, 2023
73901c6
add new age, sex, race strata, and season to schema
nmdefries Sep 15, 2023
108a000
list new signals in documentation
nmdefries Sep 15, 2023
16dc0c3
move new season column to end of table
nmdefries Sep 18, 2023
9a481de
add migration script to add new fields to flusurv table
nmdefries Sep 18, 2023
3e5f5b6
add field descriptions; reformat table so don't need to match line le…
nmdefries Sep 27, 2023
cd9a46b
point from acquisition script to docs and schema def to deduplicate
nmdefries Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions docs/api/flusurv.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,22 +52,40 @@ If neither is specified, the current issues are used.

## Response

| Field | Description | Type |
|--------------------------|-----------------------------------------------------------------|------------------|
| `result` | result code: 1 = success, 2 = too many results, -2 = no results | integer |
| `epidata` | list of results | array of objects |
| `epidata[].release_date` | | string |
| `epidata[].location` | | string |
| `epidata[].issue` | | integer |
| `epidata[].epiweek` | | integer |
| `epidata[].lag` | | integer |
| `epidata[].rate_age_0` | | float |
| `epidata[].rate_age_1` | | float |
| `epidata[].rate_age_2` | | float |
| `epidata[].rate_age_3` | | float |
| `epidata[].rate_age_4` | | float |
| `epidata[].rate_overall` | | float |
| `message` | `success` or error message | string |
| Field | Description | Type |
|-------------------------------|-----------------------------------------------------------------|------------------|
| `result` | result code: 1 = success, 2 = too many results, -2 = no results | integer |
| `epidata` | list of results | array of objects |
| `epidata[].release_date` | | string |
| `epidata[].location` | | string |
| `epidata[].issue` | | integer |
| `epidata[].epiweek` | | integer |
| `epidata[].lag` | | integer |
| `epidata[].season` | | string |
| `epidata[].rate_age_0` | | float |
| `epidata[].rate_age_1` | | float |
| `epidata[].rate_age_2` | | float |
| `epidata[].rate_age_3` | | float |
| `epidata[].rate_age_4` | | float |
| `epidata[].rate_overall` | | float |
| `epidata[].rate_age_5` | | float |
| `epidata[].rate_age_6` | | float |
| `epidata[].rate_age_7` | | float |
| `epidata[].rate_age_18t29` | | float |
| `epidata[].rate_age_30t39` | | float |
| `epidata[].rate_age_40t49` | | float |
| `epidata[].rate_age_5t11` | | float |
| `epidata[].rate_age_12t17` | | float |
| `epidata[].rate_age_lt18` | | float |
| `epidata[].rate_age_gte18` | | float |
| `epidata[].rate_race_white` | | float |
| `epidata[].rate_race_black` | | float |
| `epidata[].rate_race_hisp` | | float |
| `epidata[].rate_race_asian` | | float |
| `epidata[].rate_race_natamer` | | float |
| `epidata[].rate_sex_male` | | float |
| `epidata[].rate_sex_female` | | float |
| `message` | `success` or error message | string |

Notes:
* The `flusurv` age groups are, in general, not the same as the ILINet
Expand Down
70 changes: 51 additions & 19 deletions src/acquisition/flusurv/flusurv_update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,32 +20,50 @@
=======================

`flusurv` is the table where US flu hospitalization rates are stored.
+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
+-------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| season | char(7) | YES | | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
| rate_age_18t29 | double | YES | | NULL | |
| rate_age_30t39 | double | YES | | NULL | |
| rate_age_40t49 | double | YES | | NULL | |
| rate_age_5t11 | double | YES | | NULL | |
| rate_age_12t17 | double | YES | | NULL | |
| rate_age_lt18 | double | YES | | NULL | |
| rate_age_gte18 | double | YES | | NULL | |
| rate_race_white | double | YES | | NULL | |
| rate_race_black | double | YES | | NULL | |
| rate_race_hisp | double | YES | | NULL | |
| rate_race_asian | double | YES | | NULL | |
| rate_race_natamer | double | YES | | NULL | |
| rate_sex_male | double | YES | | NULL | |
| rate_sex_female | double | YES | | NULL | |
+-------------------+-------------+------+-----+---------+----------------+
id: unique identifier for each record
release_date: the date when this record was first published by the CDC
issue: the epiweek of publication (e.g. issue 201453 includes epiweeks up to
and including 2014w53, but not 2015w01 or following)
epiweek: the epiweek during which the data was collected
location: the name of the catchment (e.g. 'network_all', 'CA', 'NY_albany')
lag: number of weeks between `epiweek` and `issue`
season: indicates the start and end years of the winter flu season in the
format YYYY-YY (e.g. 2022-23 indicates the flu season running late 2022
through early 2023)
rate_age_0: hospitalization rate for ages 0-4
rate_age_1: hospitalization rate for ages 5-17
rate_age_2: hospitalization rate for ages 18-49
Expand All @@ -55,6 +73,20 @@
rate_age_5: hospitalization rate for ages 65-74
rate_age_6: hospitalization rate for ages 75-84
rate_age_7: hospitalization rate for ages 85+
rate_age_18t29: hospitalization rate for ages 18 to 29
rate_age_30t39: hospitalization rate for ages 30 to 39
rate_age_40t49: hospitalization rate for ages 40 to 49
rate_age_5t11: hospitalization rate for ages 5 to 11
rate_age_12t17: hospitalization rate for ages 12 to 17
rate_age_lt18: hospitalization rate for ages <18
rate_age_gte18: hospitalization rate for ages >=18
rate_race_white: hospitalization rate for white people
rate_race_black: hospitalization rate for black people
rate_race_hisp: hospitalization rate for Hispanic/Latino people
rate_race_asian: hospitalization rate for Asian people
rate_race_natamer: hospitalization rate for American Indian/Alaskan Native people
rate_sex_male: hospitalization rate for males
rate_sex_female: hospitalization rate for females

=================
=== Changelog ===
Expand Down
70 changes: 50 additions & 20 deletions src/ddl/fluview.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ CREATE TABLE `fluview_public` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

/*
`flusurv` stores FluSurv-NET data (flu hospitaliation rates) as published by
`flusurv` stores FluSurv-NET data (flu hospitalization rates) as published by
CDC.

Data is public.
Expand All @@ -345,25 +345,40 @@ Note that the flusurv age groups are, in general, not the same as the ILINet
particular "catchment" (e.g. 'network_all', 'CA', 'NY_albany') rather than by
regions and states in general.

+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
+-------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| season | char(7) | YES | | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
| rate_age_18t29 | double | YES | | NULL | |
| rate_age_30t39 | double | YES | | NULL | |
| rate_age_40t49 | double | YES | | NULL | |
| rate_age_5t11 | double | YES | | NULL | |
| rate_age_12t17 | double | YES | | NULL | |
| rate_age_lt18 | double | YES | | NULL | |
| rate_age_gte18 | double | YES | | NULL | |
| rate_race_white | double | YES | | NULL | |
| rate_race_black | double | YES | | NULL | |
| rate_race_hisp | double | YES | | NULL | |
| rate_race_asian | double | YES | | NULL | |
| rate_race_natamer | double | YES | | NULL | |
| rate_sex_male | double | YES | | NULL | |
| rate_sex_female | double | YES | | NULL | |
+-------------------+-------------+------+-----+---------+----------------+
*/

CREATE TABLE `flusurv` (
Expand All @@ -373,6 +388,7 @@ CREATE TABLE `flusurv` (
`epiweek` int(11) NOT NULL,
`location` varchar(32) NOT NULL,
`lag` int(11) NOT NULL,
`season` char(7) DEFAULT NULL,
`rate_age_0` double DEFAULT NULL,
`rate_age_1` double DEFAULT NULL,
`rate_age_2` double DEFAULT NULL,
Expand All @@ -382,6 +398,20 @@ CREATE TABLE `flusurv` (
`rate_age_5` double DEFAULT NULL,
`rate_age_6` double DEFAULT NULL,
`rate_age_7` double DEFAULT NULL,
`rate_age_18t29` double DEFAULT NULL,
`rate_age_30t39` double DEFAULT NULL,
`rate_age_40t49` double DEFAULT NULL,
`rate_age_5t11` double DEFAULT NULL,
`rate_age_12t17` double DEFAULT NULL,
`rate_age_lt18` double DEFAULT NULL,
`rate_age_gte18` double DEFAULT NULL,
`rate_race_white` double DEFAULT NULL,
`rate_race_black` double DEFAULT NULL,
`rate_race_hisp` double DEFAULT NULL,
`rate_race_asian` double DEFAULT NULL,
`rate_race_natamer` double DEFAULT NULL,
`rate_sex_male` double DEFAULT NULL,
`rate_sex_female` double DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `issue` (`issue`,`epiweek`,`location`),
KEY `release_date` (`release_date`),
Expand Down
23 changes: 22 additions & 1 deletion src/server/endpoints/flusurv.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def handle():
# basic query info
q = QueryBuilder("flusurv", "fs")

fields_string = ["release_date", "location"]
fields_string = ["release_date", "location", "season"]
fields_int = ["issue", "epiweek", "lag"]
fields_float = [
"rate_age_0",
Expand All @@ -28,6 +28,27 @@ def handle():
"rate_age_3",
"rate_age_4",
"rate_overall",

"rate_age_5",
"rate_age_6",
"rate_age_7",

"rate_age_18t29",
"rate_age_30t39",
"rate_age_40t49",
"rate_age_5t11",
"rate_age_12t17",
"rate_age_lt18",
"rate_age_gte18",

"rate_race_white",
"rate_race_black",
"rate_race_hisp",
"rate_race_asian",
"rate_race_natamer",

"rate_sex_male",
"rate_sex_female",
]
q.set_fields(fields_string, fields_int, fields_float)
q.set_sort_order("epiweek", "location", "issue")
Expand Down