Skip to content

Commit efd88e1

Browse files
authored
Add cubed notebook for hourly climatology example using "map-reduce" method (#356)
* Add cubed notebook for hourly climatology example using "map-reduce" method * Add cubed dependencies to docs build * Use Cubed version with HTML repr fix
1 parent 627bf2b commit efd88e1

File tree

3 files changed

+109
-0
lines changed

3 files changed

+109
-0
lines changed

ci/docs.yml

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ name: flox-doc
22
channels:
33
- conda-forge
44
dependencies:
5+
- cubed>=0.14.3
6+
- cubed-xarray
57
- dask-core
68
- pip
79
- xarray

docs/source/user-stories.md

+1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
user-stories/overlaps.md
88
user-stories/climatology.ipynb
99
user-stories/climatology-hourly.ipynb
10+
user-stories/climatology-hourly-cubed.ipynb
1011
user-stories/custom-aggregations.ipynb
1112
user-stories/nD-bins.ipynb
1213
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "0",
6+
"metadata": {},
7+
"source": [
8+
"# More climatology reductions using Cubed\n",
9+
"\n",
10+
"This is the Cubed equivalent of [More climatology reductions](climatology-hourly.ipynb).\n",
11+
"\n",
12+
"The task is to compute an hourly climatology from an hourly dataset with 744 hours in each chunk, using the \"map-reduce\" strategy."
13+
]
14+
},
15+
{
16+
"cell_type": "code",
17+
"execution_count": null,
18+
"id": "1",
19+
"metadata": {},
20+
"outputs": [],
21+
"source": [
22+
"import cubed\n",
23+
"import cubed.array_api as xp\n",
24+
"import numpy as np\n",
25+
"import pandas as pd\n",
26+
"import xarray as xr\n",
27+
"\n",
28+
"import flox.xarray"
29+
]
30+
},
31+
{
32+
"cell_type": "markdown",
33+
"id": "2",
34+
"metadata": {},
35+
"source": [
36+
"## Create data\n",
37+
"\n",
38+
"Note that we use fewer lat/long points so the computation can be run locally."
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"execution_count": null,
44+
"id": "3",
45+
"metadata": {},
46+
"outputs": [],
47+
"source": [
48+
"spec = cubed.Spec(allowed_mem=\"2GB\")\n",
49+
"ds = xr.Dataset(\n",
50+
" {\n",
51+
" \"tp\": (\n",
52+
" (\"time\", \"latitude\", \"longitude\"),\n",
53+
" xp.ones((8760, 72, 144), chunks=(744, 5, 144), dtype=np.float32, spec=spec),\n",
54+
" )\n",
55+
" },\n",
56+
" coords={\"time\": pd.date_range(\"2021-01-01\", \"2021-12-31 23:59\", freq=\"h\")},\n",
57+
")\n",
58+
"ds"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"id": "4",
64+
"metadata": {},
65+
"source": [
66+
"## Computation"
67+
]
68+
},
69+
{
70+
"cell_type": "code",
71+
"execution_count": null,
72+
"id": "5",
73+
"metadata": {},
74+
"outputs": [],
75+
"source": [
76+
"hourly = flox.xarray.xarray_reduce(ds.tp, ds.time.dt.hour, func=\"mean\", reindex=True)\n",
77+
"hourly"
78+
]
79+
},
80+
{
81+
"cell_type": "code",
82+
"execution_count": null,
83+
"id": "6",
84+
"metadata": {},
85+
"outputs": [],
86+
"source": [
87+
"hourly.compute()"
88+
]
89+
}
90+
],
91+
"metadata": {
92+
"language_info": {
93+
"codemirror_mode": {
94+
"name": "ipython",
95+
"version": 3
96+
},
97+
"file_extension": ".py",
98+
"mimetype": "text/x-python",
99+
"name": "python",
100+
"nbconvert_exporter": "python",
101+
"pygments_lexer": "ipython3"
102+
}
103+
},
104+
"nbformat": 4,
105+
"nbformat_minor": 5
106+
}

0 commit comments

Comments
 (0)