Skip to content

Commit c08fa4a

Browse files
webbugtpieh
andauthored
feat(gatsby-source-filesystem): allow adjusting createRemoteFileNode retry/timeout settings via env vars (#24535)
* (gatsby-source-filesystem) Remote File Node env variables For developers with awful internet connection (like me), when using createRemoteFileNode, default timeouts of 30000ms are too short. Out of 300 files, it'd fail 1-4 images due to timeout and mess up the build process. Thus, it would be useful to overwrite default remote file download settings (retry limits and timeouts) with .env variables. For my specific case, just doubling all the values solved all issues. * Update create-remote-file-node.js removed ";"s * updated readme * update Co-authored-by: Michal Piechowiak <[email protected]>
1 parent e8ec13b commit c08fa4a

File tree

2 files changed

+11
-3
lines changed

2 files changed

+11
-3
lines changed

packages/gatsby-source-filesystem/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -335,3 +335,11 @@ function createMySqlNodes({ name, __sql, idField, keys }, results, ctx) {
335335

336336
module.exports = createMySqlNodes
337337
```
338+
339+
## Troubleshooting
340+
341+
In case that due to spotty network, or slow connection, some remote files fail to download. Even after multiple retries and adjusting concurrent downloads, you can adjust timeout and retry settings with these environment variables:
342+
343+
- `STALL_RETRY_LIMIT`, default: `3`
344+
- `STALL_TIMEOUT`, default: `30000`
345+
- `CONNECTION_TIMEOUT`, default: `30000`

packages/gatsby-source-filesystem/src/create-remote-file-node.js

+3-3
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,10 @@ let totalJobs = 0
5555
* @param {Reporter} [options.reporter]
5656
*/
5757

58-
const STALL_RETRY_LIMIT = 3
59-
const STALL_TIMEOUT = 30000
58+
const STALL_RETRY_LIMIT = process.env.GATSBY_STALL_RETRY_LIMIT || 3
59+
const STALL_TIMEOUT = process.env.GATSBY_STALL_TIMEOUT || 30000
6060

61-
const CONNECTION_TIMEOUT = 30000
61+
const CONNECTION_TIMEOUT = process.env.GATSBY_CONNECTION_TIMEOUT || 30000
6262

6363
/********************
6464
* Queue Management *

0 commit comments

Comments
 (0)