Skip to content

Commit 19b1f9a

Browse files
authored
Automatically parse DB bigints as JavaScript Ints (#83)
https://trello.com/c/RIATlpAS In the [charging-module-api](https://github.com/defra/charging-module-api) we hit an issue caused by integer overflow. We hold all our monetary values as pennies rather than pounds and pence. This is common practise in financial systems as it avoids some issues with rounding in equations. The issue was all our monetary fields were set as `integer` but we found examples of bill runs with values larger than 2147483647 (the max an integer can hold). To solve the issue we needed to set the DB fields to `bigint`. [Bigint's](https://www.postgresql.org/docs/10/datatype-numeric.html#DATATYPE-INT) can hold values in the range of -9223372036854775808 to +9223372036854775807 in PostgreSQL. JavaScript on the other hand uses [Number](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number) to manage integers. An integer in JavaScript has a range of -9007199254740991 to 9007199254740991. If you need to go as large as PostgreSQL you need to use a [JavaScript BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt). Because of this discrepancy, the PostgreSQL driver for Node by default returns all BigInt values as strings. This is a [design decision by PostgreSQL](brianc/node-postgres#353) to avoid any loss of precision. However, if you are confident you won't be affected by this precision loss (which we are) you can tell the PostgreSQL driver to return an integer instead. So as we do not expect to generate bill runs with values greater than 9.0071993e+13 (I don't even know how to say what that number is!) this change adds the config needed so we can forget about BigInt's in the rest of our code.
1 parent 7efcb7b commit 19b1f9a

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

db/index.js

+12
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,18 @@ const environment = process.env.NODE_ENV || 'development'
44

55
const dbConfig = require('../knexfile')[environment]
66

7+
// Some of our db fields are of type BigInt. The 'pg' driver by default will return these as strings because of concerns
8+
// about precision loss (a pg BigInt has a bigger range than a JavaScript integer). However, a JavaScript integer has
9+
// a range of -9007199254740991 to 9007199254740991. This is well, well within the bounds of what we need (assuming
10+
// we won't generate a bill for more than the national debt of the UK!).
11+
// So to avoid having to deal with converting strings to BigInt in our code we can tell the pg driver to parse BigInt's
12+
// as integers instead. See the following for more details about both the issue and this config change
13+
// https://github.com/brianc/node-postgres/pull/353
14+
// https://github.com/knex/knex/issues/387#issuecomment-51554522
15+
const pg = require('pg')
16+
// The magic number 20 comes from `SELECT oid FROM pg_type WHERE typname = 'int8';` and is unlikely to change.
17+
pg.types.setTypeParser(20, 'text', parseInt)
18+
719
const db = require('knex')(dbConfig)
820

921
module.exports = { db, dbConfig }

0 commit comments

Comments
 (0)