Skip to content

Commit 44dde17

Browse files
cbaker6cbaker6
authored
Added AES encryption/decryption using native crypto (#15)
* add native AES file encryption * removed update of package.json * update package.json * update node in travis * update node in travis * trying different verion of key and iv hash * downgrading node to match minimal parse-server * removed commented code * add random iv for each file instead of using constant. Also removed encrypted option, will encrypt automatically if secretKey is provided * remove unneccesary files * Use AES 256 GCM to detect file tampering * remove codecov from package.json * add repo field to get rid of npm install warning * Fix options * switch secretKey to fileKey * added the ability to rotate fileKeys * add syntax highlighting to readme * bump version * attempt to fix coverage * update testcase title * clean up unused vars * add directions for multiple instances of parse-server * update readme * update file names in readme * add testcase for rotating key from oldKey to noKey leaving all files decrypted * Add notice about previous versions of parse-server * Update README.md * Update README.md * Update README.md * Update README.md * make createFile and getFile use streams instead of putting whole file in memory * don't read file into memory while deleting * clean up code * make more consistant with GridFS adapter * fixed formatting * Update .travis.yml * Remove unnecessary testcase The test is already covered in parse-server-conformance-tests * Update secureFiles.spec.js * add directions for dev server to readme * Revert version * Update package.json * Update .travis.yml
1 parent 9cd39af commit 44dde17

File tree

8 files changed

+501
-39
lines changed

8 files changed

+501
-39
lines changed

‎.nycrc

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"reporter": [
3+
"lcov",
4+
"text-summary"
5+
],
6+
"exclude": [
7+
"**/spec/**"
8+
]
9+
}

‎.travis.yml

+6-3
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ branches:
22
only:
33
- master
44
language: node_js
5-
node_js:
6-
- "4.3"
7-
after_success: ./node_modules/.bin/codecov
5+
node_js: '10'
6+
env:
7+
global:
8+
- COVERAGE_OPTION='./node_modules/.bin/nyc'
9+
script: npm run coverage
10+
after_success: bash <(curl -s https://codecov.io/bash)

‎README.md

+60-9
Original file line numberDiff line numberDiff line change
@@ -2,44 +2,95 @@
22
[![Build Status](https://travis-ci.org/parse-community/parse-server-fs-adapter.svg?branch=master)](https://travis-ci.org/parse-community/parse-server-fs-adapter)
33
[![codecov.io](https://codecov.io/github/parse-community/parse-server-fs-adapter/coverage.svg?branch=master)](https://codecov.io/github/parse-community/parse-server-fs-adapter?branch=master)
44

5-
parse-server file system storage adapter
5+
parse-server file system storage adapter.
66

77

8-
# installation
8+
# Multiple instances of parse-server
9+
When using parse-server-fs-adapter across multiple parse-server instances it's important to establish "centralization" of your file storage (this is the same premise as the other file adapters, you are sending/recieving files through a dedicated link). You can accomplish this at the file storage level by Samba mounting (or any other type of mounting) your storage to each of your parse-server instances, e.g if you are using parse-server via docker (volume mount your SMB drive to `- /Volumes/SMB-Drive/MyParseApp1/files:/parse-server/files`). All parse-server instances need to be able to read and write to the same storage in order for parse-server-fs-adapter to work properly with parse-server. If the file storage isn't centralized, parse-server will have trouble locating files and you will get random behavior on client-side.
10+
11+
# Installation
912

1013
`npm install --save @parse/fs-files-adapter`
1114

12-
# usage with parse-server
15+
# Usage with parse-server
1316

14-
### using a config file
17+
### Using a config file
1518

16-
```
19+
```javascript
1720
{
1821
"appId": 'my_app_id',
1922
"masterKey": 'my_master_key',
2023
// other options
2124
"filesAdapter": {
2225
"module": "@parse/fs-files-adapter",
2326
"options": {
24-
"filesSubDirectory": "my/files/folder" // optional
27+
"filesSubDirectory": "my/files/folder", // optional
28+
"fileKey": "someKey" //optional, but mandatory if you want to encrypt files
2529
}
2630
}
2731
}
2832
```
2933

30-
### passing as an instance
34+
### Passing as an instance
35+
***Notice: If used with parse-server versions <= 4.2.0, DO NOT PASS in `PARSE_SERVER_FILE_KEY` or `fileKey` from parse-server. Instead pass your key directly to `FSFilesAdapter` using your own environment variable or hardcoding the string. parse-server versions > 4.2.0 can pass in `PARSE_SERVER_FILE_KEY` or `fileKey`.***
36+
37+
```javascript
38+
var FSFilesAdapter = require('@parse/fs-files-adapter');
39+
40+
var fsAdapter = new FSFilesAdapter({
41+
"filesSubDirectory": "my/files/folder", // optional
42+
"fileKey": "someKey" //optional, but mandatory if you want to encrypt files
43+
});
3144

45+
var api = new ParseServer({
46+
appId: 'my_app',
47+
masterKey: 'master_key',
48+
filesAdapter: fsAdapter
49+
})
3250
```
51+
52+
### Rotating to a new fileKey
53+
Periodically you may want to rotate your fileKey for security reasons. When this is the case, you can start up a development parse-server that has the same configuration as your production server. In the development server, initialize the file adapter with the new key and do the following in your `index.js`:
54+
55+
#### Files were previously unencrypted and you want to encrypt
56+
```javascript
3357
var FSFilesAdapter = require('@parse/fs-files-adapter');
3458

3559
var fsAdapter = new FSFilesAdapter({
36-
"filesSubDirectory": "my/files/folder" // optional
37-
});
60+
"filesSubDirectory": "my/files/folder", // optional
61+
"fileKey": "newKey" //Use the newKey
62+
});
3863

3964
var api = new ParseServer({
4065
appId: 'my_app',
4166
masterKey: 'master_key',
4267
filesAdapter: fsAdapter
4368
})
69+
70+
//This can take awhile depending on how many files and how larger they are. It will attempt to rotate the key of all files in your filesSubDirectory
71+
//It is not recommended to do this on the production server, deploy a development server to complete the process.
72+
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey();
73+
console.log('Files rotated to newKey: ' + rotated);
74+
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
4475
```
4576
77+
After successfully rotating your key, you should change the `fileKey` to `newKey` on your production server and then restart the server.
78+
79+
80+
#### Files were previously encrypted with `oldKey` and you want to encrypt with `newKey`
81+
The same process as above, but pass in your `oldKey` to `rotateFileKey()`.
82+
```javascript
83+
//This can take awhile depending on how many files and how larger they are. It will attempt to rotate the key of all files in your filesSubDirectory
84+
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey});
85+
console.log('Files rotated to newKey: ' + rotated);
86+
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
87+
```
88+
89+
#### Only rotate a select list of files that were previously encrypted with `oldKey` and you want to encrypt with `newKey`
90+
This is useful if for some reason there errors and some of the files werent rotated and returned in `notRotated`. The same process as above, but pass in your `oldKey` along with the array of `fileNames` to `rotateFileKey()`.
91+
```javascript
92+
//This can take awhile depending on how many files and how larger they are. It will attempt to rotate the key of all files in your filesSubDirectory
93+
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey, fileNames: ["fileName1.png","fileName2.png"]});
94+
console.log('Files rotated to newKey: ' + rotated);
95+
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
96+
```

‎index.js

+130-20
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,16 @@
77
var fs = require('fs');
88
var path = require('path');
99
var pathSep = require('path').sep;
10+
const crypto = require("crypto");
11+
const algorithm = 'aes-256-gcm';
1012

1113
function FileSystemAdapter(options) {
1214
options = options || {};
15+
this._fileKey = null;
16+
17+
if (options.fileKey !== undefined){
18+
this._fileKey = crypto.createHash('sha256').update(String(options.fileKey)).digest('base64').substr(0, 32);
19+
}
1320
let filesSubDirectory = options.filesSubDirectory || '';
1421
this._filesDir = filesSubDirectory;
1522
this._mkdir(this._getApplicationDir());
@@ -19,44 +26,147 @@ function FileSystemAdapter(options) {
1926
}
2027

2128
FileSystemAdapter.prototype.createFile = function(filename, data) {
29+
let filepath = this._getLocalFilePath(filename);
30+
const stream = fs.createWriteStream(filepath);
2231
return new Promise((resolve, reject) => {
23-
let filepath = this._getLocalFilePath(filename);
24-
fs.writeFile(filepath, data, (err) => {
25-
if(err !== null) {
26-
return reject(err);
27-
}
28-
resolve(data);
29-
});
32+
try{
33+
if(this._fileKey !== null){
34+
const iv = crypto.randomBytes(16);
35+
const cipher = crypto.createCipheriv(
36+
algorithm,
37+
this._fileKey,
38+
iv
39+
);
40+
const encryptedResult = Buffer.concat([
41+
cipher.update(data),
42+
cipher.final(),
43+
iv,
44+
cipher.getAuthTag(),
45+
]);
46+
stream.write(encryptedResult);
47+
stream.end();
48+
stream.on('finish', function() {
49+
resolve(data);
50+
});
51+
}else{
52+
stream.write(data);
53+
stream.end();
54+
stream.on('finish', function() {
55+
resolve(data);
56+
});
57+
}
58+
}catch(err){
59+
return reject(err);
60+
}
3061
});
3162
}
3263

3364
FileSystemAdapter.prototype.deleteFile = function(filename) {
65+
let filepath = this._getLocalFilePath(filename);
66+
const chunks = [];
67+
const stream = fs.createReadStream(filepath);
3468
return new Promise((resolve, reject) => {
35-
let filepath = this._getLocalFilePath(filename);
36-
fs.readFile( filepath , function (err, data) {
37-
if(err !== null) {
38-
return reject(err);
39-
}
40-
fs.unlink(filepath, (unlinkErr) => {
41-
if(err !== null) {
42-
return reject(unlinkErr);
69+
stream.read();
70+
stream.on('data', (data) => {
71+
chunks.push(data);
72+
});
73+
stream.on('end', () => {
74+
const data = Buffer.concat(chunks);
75+
fs.unlink(filepath, (err) => {
76+
if(err !== null) {
77+
return reject(err);
4378
}
4479
resolve(data);
4580
});
4681
});
47-
82+
stream.on('error', (err) => {
83+
reject(err);
84+
});
4885
});
4986
}
5087

5188
FileSystemAdapter.prototype.getFileData = function(filename) {
89+
let filepath = this._getLocalFilePath(filename);
90+
const stream = fs.createReadStream(filepath);
91+
stream.read();
5292
return new Promise((resolve, reject) => {
53-
let filepath = this._getLocalFilePath(filename);
54-
fs.readFile( filepath , function (err, data) {
55-
if(err !== null) {
56-
return reject(err);
93+
const chunks = [];
94+
stream.on('data', (data) => {
95+
chunks.push(data);
96+
});
97+
stream.on('end', () => {
98+
const data = Buffer.concat(chunks);
99+
if(this._fileKey !== null){
100+
const authTagLocation = data.length - 16;
101+
const ivLocation = data.length - 32;
102+
const authTag = data.slice(authTagLocation);
103+
const iv = data.slice(ivLocation,authTagLocation);
104+
const encrypted = data.slice(0,ivLocation);
105+
try{
106+
const decipher = crypto.createDecipheriv(algorithm, this._fileKey, iv);
107+
decipher.setAuthTag(authTag);
108+
const decrypted = Buffer.concat([decipher.update(encrypted), decipher.final()]);
109+
return resolve(decrypted);
110+
}catch(err){
111+
return reject(err);
112+
}
57113
}
58114
resolve(data);
59115
});
116+
stream.on('error', (err) => {
117+
reject(err);
118+
});
119+
});
120+
}
121+
122+
FileSystemAdapter.prototype.rotateFileKey = function(options = {}) {
123+
const applicationDir = this._getApplicationDir();
124+
var fileNames = [];
125+
var oldKeyFileAdapter = {};
126+
if (options.oldKey !== undefined) {
127+
oldKeyFileAdapter = new FileSystemAdapter({filesSubDirectory: this._filesDir, fileKey: options.oldKey});
128+
}else{
129+
oldKeyFileAdapter = new FileSystemAdapter({filesSubDirectory: this._filesDir});
130+
}
131+
if (options.fileNames !== undefined){
132+
fileNames = options.fileNames;
133+
}else{
134+
fileNames = fs.readdirSync(applicationDir);
135+
fileNames = fileNames.filter(fileName => fileName.indexOf('.') !== 0);
136+
}
137+
return new Promise((resolve, _reject) => {
138+
var fileNamesNotRotated = fileNames;
139+
var fileNamesRotated = [];
140+
var fileNameTotal = fileNames.length;
141+
var fileNameIndex = 0;
142+
fileNames.forEach(fileName => {
143+
oldKeyFileAdapter
144+
.getFileData(fileName)
145+
.then(plainTextData => {
146+
//Overwrite file with data encrypted with new key
147+
this.createFile(fileName, plainTextData)
148+
.then(() => {
149+
fileNamesRotated.push(fileName);
150+
fileNamesNotRotated = fileNamesNotRotated.filter(function(value){ return value !== fileName;})
151+
fileNameIndex += 1;
152+
if (fileNameIndex == fileNameTotal){
153+
resolve({rotated: fileNamesRotated, notRotated: fileNamesNotRotated});
154+
}
155+
})
156+
.catch(() => {
157+
fileNameIndex += 1;
158+
if (fileNameIndex == fileNameTotal){
159+
resolve({rotated: fileNamesRotated, notRotated: fileNamesNotRotated});
160+
}
161+
})
162+
})
163+
.catch(() => {
164+
fileNameIndex += 1;
165+
if (fileNameIndex == fileNameTotal){
166+
resolve({rotated: fileNamesRotated, notRotated: fileNamesNotRotated});
167+
}
168+
});
169+
});
60170
});
61171
}
62172

‎package.json

+9-5
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,15 @@
11
{
22
"name": "@parse/fs-files-adapter",
3-
"version": "1.0.1",
3+
"version": "1.0.2",
44
"description": "File system adapter for parse-server",
55
"main": "index.js",
6+
"repository": {
7+
"type": "git",
8+
"url": "https://github.com/parse-community/parse-server-fs-adapter"
9+
},
610
"scripts": {
7-
"test": "istanbul cover -x **/spec/** jasmine --captureExceptions"
11+
"test": "jasmine",
12+
"coverage": "nyc jasmine"
813
},
914
"keywords": [
1015
"parse-server",
@@ -15,9 +20,8 @@
1520
"author": "Parse",
1621
"license": "MIT",
1722
"devDependencies": {
18-
"codecov": "^1.0.1",
19-
"istanbul": "^0.4.2",
20-
"jasmine": "^2.4.1",
23+
"nyc": "^15.1.0",
24+
"jasmine": "^3.5.0",
2125
"parse-server-conformance-tests": "^1.0.0"
2226
}
2327
}

0 commit comments

Comments
 (0)