Skip to content

[LiveQuery]RangeError: Invalid WebSocket frame: invalid payload length 126 #6413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JeffGuKang opened this issue Feb 15, 2020 · 10 comments · Fixed by #6418
Open

[LiveQuery]RangeError: Invalid WebSocket frame: invalid payload length 126 #6413

JeffGuKang opened this issue Feb 15, 2020 · 10 comments · Fixed by #6418

Comments

@JeffGuKang
Copy link
Contributor

JeffGuKang commented Feb 15, 2020

Related to #6173

Issue Description

RangeError: Invalid WebSocket frame: invalid payload length 126
This error occurs intermittently and stops the server when I use livequery.

Steps to reproduce

  1. Run livequery by Parse Server
  2. Use livequery
  3. It happens irregularly

Expected Results

Parse Server should not terminate by the error.

Actual Outcome

Parse Server is terminated.

RangeError: Invalid WebSocket frame: invalid payload length 126
    at Receiver.getInfo (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:213:16)
    at Receiver.startLoop (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:127:22)
    at Receiver._write (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:74:10)
    at doWrite (_stream_writable.js:435:12)
    at writeOrBuffer (_stream_writable.js:419:5)
    at Receiver.Writable.write (_stream_writable.js:309:11)
    at Socket.socketOnData (/var/app/current/node_modules/parse-server/node_modules/ws/lib/websocket.js:875:35)
    at Socket.emit (events.js:223:5)
    at Socket.EventEmitter.emit (domain.js:475:20)
    at addChunk (_stream_readable.js:309:12)
/var/app/current/node_modules/parse-server/lib/ParseServer.js:229
          throw err;
          ^

RangeError: Invalid WebSocket frame: invalid payload length 126
    at Receiver.getInfo (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:213:16)
    at Receiver.startLoop (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:127:22)
    at Receiver._write (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:74:10)
    at doWrite (_stream_writable.js:435:12)
    at writeOrBuffer (_stream_writable.js:419:5)
    at Receiver.Writable.write (_stream_writable.js:309:11)
    at Socket.socketOnData (/var/app/current/node_modules/parse-server/node_modules/ws/lib/websocket.js:875:35)
    at Socket.emit (events.js:223:5)
    at Socket.EventEmitter.emit (domain.js:475:20)
    at addChunk (_stream_readable.js:309:12)
Emitted 'error' event on WebSocket instance at:
    at Receiver.receiverOnError (/var/app/current/node_modules/parse-server/node_modules/ws/lib/websocket.js:780:13)
    at Receiver.emit (events.js:223:5)
    at Receiver.EventEmitter.emit (domain.js:475:20)
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at onwriteError (_stream_writable.js:450:5)
    at onwrite (_stream_writable.js:477:5)
    at Receiver.startLoop (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:148:5)
    at Receiver._write (/var/app/current/node_modules/parse-server/node_modules/ws/lib/receiver.js:74:10)
    at doWrite (_stream_writable.js:435:12)
    at writeOrBuffer (_stream_writable.js:419:5) {
  [Symbol(status-code)]: 1002
}

Environment Setup

  • Server

    • NodeJS version: [12.14.1]
    • parse-server version (Be specific! Don't say 'latest'.) : [3.10.0]
    • Operating System: [Mac and Linux]
    • Hardware: []
    • Localhost or remote server?: [Local and AWS]
  • Database

    • PostgreSQL version: [11.4 ~ 11.5]
    • Storage engine: [RDS]
    • Hardware: []
    • Localhost or remote server? (AWS, mLab, ObjectRocket, Digital Ocean, etc): [Local and AWS]
@dplewis
Copy link
Member

dplewis commented Feb 16, 2020

Is there a way you can view the frames or debug the socket? What client are you using?

@JeffGuKang
Copy link
Contributor Author

JeffGuKang commented Feb 16, 2020

Is there a way you can view the frames or debug the socket? What client are you using?

I use default socket on parse server.

const api = new ParseServer({
  databaseURI,
  cloud: __dirname + '/src/cloud/', // location after tsc
  appId: process.env.APP_ID,
  masterKey: process.env.MASTER_KEY,

  serverURL: 'http://localhost:1337/parse',
  // enableAnonymousUsers: true,
  liveQuery: {
    classNames: ['A', 'B', 'C', 'D'],
  },
  auth: {    apple: {
      client_id: process.env.APPLE_SIGNIN_CLIENT_ID, // optional (for extra validation), use the Service ID from Apple.
    },
  },
  maxUploadSize: '5mb',
  logLevel: process.env.NODE_ENV !== 'local' ? 'warn' : undefined,
})

And I did not debug frames of the socket. I will share it after digging.

@dplewis
Copy link
Member

dplewis commented Feb 17, 2020

At least we can prevent the server from crashing when an error occurs, even if we can't replicate it.

dplewis added a commit that referenced this issue Feb 18, 2020
Closes: #6413, #6173
Prevent crashing on websocket error.

Bonus points to anybody who can post a specific payload that the client sends that returns an error.
davimacedo pushed a commit that referenced this issue Feb 19, 2020
* Websocket: unhandle rejection

Closes: #6413, #6173
Prevent crashing on websocket error.

Bonus points to anybody who can post a specific payload that the client sends that returns an error.

* log the socket

* fix tests

* fix payload reference link
@JeffGuKang
Copy link
Contributor Author

JeffGuKang commented Mar 2, 2020

@dplewis
I got a error message through the latest PR(#6418)

error: Invalid WebSocket frame: invalid payload length 126
error: {
  "_events": {
    "close": [
      null,
      null
    ]
  },
  "_eventsCount": 3,
  "readyState": 2,
  "protocol": "",
  "_binaryType": "nodebuffer",
  "_closeFrameReceived": false,
  "_closeFrameSent": false,
  "_closeMessage": "",
  "_closeTimer": null,
  "_closeCode": 1002,
  "_extensions": {},
  "_receiver": {
    "_writableState": {
      "objectMode": false,
      "highWaterMark": 16384,
      "finalCalled": false,
      "needDrain": false,
      "ending": false,
      "ended": false,
      "finished": false,
      "destroyed": false,
      "decodeStrings": true,
      "defaultEncoding": "utf8",
      "length": 0,
      "writing": false,
      "corked": 0,
      "sync": true,
      "bufferProcessing": false,
      "writecb": null,
      "writelen": 0,
      "afterWriteTickInfo": null,
      "bufferedRequest": null,
      "lastBufferedRequest": null,
      "pendingcb": 0,
      "prefinished": false,
      "errorEmitted": true,
      "emitClose": true,
      "autoDestroy": false,
      "bufferedRequestCount": 0,
      "corkedRequestsFree": {
        "next": null,
        "entry": null
      }
    },
    "writable": true,
    "_events": {},
    "_eventsCount": 6,
    "_binaryType": "nodebuffer",
    "_extensions": {},
    "_maxPayload": 104857600,
    "_bufferedBytes": 650,
    "_buffers": [
      {
        "type": "Buffer",
        "data": [
          2, 
...(VERY LONG),
          125
        ]
      }
    ],
    "_compressed": false,
    "_payloadLength": 126,
    "_mask": {
      "type": "Buffer",
      "data": [
        84,
        126,
        124,
        222
      ]
    },
    "_fragmented": 0,
    "_masked": true,
    "_fin": true,
    "_opcode": 10,
    "_totalPayloadLength": 0,
    "_messageLength": 0,
    "_fragments": [],
    "_state": 0,
    "_loop": false
  },
  "_sender": {
    "_extensions": {},
    "_socket": {
      "connecting": false,
      "_hadError": false,
      "_parent": null,
      "_host": null,
      "_readableState": {
        "objectMode": false,
        "highWaterMark": 16384,
        "buffer": {
          "head": null,
          "tail": null,
          "length": 0
        },
        "length": 0,
        "pipes": null,
        "pipesCount": 0,
        "flowing": true,
        "ended": false,
        "endEmitted": false,
        "reading": false,
        "sync": false,
        "needReadable": true,
        "emittedReadable": false,
        "readableListening": false,
        "resumeScheduled": false,
        "paused": false,
        "emitClose": false,
        "autoDestroy": false,
        "destroyed": false,
        "defaultEncoding": "utf8",
        "awaitDrain": 0,
        "readingMore": false,
        "decoder": null,
        "encoding": null
      },
      "readable": true,
      "_events": {
        "end": [
          null,
          null
        ]
      },
      "_eventsCount": 4,
      "_writableState": {
        "objectMode": false,
        "highWaterMark": 16384,
        "finalCalled": false,
        "needDrain": false,
        "ending": false,
        "ended": false,
        "finished": false,
        "destroyed": false,
        "decodeStrings": false,
        "defaultEncoding": "utf8",
        "length": 0,
        "writing": false,
        "corked": 0,
        "sync": false,
        "bufferProcessing": false,
        "writecb": null,
        "writelen": 0,
        "afterWriteTickInfo": null,
        "bufferedRequest": null,
        "lastBufferedRequest": null,
        "pendingcb": 0,
        "prefinished": false,
        "errorEmitted": false,
        "emitClose": false,
        "autoDestroy": false,
        "bufferedRequestCount": 0,
        "corkedRequestsFree": {
          "next": {
            "next": null,
            "entry": null
          },
          "entry": null
        }
      },
      "writable": true,
      "allowHalfOpen": true,
      "_sockname": null,
      "_pendingData": null,
      "_pendingEncoding": "",
      "server": {
        "_events": {},
        "_eventsCount": 5,
        "_connections": 532,
        "_handle": {},
        "_usingWorkers": false,
        "_workers": [],
        "_unref": false,
        "allowHalfOpen": true,
        "pauseOnConnect": false,
        "httpAllowHalfOpen": false,
        "timeout": 120000,
        "keepAliveTimeout": 5000,
        "maxHeadersCount": null,
        "headersTimeout": 40000,
        "_connectionKey": "4:null:1337"
      },
      "_server": {
        "_events": {},
        "_eventsCount": 5,
        "_connections": 532,
        "_handle": {},
        "_usingWorkers": false,
        "_workers": [],
        "_unref": false,
        "allowHalfOpen": true,
        "pauseOnConnect": false,
        "httpAllowHalfOpen": false,
        "timeout": 120000,
        "keepAliveTimeout": 5000,
        "maxHeadersCount": null,
        "headersTimeout": 40000,
        "_connectionKey": "4:null:1337"
      },
      "timeout": 0,
      "parser": null,
      "_paused": false,
      "_httpMessage": null,
      "_peername": {
        "address": "::ffff:127.0.0.1",
        "family": "IPv6",
        "port": 54414
      }
    },
    "_firstFragment": true,
    "_compress": false,
    "_bufferedBytes": 0,
    "_deflating": false,
    "_queue": []
  },
  "_socket": {
    "connecting": false,
    "_hadError": false,
    "_parent": null,
    "_host": null,
    "_readableState": {
      "objectMode": false,
      "highWaterMark": 16384,
      "buffer": {
        "head": null,
        "tail": null,
        "length": 0
      },
      "length": 0,
      "pipes": null,
      "pipesCount": 0,
      "flowing": true,
      "ended": false,
      "endEmitted": false,
      "reading": false,
      "sync": false,
      "needReadable": true,
      "emittedReadable": false,
      "readableListening": false,
      "resumeScheduled": false,
      "paused": false,
      "emitClose": false,
      "autoDestroy": false,
      "destroyed": false,
      "defaultEncoding": "utf8",
      "awaitDrain": 0,
      "readingMore": false,
      "decoder": null,
      "encoding": null
    },
    "readable": true,
    "_events": {
      "end": [
        null,
        null
      ]
    },
    "_eventsCount": 4,
    "_writableState": {
      "objectMode": false,
      "highWaterMark": 16384,
      "finalCalled": false,
      "needDrain": false,
      "ending": false,
      "ended": false,
      "finished": false,
      "destroyed": false,
      "decodeStrings": false,
      "defaultEncoding": "utf8",
      "length": 0,
      "writing": false,
      "corked": 0,
      "sync": false,
      "bufferProcessing": false,
      "writecb": null,
      "writelen": 0,
      "afterWriteTickInfo": null,
      "bufferedRequest": null,
      "lastBufferedRequest": null,
      "pendingcb": 0,
      "prefinished": false,
      "errorEmitted": false,
      "emitClose": false,
      "autoDestroy": false,
      "bufferedRequestCount": 0,
      "corkedRequestsFree": {
        "next": {
          "next": null,
          "entry": null
        },
        "entry": null
      }
    },
    "writable": true,
    "allowHalfOpen": true,
    "_sockname": null,
    "_pendingData": null,
    "_pendingEncoding": "",
    "server": {
      "_events": {},
      "_eventsCount": 5,
      "_connections": 532,
      "_handle": {},
      "_usingWorkers": false,
      "_workers": [],
      "_unref": false,
      "allowHalfOpen": true,
      "pauseOnConnect": false,
      "httpAllowHalfOpen": false,
      "timeout": 120000,
      "keepAliveTimeout": 5000,
      "maxHeadersCount": null,
      "headersTimeout": 40000,
      "_connectionKey": "4:null:1337"
    },
    "_server": {
      "_events": {},
      "_eventsCount": 5,
      "_connections": 532,
      "_handle": {},
      "_usingWorkers": false,
      "_workers": [],
      "_unref": false,
      "allowHalfOpen": true,
      "pauseOnConnect": false,
      "httpAllowHalfOpen": false,
      "timeout": 120000,
      "keepAliveTimeout": 5000,
      "maxHeadersCount": null,
      "headersTimeout": 40000,
      "_connectionKey": "4:null:1337"
    },
    "timeout": 0,
    "parser": null,
    "_paused": false,
    "_httpMessage": null,
    "_peername": {
      "address": "::ffff:127.0.0.1",
      "family": "IPv6",
      "port": 54414
    }
  },
  "_isServer": true
}

@dplewis
Copy link
Member

dplewis commented Mar 2, 2020

Thanks for the feedback. Did your server crash?

We can try to use the following to reverse engineer the payload that is failing. Knowing _buffers.data would be nice, I don't know if would help.

Also we should show the following for VERBOSE=1.

"_receiver": {
    "_writableState": {
      ...
    },
    "writable": true,
    "_events": {},
    "_eventsCount": 6,
    "_binaryType": "nodebuffer",
    "_extensions": {},
    "_maxPayload": 104857600,
    "_bufferedBytes": 650,
    "_buffers": [
      {
        "type": "Buffer",
        "data": [
          2, 
...(VERY LONG),
          125
        ]
      }
    ],
    "_compressed": false,
    "_payloadLength": 126,
    "_mask": {
      "type": "Buffer",
      "data": [
        84,
        126,
        124,
        222
      ]
    },
    "_fragmented": 0,
    "_masked": true,
    "_fin": true,
    "_opcode": 10,
    "_totalPayloadLength": 0,
    "_messageLength": 0,
    "_fragments": [],
    "_state": 0,
    "_loop": false
  },

@JeffGuKang
Copy link
Contributor Author

JeffGuKang commented Mar 3, 2020

@dplewis
My EC2 server did not crash anymore in v4.0.x. 😄
But sometimes the EC2's CPU load reached 100%. I am not sure it is related to this issue or not. (parse server v4.0.1). I will rollback and check my server with v3.10.0 again today.

I will attach the more samples to files.

nodejserror.json.zip

UnderratedDev pushed a commit to UnderratedDev/parse-server that referenced this issue Mar 21, 2020
* Websocket: unhandle rejection

Closes: parse-community#6413, parse-community#6173
Prevent crashing on websocket error.

Bonus points to anybody who can post a specific payload that the client sends that returns an error.

* log the socket

* fix tests

* fix payload reference link
@majidhassan
Copy link

Hello, I'm getting a similar issue with the livequery server (parse server & parse livequery server versions are 2.8.4 - yes, I know quite outdated XD )

I'm connecting to the livequery server from my Angular components and I sometimes get this error below, which then crashes the server and sends it into a restart loop since pm2 attempts to restart, which could last anywhere between 5 minutes to an hour after which it resolves by itself.

Any ideas what this error means and how to solve it, or at least how to stop it from crashing the server without having to upgrade to a more recent parse-server version for the livequery server?

If not, any idea if it's possible to run different versions of parse for the main server and the livequery server?

RangeError: Invalid WebSocket frame: RSV1 must be clear
    at Receiver.getInfo (/home/parse/node_modules/parse-server/node_modules/ws/lib/receiver.js:194:9)
    at Receiver.startLoop (/home/parse/node_modules/parse-server/node_modules/ws/lib/receiver.js:152:16)
    at Receiver.add (/home/parse/node_modules/parse-server/node_modules/ws/lib/receiver.js:138:10)
    at Socket._ultron.on (/home/parse/node_modules/parse-server/node_modules/ws/lib/websocket.js:143:54)
    at Socket.emit (events.js:182:13)
    at Socket.EventEmitter.emit (domain.js:441:20)
    at addChunk (_stream_readable.js:287:12)
    at readableAddChunk (_stream_readable.js:268:11)
    at Socket.Readable.push (_stream_readable.js:223:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:122:17)
/home/parse/node_modules/parse-server/lib/ParseServer.js:218
          throw err;

@mtrezza
Copy link
Member

mtrezza commented May 25, 2021

@majidhassan Please note that Parse does not currently offer Long Terms Support and we only address issues that are reproducible with latest version of Parse Server.

If not, any idea if it's possible to run different versions of parse for the main server and the livequery server?

LiveQuery is a feature of Parse Server, not an individual repository. Therefore I would generally recommend to use a single version of that repository. Mixing instances of different Parse Server versions using the same DB may or may not work. It is not officially documented as supported, therefore the compatibility is undefined and not recommended for production environments.

@majidhassan
Copy link

@mtrezza thanks for your reply, and I understand that it's an older version. However, any ideas where to start on that error?

@mtrezza
Copy link
Member

mtrezza commented May 26, 2021

To confirm that this is an issue we'd need to be able to reproduce this with the latest version of Parse Server and without mixing different versions of Parse Server. Then we can handle the issue here.

If you need support for your custom deployment with older versions of Parse Server and/or mixed up Parse Server versions, you would post a topic in the Parse community forum or on StackOverflow. Code-level support is not provided in GitHub issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants