Discussion: Deno Port #2225
Comments
|
It would be very ideal if this could happen, and happy to contribute if the |
|
Fwiw, I wrote a rather fully-featured client for Node.js in TypeScript: https://github.com/malthe/ts-postgres. I do have plans to port it to Deno, but I was rather hoping for a single-source approach. |
|
I'm totally cool with any kind of port or fork you think is good to get postgres support more built out on Deno. This part of the library in particular should be useful as its written in TypeScript already and doesn't have many 3rd party dependencies. If there's a way to remove all the node dependencies and make it portable to Deno while maintaining the same performance characteristics I'd be open to discussing merging that here & have re-use across platforms. That being said, my primary focus for the rest of 2020 is to continue to improve perf on this library, modernize it (some of the code is >10 years old!) and provide an extremely robust and well tested platform for node.js postgres connections. I'm happy to help out in a limited capacity with Deno support, but I think its best for the community if I continue to focus on primarily supporting node.js. I've been involved in the node community for >10 years now, and I watched trends come and go. If Deno becomes huge and replaces node, that'll be interesting, but even then I will still focus on support this library and the millions of installations it has throughout the world. That's my primary focus, and I don't have any plans to refocus in the foreseeable future. If I check the weekly installs and it drops below a few thousand I'll probably retire from open source at that point...but as far as I can see that's years from now. 😄 Who knows! Anything could happen & I'm happy to help out anywhere I can so long as it doesn't detract from my focus on this lib! Also, this code is all open source so feel free to base any implementation on it. All I ask, for the community's sake, is that you take the time to PR any bug fixes you find in any ports back here....I've sunk literally thousands of hours into this project over the years (mostly GH issue triage, bug research, etc) and would be happy to merge any improvements that "life all boats" like rising tides are wont to do. @malthe would you be interested in trying pg-protocol in your own library actually?...it's very, very battle tested protocol parser for postgres supporting every message type defined & totally written in TypeScript! Might be a good move to pool our efforts rather than write different protocol parsers. An example would be you find a perf improvement in parsing, you put it into I know ts-postgres does some different things wrt batching & stuff I'm going to work on here in the nearish future, but an entire from the ground rewrite of parsing in your module is probably redundant if you want to use Perhaps in the future we can consolidate our efforts and improve things for node & postgres as a whole in this project instead of fragmenting the module landscape...though I'm well aware that's the open source way and a the node community has always trended really hard that way. 🤷 |
|
That's a great reply. Thanks @brianc. How about moving these projects under a GitHub Org? It's great to have a package under your username and being in control. I love working in my own little world. That being said, I believe this is one of the reasons that the js/open source is fragmented. Having a Postgres based org would give confidence to the community that the project will not be abandoned by an author. I forked the pg-format module to convert to Deno and am offering to give it back to @rpedela here. It would make sense to put these types of modules under a common org. @brianc, you are the perfect person to start such an organization. |
|
@brianc pg-protocol looks good; if I can get around to it, I'll try and see if it'll fit easily into the codebase – because you're absolutely right, it would be great to consolidate around a single implementation for the protocol. When I wrote the code I don't think there was such a package available, at least not written in TypeScript. |
|
This library actually does work in Deno now, but the performance is so bad unfortunately. deno-postgres now exists, which has acceptable performance on Deno, but lacks support for streaming/listen/notify and has some other problems I am still struggling with. |
strange that performance is so bad...I wonder why? Maybe some kinda stream adapter? or their buffer adapter is slow? There aren't very many node dependencies in this library outside of buffer & stream...shouldn't be too hard to speed it up. I haven't done a benchmark recently on converting pg-protocol from using node's buffer object to arraybuffers & typed arrays...that'd probably be a good first step |
|
@brianc I actually do have a high-level benchmark of my application code. deno-postgres on Deno: node-postgres on Deno: There are some confounding variables. I am using Kysely for both benchmarks. The built-in Postgres adapter for node-postgres, and a custom adapter for deno-postgres. However I'm not sure what causes a 100x decline in performance. If you have any existing benchmarks you'd like me to try, I can adapt them to Deno and compare the libraries directly. Being able to use this library would help me immensely because I need streaming/listen/notify for my project, so I'm motivated to do it. |
|
@brianc Is this the benchmark you recommend? https://github.com/brianc/node-postgres/blob/master/packages/pg/bench.js I will port it and test directly against deno-postgres in Deno. |
|
yeah it's a hideously bad benchmark but its what i use to check for major perf regressions 😬 |
|
@brianc Here's the benchmark on my laptop with Deno. Results are similar to the earlier benchmark - it still seems very slow. Do you glean anything from this? The benchmark is here: https://gist.github.com/alexgleason/64ed1a1e44e40284b3c379b4331eb340 You can reproduce it by running this: deno run -A https://gist.githubusercontent.com/alexgleason/64ed1a1e44e40284b3c379b4331eb340/raw/21ef7351ff564e51a1790f5accb169cbed1ff7e2/node-postgres-deno-bench.tscc @bartlomieju, might find this interesting. |
|
@brianc Any other thoughts on this benchmark? Thanks! |
|
Are there any updates? |
|
@zhang-wenchao I switched to postgresjs and it has the best performance on Deno of any library I tried, plus all the same features of this library. For best performance, you have to use the version from deno.land, not npm: |
|
EDIT: right after reading the entire issue, seems to be something to do with node's compat API in deno. coming to this thread since deno 2.0 release. noticed that simple SELECT WHERE query are 4-5x slower than node/bun. anything I can do to be helpful? 
import pg from 'pg'
const { Client } = pg
const client = new Client({
connectionString: 'postgresql://a:[email protected]:6432/public?sslmode=true'
})
await client.connect()
async function fetch() {
const startPerf = performance.now();
const res = await client.query('SELECT "id", "name", "email", "is_blocked" from "Users" where "Users"."id" = $1', ['1129c38d-874b-4d1b-9b88-08fee0f39f95'])
console.log(`${new Date()}: ${Math.round(performance.now() - startPerf)}ms`)
}
setInterval(() => {
fetch()
}, 1000) |
|
fix: denoland/deno#26263 |
I have been having a close look at deno and have had an attempt to build a solution with it. It is delightfully refreshing to not have to build a project, and to be able to use the current JavaScript language features.
At this point in time, there are limited options for working with PostgreSQL.
The main project on the Third Party Modules list, deno-postgres, has one major feature missing and one show stopper bug. I considered trying to contribute to the project however after having a look at what has been done with this project, there are huge amounts of work required.
I looked for an issue being raised here about porting over to
Denoand there was none so I'm bringing up the discussion.I am no expert on
Denoalthough I did migrate a couple of projects over to it already being pg-format and cuid.As far as I am aware, to migrate this project so that it will work on
Denowould require the following:requirewithimportmodule.exportswithexportNode.jsAPIs toDenoor JS and Web standard APIs (EventEmitter to EventTarget as an example)lernato the built-in Deno.test runner and standard library assertions.I imagine the mix of
jsandtsmodules would be an issue. I'm not a TypeScript guy. There are probably a few other issues which will arise as you would expect.The deno-postgres project has been created by one of the main
Denodevelopers, Bartek Iwańczuk, and he has stated that he is busy with the core ofDenoat the moment. He is going to work on it soon. It is not a fork of this project though and is missing many features at this time.When you consider the vast amount of work that has been put into this project, it seems a shame to re-invent the wheel. Both
Node.jsandDenorun JavaScript. That being said,Denotries as much as possible to stick to the Web Standard APIs. This will mean significant changes and therefore it is unlikely to be able to keep theNode.jsandDenoport in sync.I'm just opening the discussion here to gauge interest.
The text was updated successfully, but these errors were encountered: