wip

Author: ais-one

.github/workflows/manual-gcp-expressjs.yml

@@ -1,16 +1,17 @@
 
 
 ### Version 0.7.0
+- [IMPORTANT] moved `js-node/expressjs` and `vue-nobundler` to [https://github.com/es-labs/express-template]()
 - [chore] update packages & cleanup & work on improving documentation
 - [chore] prototype projects `solid`, `react-admin` and `nest-admin` have been removed
 - [jsonwebtoken] update version handle minimum key size requirement change
-- [@es-labs/node] moved to https://github.com/es-labs/jslib monorepo
-- [@es-labs/esm] moved to https://github.com/es-labs/jslib monorepo
-- [js-web/vue-vite] moved to https://github.com/ais-one/vue-antd-template
+- [@es-labs/node] moved to https://github.com/es-labs/jscommon monorepo
+- [@es-labs/esm] moved to https://github.com/es-labs/jscommon monorepo
+- [js-web/vue-vite] moved to https://github.com/es-labs/vue-antd-template
 - [js-web/react-vite] moved to https://github.com/ais-one/react-template (work in progress)
 - [js-node/jsdoc-ts] example on how to use jsdoc for typescript
-- [js-node/expressjs] webpush push notification sample [sample](js-node/expressjs/public/demo-express/pn.html)
-- [js-node/expressjs] SSO (OAuth, OIDC, SAML) sample [sample](js-node/expressjs/public/demo-express/sso.html)
+- [js-node/expressjs] webpush push notification sample at `public/demo-express/pn.html`
+- [js-node/expressjs] SSO (OAuth, OIDC, SAML) sample at `public/demo-express/sso.html`
 - [js-node/expressjs] try to removed passport & passport-saml, use @node-saml/node-saml
 - [js-node/expressjs] remove sentry, APM for userland to choose and implement
 - [@es-labs/node] SSO for oidc to opt to use IDP or issue own token
@@ -19,14 +20,7 @@
 - [all] APP_NAME is now only to identify name based on package.json name property. APP_PATH removed
 - [js-node/expressjs] rename apps/app-template to app-sample-template
 - [js-node/expressjs] replace openapi file joiner, swagger-ui-express and express-openapi-validator with express-jsdoc-swagger 
-- [in-progress] rename `js-node` to js-apps, add js-libs, js-tools (https://github.com/mikemajesty/nestjs-monorepo) should I add back libs here?
-  - clone a package from monorepo of a github template project? and able to have updates from upstream?
-- [in-progress] move error handling to @ss-labs/node/express/init or preRoute
-- [in-progress] clean up @es-labs/node/auth
-- [next-in-pipeline] add more express-jsdoc-swagger to routes
-- [next-in-pipeline] fix broken tests and examples
-- [next-in-pipeline] re-implement MQ
-- [next-in-pipeline] to use Typescript and tsoa ? or use JSDoc?
+
 
 ### Version 0.6.11
 - [chore] update packages & cleanup & work on improving documentation

BACKLOG.md

@@ -13,14 +13,20 @@
 Latest Version [0.7.0](https://github.com/ais-one/cookbook/releases/tag/0.7.0) - Released 2023 Sep 01 0830 +8GMT. åSee changes history in [CHANGELOG.md](CHANGELOG.md) and discuss [here](https://github.com/ais-one/cookbook/discussions)
 
 Companion Projects:
-- Frontend Template to go with the [express API backend](js-node/expressjs) in this repo
-  - [vue-antd-template](https://github.com/ais-one/vue-antd-template)
+- Backend Template & Sample App:
+  - [https://github.com/es-labs/express-template]()
+- Frontend Template & Sample App:
+  - [https://github.com/es-labs/vue-antd-template]()
 - Reusable Components:
   - [es-labs/jslib](https://github.com/es-labs/jslibs) reusable CJS and ESM codes
 - Python:
   - [favv](https://github.com/ais-one/favv) API backend implement in Python FastAPI
   - streamlit componennt examples
 
+# IMPORTANT
+
+[Read Me First](_READMEFIRST.md)
+
 # Features
 
 Folder | Description | Features
@@ -32,7 +38,7 @@ Folder | Description | Features
 [js-node/expressjs/public/demo-express](js-node/expressjs/public/demo-express) | Frontend to test backend features | - GraphQL, File uploads, Signed URL file upload to GCP Storage, websockets, SSE, webworkers (frontend demo)<br>- JWT using RSA, JWT refresh token, token in HttpOnly cookies, GA OTP, role SAML, OIDC<br>- Github OAuth2 login (setup - https://www.sohamkamani.com/blog/javascript/2018-06-24-oauth-with-node-js)<br>- Fido & Webauthn<br>- Push Notification (Webpush & FCM)
 [js-node/*](js-node) | **Other Backend applications** | - [Services](js-node/services) (TCP event/stream, scalable Websocket with Redis Pubsub, etc.)<br>- [Serial server](js-node/serialserver)
 [vue-nobundler](vue-nobundler/) | Vue 3 SPA no bundler + Bulma | - signed uploads, recaptcha<br>- **Web component table, form & CRUD backend** (files to note)<br><table><tr><td>[js-node/expressjs/app-template-sample/tables/]()</td><td>sample custom app table configurations</td></tr><tr><td>[js-node/expressjs/router/t4t.js](js-node/expressjs/router/t4t.js)</td><td>handle backend CRUD API</td></tr><tr><td>[@es-labs/esm/t4t-fe.js](@es-labs/esm/t4t-fe.js)</td><td>frontend operations to interact with t4t.js</td></tr><tr><td>[@es-labs/esm/t4t-validate.js](@es-labs/esm/t4t-validate.js)</td><td>validation used by both front and backend</td></tr><tr><td>[@es-labs/esm/bwc-table](@es-labs/esm/bwc-table)</td><td>used to display table</td></tr><tr><td>[@es-labs/esm/bwc-t4t-form.js](@es-labs/esm/bwc-t4t-form.js)</td><td>form generated from table configurations</td></tr><tr><td>[vue-nobundler/views/ui1.js](vue-nobundler/views/ui1.js)</td><td>autcomplete, combobox, file upload example</td></tr><tr><td>[vue-nobundler/views/ui2.js](vue-nobundler/views/ui2.js)</td><td>table example</td></tr><tr><td>[vue-nobundler/views/ui3.js](vue-nobundler/views/ui3.js)</td><td>form example (with connection to backend)</td></tr><tr><td>[vue-nobundler/views/ui4.js](vue-nobundler/views/ui4.js)</td><td>table and form example (with connection to backend)</td></tr></table>
-[js-web/vue-vite] | - Vue 3 SPA<br>- vite / vue-router / Pinia<br>- Ant Design | - Leaflet Map, AntV Charts, PWA, Websockets, rxJS<br>- 2FA GA OTP, OIDC + Refresh, SAML, Github OAuth<br>- Web Components (Webcam, Signature)<br>- Playwrite Testing<br>- i18n / graphql (TBD)<br>- moved to https://github.com/ais-one/vue-antd-template
+[js-web/vue-vite] | - Vue 3 SPA<br>- vite / vue-router / Pinia<br>- Ant Design | - Leaflet Map, AntV Charts, PWA, Websockets, rxJS<br>- 2FA GA OTP, OIDC + Refresh, SAML, Github OAuth<br>- Web Components (Webcam, Signature)<br>- Playwrite Testing<br>- i18n / graphql (TBD)<br>- moved to https://github.com/es-labs/vue-antd-template
 js-web/react-vite | - react SPA<br>- react-router-dom<br>- zustand/@tanstack<br>- react-query | - moved to https://github.com/ais-one/react-template
 [.github/workflows](.github/workflows) | Github Actions (CI/CD) | - Manually Trigger<br>- On-push Trigger (WIP)
 AMP Website | [removed](https://plausible.io/blog/google-amp) | -
@@ -121,7 +127,7 @@ See [js-node/README.md](js-node/README.md)
 
 ### Vite SPA Setup & Run - development environment
 
-See [vue-antd-template project](https://github.com/ais-one/vue-antd-template).
+See [https://github.com/es-labs/vue-antd-template]().
 
 Why No SSR or SSG:
 - potential slow rendering by server app, added complexity in code, rehydration errors, added complexity in server

CHANGELOG.md

@@ -0,0 +1,28 @@
+# Read Me First
+
+## Resuable JS frameworks how to...
+
+for template (framework) repo
+1. exclude configurable files from repo (e.g. .env), but include sample config files (e.g. .env.sample)
+2. create an folder where all custom code is placed, framework must not touch anything inside this folder
+
+for app based on template repo
+1. clone template repo
+2. create a remote called upstream pointing to template repo
+3. update framework when necessary by merging upstream into origin
+
+template repo is advisable not to be monorepo
+
+## Shareable libraries and contracts
+
+TBD
+
+
+
+
+- [done] rename `js-node` to js-apps, add js-libs, js-tools (https://github.com/mikemajesty/nestjs-monorepo) should I add back libs here?
+  - clone a package from monorepo of a github template project? and able to have updates from upstream?
+
+
+
+

README.md

@@ -4,14 +4,14 @@ Running the example
 
 ```
 npm i
-cd js-node/expressjs/sandbox
+cd js-node/services/sandbox
 node tcpServer.js
 ```
 
 A TCP server will listen on port 7070. Use putty (for windows) as client to connect (Raw TCP) and test
 
-Example Usage file location [../../js-node/expressjs/sandbox/tcpServer.js](../../js-node/expressjs/sandbox/tcpServer.js)
+Example Usage file location [../../js-node/services/sandbox/tcpServer.js]()
 
 More Samples
-- More TCP Usage [../../js-node/expressjs/sandbox/net.js](../../js-node/expressjs/sandbox/net.js)
-- TCP Retries [../../js-node/expressjs/sandbox/net-retries.js](../../js-node/expressjs/sandbox/net-retries.js)
+- More TCP Usage [../../js-node/services/sandbox/net.js]()
+- TCP Retries [../../js-node/services/sandbox/net-retries.js]()

_READMEFIRST.md

@@ -130,7 +130,7 @@ ps ax | grep 'node index.js' | grep -v grep | awk '{print $1}' | xargs kill
 
 ### use pm2
 
-See js-node/expressjs/ecosystem.config.js
+look for `ecosystem.config.js`
 
 
 # Startup on VM using SystemD

docs/backend/tcp.md

@@ -55,56 +55,6 @@ In GCP
 In Mongo Atlas
 - create mongo atlas account
 
-### Backend - js-node/expressjs
-
-In js-node/expressjs folder
-
-1. place service account json file into [../../js-node/expressjs/.env.secret]()
-2. set the mongodb info in [../../js-node/expressjs/.env.secret]()
-3. set the CORS to allow frontend origin from the frontend setup 
-
-```bash
-# on windows, need to use bash shell
-deploy.sh uat app-template
-```
-
-Note the URL returned you can test it using <URL>/api/healthcheck, you may need to wait awhile for initial response or retry a few times
-
-
----
-
-## Local Development
-
-This is for local development purpose and tries to replicate as much of the real-life environment is possible
-
-- git
-- webhooks testing
-  - called by internal - http://xip.io/ - e.g. https://www.127.0.0.1.xip.io/api/health
-  - called by external site - ngrok
-- NodeJS
-  - with self signed cert
-- WSL Ubuntu
-- Docker
-  - MongoDB, SAML, Kafka, MySQL, Vault docker compose files can be found in [../../docker-devenv](../../docker-devenv)
-
-## Backend Application Container Build
-
-We can build backend on container so that we can orchestrate using K8s
-
-See [deployment-container.md](deployment-container.md)
-
-And [../../js-node/expressjs/Dockerfile](../../js-node/expressjs/Dockerfile)
-
-## Deployment Preparation
-
-View files, folders and readme.md below, and configure them
-
-- js-node/expressjs/.env
-- js-node/expressjs/.env.secret (enter your secrets as required)
-- js-node/expressjs/ecosystem.config.js (for pm2 deployments)
-- js-node/expressjs/Dockerfile (for docker deployments)
-- js-node/expressjs/deploy.sh
-
 ---
 
 ## Deployment On Single VM

docs/deployment/deployment-vm.md

@@ -14,7 +14,7 @@ Need to take into consideration, the following:
 
 ### Secret Managers
 
-- Hashicorp Vault [we use the docker version of this for our testing js-node/expressjs on local machine]
+- Hashicorp Vault
   - self hosted, docker
   - helm, sidecar
   - vault + mongoatlas

docs/deployment/home.md

@@ -66,3 +66,5 @@
 
 - https://www.aspecto.io/blog/lerna-hello-world-how-to-create-a-monorepo-for-multiple-node-packages/
 - https://medium.com/@anoop_gupta/cross-repository-component-sharing-using-mono-repo-multi-packages-architecture-2797c3e02c6c
+
+- https://www.codemzy.com/blog/nodejs-file-folder-structure

docs/deployment/secrets.md

@@ -1,6 +1,5 @@
 #!/bin/sh
 
 echo ---------- pre-commit execute start ----------
-cd js-node/expressjs && npm run lint
 # you can do a git add . here also
 echo ---------- pre-commit execute end ----------

docs/home.md

@@ -4,8 +4,6 @@ Below are a list of NodeJS applications
 
 - dbdeploy
   - see [dbdeploy/README.md]()
-- expressjs
-  - see [expressjs/README.md]()
 - jsdoc-ts
   - see [jsdoc-ts/README.md]() may eventually replace swagger-jsdoc-swagger
 - scaled-ws
@@ -42,9 +40,6 @@ npm run process-long
 npm run process-cron
 ```
 
-
-
-
 # Useful commands
 
 redirect to a file with date 

git-hooks/pre-commit

@@ -10,7 +10,9 @@ https://knexjs.org/guide/migrations.html#migration-cli
 when DB could be shared between packages
 
 
-## Quick Create DB For demo expressjs App
+## Quick Create DB
+
+For [express-template](https://github.com/es-labs/express-template) app sample
 
 ```bash
 npx knex --knexfile knex-sample/knexfile.js migrate:up

js-node/README.md

@@ -8,7 +8,7 @@ module.exports = {
   development: {
     client: 'sqlite3',
     connection: {
-      filename: '../../expressjs/dev.sqlite3'
+      filename: 'dev.sqlite3'
     }
   },