Skip to content

[Cloud Run] Identity Platform + Cloud SQL sample #1984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Oct 21, 2020
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
a5f92fb
App Drafted
averikitsch Sep 16, 2020
c41ff9a
Rename
averikitsch Sep 24, 2020
012918f
updates
averikitsch Sep 25, 2020
f262529
minor style fixes
averikitsch Sep 25, 2020
18c0063
Merge branch 'master' into cats-v-dogs
averikitsch Sep 25, 2020
5e2363d
Merge branch 'master' into cats-v-dogs
averikitsch Sep 25, 2020
e9c54de
Updates
averikitsch Oct 1, 2020
816a95a
Update 2
averikitsch Oct 1, 2020
4cc2a23
Add more tests and updates
averikitsch Oct 1, 2020
7e44fe7
Merge branch 'cats-v-dogs' of https://github.com/GoogleCloudPlatform/…
averikitsch Oct 1, 2020
08eb839
Comment updates
averikitsch Oct 5, 2020
c04234e
Tests updated
averikitsch Oct 6, 2020
bd25555
Add createTable
averikitsch Oct 6, 2020
fffabc2
Add scripts
averikitsch Oct 7, 2020
3718c26
secret manager clean up
averikitsch Oct 7, 2020
b17cf63
Add Kokoro set up
averikitsch Oct 7, 2020
8af2289
Remove trace agent
averikitsch Oct 8, 2020
1699e48
Merge branch 'master' into cats-v-dogs
fhinkel Oct 8, 2020
e039444
Merge branch 'master' into cats-v-dogs
averikitsch Oct 8, 2020
b7ccc59
Update tests
averikitsch Oct 8, 2020
7ac9172
Merge branch 'cats-v-dogs' of https://github.com/GoogleCloudPlatform/…
averikitsch Oct 8, 2020
b6b3d84
Add env vars
averikitsch Oct 8, 2020
b5194ed
Update kokoro
averikitsch Oct 8, 2020
86a74d4
Add and drop table for unit tests
averikitsch Oct 8, 2020
60d5254
Add debugging
averikitsch Oct 8, 2020
1bdc6e0
Update README
averikitsch Oct 8, 2020
eff522c
PR comments
averikitsch Oct 13, 2020
eee77a1
Restructure startup
averikitsch Oct 15, 2020
9777c17
Update retry
averikitsch Oct 15, 2020
11bd6ec
Update testing
averikitsch Oct 15, 2020
71a7bda
add more tests
averikitsch Oct 15, 2020
4e4295d
Small updates
averikitsch Oct 19, 2020
1091913
Update CR button and cloud sql create table
averikitsch Oct 20, 2020
5577713
Merge branch 'master' into cats-v-dogs
averikitsch Oct 20, 2020
072391e
fix tests
averikitsch Oct 20, 2020
3e8b565
Merge branch 'cats-v-dogs' of https://github.com/GoogleCloudPlatform/…
averikitsch Oct 20, 2020
e90bb27
Update logging
averikitsch Oct 20, 2020
be99235
Update comment
averikitsch Oct 20, 2020
f540bbe
Add sigterm handling
averikitsch Oct 21, 2020
3a4b161
Update log
averikitsch Oct 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions run/idp-sql/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright 2030 Google LLC. All rights reserved.
# Use of this source code is governed by the Apache 2.0
# license that can be found in the LICENSE file.

# Use the official lightweight Node.js 10 image.
# https://hub.docker.com/_/node
FROM node:10-slim

# Create and change to the app directory.
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./

# Install dependencies.
# If you add a package-lock.json speed your build by switching to 'npm ci'.
# RUN npm ci --only=production
RUN npm install --production

# Copy local code to the container image.
COPY . ./

# Run the web service on container startup.
CMD [ "npm", "start" ]
34 changes: 34 additions & 0 deletions run/idp-sql/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Cloud Run End User Authentication with PostgreSQL Database Sample

This sample integrates with the Identity Platform to authenticate users to the
application and connects to a Cloud SQL postgreSQL database for data storage.

Use it with the [End user Authentication for Cloud Run](http://cloud.google.com/run/docs/tutorials/end-user).

For more details on how to work with this sample read the [Google Cloud Run Node.js Samples README](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/tree/master/run).

## Dependencies

* **express**: Web server framework
* **@google-cloud/logging-winston** + **winston**: Logging library
* **@google-cloud/secret-manager**: Google Secret Manager client library
* **firebase-admin**: Verifying JWT token
* **knex**: A SQL query builder library
* **pug**: Template engine


## Environment Variables

Cloud Run services can be [configured with Environment Variables](https://cloud.google.com/run/docs/configuring/environment-variables).
Required variables for this sample include:

* `SECRETS`: The Cloud Run service will be notified of images uploaded to this Cloud Storage bucket. The service will then retreive and process the image.

OR

* `CLOUD_SQL_CONNECTION_NAME`='<MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>'
* `DB_USER`='my-db-user'
* `DB_PASS`='my-db-pass'
* `DB_NAME`='my_db'

Note: Saving credentials in environment variables is convenient, but not secure.
141 changes: 141 additions & 0 deletions run/idp-sql/app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

'use strict';

const admin = require('firebase-admin');
const express = require('express');
const { logger } = require('./logging');
const { getVotes, getVoteCount, insertVote } = require('./cloud-sql');

const app = express();
app.set('view engine', 'pug');
app.enable('trust proxy');
app.use(express.static(__dirname + '/static'));

// Automatically parse request body as form data.
app.use(express.urlencoded({extended: false}));
app.use(express.json());

// Set Content-Type for all responses for these routes.
app.use((req, res, next) => {
res.set('Content-Type', 'text/html');
next();
});

// Initialize Firebase Admin SDK
admin.initializeApp();

// [START run_user_auth_jwt]
// Extract and verify Id Token from header
const authenticateJWT = (req, res, next) => {
const authHeader = req.headers.authorization;
if (authHeader) {
const token = authHeader.split(' ')[1];
// If the provided ID token has the correct format, is not expired, and is
// properly signed, the method returns the decoded ID token
admin.auth().verifyIdToken(token).then(function(decodedToken) {
let uid = decodedToken.uid;
req.uid = uid;
next();
}).catch((err) => {
logger.error(`error with authentication: ` + err)
return res.sendStatus(403);
});
} else {
return res.sendStatus(401);
}
}
// [END run_user_auth_jwt]


app.get('/', async (req, res) => {
try {
// Query the total count of "CATS" from the database.
const catsResult = await getVoteCount('CATS');
const catsTotalVotes = parseInt(catsResult[0].count);
// Query the total count of "DOGS" from the database.
const dogsResult = await getVoteCount('DOGS');
const dogsTotalVotes = parseInt(dogsResult[0].count);
// Query the last 5 votes from the database.
const votes = await getVotes();
// Calculate and set leader values.
let leadTeam = '';
let voteDiff = 0;
let leaderMessage = '';
if (catsTotalVotes !== dogsTotalVotes) {
if (catsTotalVotes > dogsTotalVotes) {
leadTeam = 'CATS';
voteDiff = catsTotalVotes - dogsTotalVotes;
} else {
leadTeam = 'DOGS';
voteDiff = dogsTotalVotes - catsTotalVotes;
}
leaderMessage = `${leadTeam} are winning by ${voteDiff} vote${voteDiff > 1 ? 's' : ''}.`;
} else {
leaderMessage = 'CATS and DOGS are evenly matched!';
}
res.render('index.pug', {
votes: votes,
catsCount: catsTotalVotes,
dogsCount: dogsTotalVotes,
leadTeam: leadTeam,
voteDiff: voteDiff,
leaderMessage: leaderMessage,
});
} catch(err) {
logger.error(`error while attempting to get vote: ${err}`);
res
.status(500)
.send('Unable to load page; see logs for more details.')
.end();
}

});

app.post('/', authenticateJWT, async (req, res) => {
// Get decoded Id Platform user id
const uid = req.uid;
// Get the team from the request and record the time of the vote.
const {team} = req.body;
const timestamp = new Date();

if (!team || (team !== 'CATS' && team !== 'DOGS')) {
res.status(400).send('Invalid team specified.').end();
return;
}

// Create a vote record to be stored in the database.
const vote = {
candidate: team,
time_cast: timestamp,
uid,
};

// Save the data to the database.
try {
await insertVote(vote);
logger.info({message: 'vote_inserted', vote})
} catch (err) {
logger.error(`error while attempting to submit vote: ${err}`);
res
.status(500)
.send('Unable to cast vote; see logs for more details.')
.end();
return;
}
res.status(200).send(`Successfully voted for ${team} at ${timestamp}`).end();
});

module.exports = app;
112 changes: 112 additions & 0 deletions run/idp-sql/cloud-sql.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

const Knex = require('knex');
const { getSecretConfig } = require('./secrets');

// [START run_user_auth_knex]
const connectWithUnixSockets = (config, secretConfig) => {
const dbSocketPath = process.env.DB_SOCKET_PATH || "/cloudsql"
// Establish a connection to the database
return Knex({
client: 'pg',
connection: {
user: secretConfig.DB_USER, // e.g. 'my-user'
password: secretConfig.DB_PASS, // e.g. 'my-user-password'
database: secretConfig.DB_NAME, // e.g. 'my-database'
host: `${dbSocketPath}/${secretConfig.CLOUD_SQL_CONNECTION_NAME}`,
},
// ... Specify additional properties here.
...config
});
}
// [END run_user_auth_knex]

const config = {
// Configure which instance and what database user to connect with.
// Remember - storing secrets in plaintext is potentially unsafe. Consider using
// something like https://cloud.google.com/kms/ to help keep secrets secret.
pool: {
// 'max' limits the total number of concurrent connections this pool will keep. Ideal
// values for this setting are highly variable on app design, infrastructure, and database.
max: 5,
// 'min' is the minimum number of idle connections Knex maintains in the pool.
// Additional connections will be established to meet this value unless the pool is full.
min: 5,
// 'acquireTimeoutMillis' is the number of milliseconds before a timeout occurs when acquiring a
// connection from the pool. This is slightly different from connectionTimeout, because acquiring
// a pool connection does not always involve making a new connection, and may include multiple retries.
// when making a connection
acquireTimeoutMillis: 60000, // 60 seconds
},
// 'createTimeoutMillis` is the maximum number of milliseconds to wait trying to establish an
// initial connection before retrying.
// After acquireTimeoutMillis has passed, a timeout exception will be thrown.
createTimeoutMillis: 30000, // 30 seconds
// 'idleTimeoutMillis' is the number of milliseconds a connection must sit idle in the pool
// and not be checked out before it is automatically closed.
idleTimeoutMillis: 600000, // 10 minutes
// 'knex' uses a built-in retry strategy which does not implement backoff.
// 'createRetryIntervalMillis' is how long to idle after failed connection creation before trying again
createRetryIntervalMillis: 200, // 0.2 seconds
};

let knex;
let secrets = getSecretConfig();

/**
* Insert a vote record into the database.
*
* @param {object} knex The Knex connection object.
* @param {object} vote The vote record to insert.
* @returns {Promise}
*/
const insertVote = async (vote) => {
if (!knex) knex = connectWithUnixSockets(config, await secrets);
return await knex('votes').insert(vote);
};

/**
* Retrieve the latest 5 vote records from the database.
*
* @param {object} knex The Knex connection object.
* @returns {Promise}
*/
const getVotes = async () => {
if (!knex) knex = connectWithUnixSockets(config, await secrets);
return await knex
.select('candidate', 'time_cast', 'uid')
.from('votes')
.orderBy('time_cast', 'desc')
.limit(5);
};

/**
* Retrieve the total count of records for a given candidate
* from the database.
*
* @param {object} knex The Knex connection object.
* @param {object} candidate The candidate for which to get the total vote count
* @returns {Promise}
*/
const getVoteCount = async (candidate) => {
if (!knex) knex = connectWithUnixSockets(config, await secrets);
return await knex('votes').count('vote_id').where('candidate', candidate);
};

module.exports = {
getVoteCount,
getVotes,
insertVote,
}
Loading