Skip to content

feat: Support dockerPrivateKey to specify path to SSH key #674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 8, 2022
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,20 @@ custom:
```

The `dockerSsh` option will mount your `$HOME/.ssh/id_rsa` and `$HOME/.ssh/known_hosts` as a
volume in the docker container. If your SSH key is password protected, you can use `ssh-agent`
because `$SSH_AUTH_SOCK` is also mounted & the env var set.
volume in the docker container.

In case you want to use a different key, you can specify the path (absolute) to it through `dockerPrivateKey` option:

```yaml
custom:
pythonRequirements:
dockerizePip: true
dockerSsh: true
dockerPrivateKey: /home/.ssh/id_ed25519
```

If your SSH key is password protected, you can use `ssh-agent`
because `$SSH_AUTH_SOCK` is also mounted & the env var is set.
It is important that the host of your private repositories has already been added in your
`$HOME/.ssh/known_hosts` file, as the install process will fail otherwise due to host authenticity
failure.
Expand Down Expand Up @@ -213,7 +225,7 @@ the names in `slimPatterns`

#### Option not to strip binaries

In some cases, stripping binaries leads to problems like "ELF load command address/offset not properly aligned", even when done in the Docker environment. You can still slim down the package without `*.so` files with
In some cases, stripping binaries leads to problems like "ELF load command address/offset not properly aligned", even when done in the Docker environment. You can still slim down the package without `*.so` files with:

```yaml
custom:
Expand Down Expand Up @@ -566,3 +578,4 @@ package:
- [@jacksgt](https://github.com/jacksgt) - Fixing pip issues
- [@lephuongbg](https://github.com/lephuongbg) - Fixing single function deployment
- [@rileypriddle](https://github.com/rileypriddle) - Introducing schema validation for `module` property
- [@martinezpl](https://github.com/martinezpl) - Fixing test issues, adding `dockerPrivateKey` option
7 changes: 5 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ const { installAllRequirements } = require('./lib/pip');
const { pipfileToRequirements } = require('./lib/pipenv');
const { pyprojectTomlToRequirements } = require('./lib/poetry');
const { cleanup, cleanupCache } = require('./lib/clean');

BbPromise.promisifyAll(fse);

/**
Expand Down Expand Up @@ -45,6 +44,7 @@ class ServerlessPythonRequirements {
: this.serverless.service.provider.runtime || 'python',
dockerizePip: false,
dockerSsh: false,
dockerPrivateKey: null,
dockerImage: null,
dockerFile: null,
dockerEnv: false,
Expand All @@ -71,7 +71,10 @@ class ServerlessPythonRequirements {
}
if (
!options.dockerizePip &&
(options.dockerSsh || options.dockerImage || options.dockerFile)
(options.dockerSsh ||
options.dockerImage ||
options.dockerFile ||
options.dockerPrivateKey)
) {
if (!this.warningLogged) {
if (this.log) {
Expand Down
8 changes: 6 additions & 2 deletions lib/pip.js
Original file line number Diff line number Diff line change
Expand Up @@ -275,12 +275,16 @@ async function installRequirements(targetFolder, pluginInstance) {

dockerCmd.push('docker', 'run', '--rm', '-v', `${bindPath}:/var/task:z`);
if (options.dockerSsh) {
const homePath = require('os').homedir();
const sshKeyPath =
options.dockerPrivateKey || `${homePath}/.ssh/id_rsa`;

// Mount necessary ssh files to work with private repos
dockerCmd.push(
'-v',
`${process.env.HOME}/.ssh/id_rsa:/root/.ssh/id_rsa:z`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check if that change to os.homedir() could be causing these issues with permissions on runner? AFAIK it still takes the result from process.env.HOME but nothing else in this PR looks like the cause of this problem and all other PRs are passing without problems 😕

`${sshKeyPath}:/root/.ssh/${sshKeyPath.split('/').splice(-1)[0]}:z`,
'-v',
`${process.env.HOME}/.ssh/known_hosts:/root/.ssh/known_hosts:z`,
`${homePath}/.ssh/known_hosts:/root/.ssh/known_hosts:z`,
'-v',
`${process.env.SSH_AUTH_SOCK}:/tmp/ssh_sock:z`,
'-e',
Expand Down
33 changes: 30 additions & 3 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ const glob = require('glob-all');
const JSZip = require('jszip');
const sha256File = require('sha256-file');
const tape = require('tape-promise/tape');

const {
chmodSync,
removeSync,
Expand All @@ -23,7 +24,7 @@ const mkCommand =
(cmd) =>
(args, options = {}) => {
options['env'] = Object.assign(
{ SLS_DEBUG: 't' },
{ SLS_DEBUG: 'true' },
process.env,
options['env']
);
Expand All @@ -32,11 +33,11 @@ const mkCommand =
args,
options
);
if (error) {
if (error && !options['noThrow']) {
console.error(`Error running: ${quote([cmd, ...args])}`); // eslint-disable-line no-console
throw error;
}
if (status) {
if (status && !options['noThrow']) {
console.error('STDOUT: ', stdout.toString()); // eslint-disable-line no-console
console.error('STDERR: ', stderr.toString()); // eslint-disable-line no-console
throw new Error(
Expand Down Expand Up @@ -200,6 +201,32 @@ const canUseDocker = () => {
// Skip if running on these platforms.
const brokenOn = (...platforms) => platforms.indexOf(process.platform) != -1;

test(
'dockerPrivateKey option correctly resolves docker command',
async (t) => {
process.chdir('tests/base');
const path = npm(['pack', '../..']);
npm(['i', path]);
const stdout = sls(['package'], {
noThrow: true,
env: {
dockerizePip: true,
dockerSsh: true,
dockerPrivateKey: `${__dirname}${sep}tests${sep}base${sep}custom_ssh`,
dockerImage: 'break the build to log the command',
},
});
t.true(
stdout.includes(
`-v ${__dirname}${sep}tests${sep}base${sep}custom_ssh:/root/.ssh/custom_ssh:z`
),
'docker command properly resolved'
);
t.end();
},
{ skip: !canUseDocker() || brokenOn('win32') }
);

test(
'default pythonBin can package flask with default options',
async (t) => {
Expand Down
1 change: 1 addition & 0 deletions tests/base/custom_ssh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
SOME KEY
6 changes: 6 additions & 0 deletions tests/base/serverless.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ custom:
pythonRequirements:
zip: ${env:zip, self:custom.defaults.zip}
dockerizePip: ${env:dockerizePip, self:custom.defaults.dockerizePip}
dockerSsh: ${env:dockerSsh, self:custom.defaults.dockerSsh}
dockerPrivateKey: ${env:dockerPrivateKey, self:custom.defaults.dockerPrivateKey}
dockerImage: ${env:dockerImage, self:custom.defaults.dockerImage}
slim: ${env:slim, self:custom.defaults.slim}
slimPatterns: ${file(./slimPatterns.yml):slimPatterns, self:custom.defaults.slimPatterns}
slimPatternsAppendDefaults: ${env:slimPatternsAppendDefaults, self:custom.defaults.slimPatternsAppendDefaults}
Expand All @@ -24,6 +27,9 @@ custom:
slimPatternsAppendDefaults: true
zip: false
dockerizePip: false
dockerSsh: false
dockerPrivateKey: ''
dockerImage: ''
individually: false
useStaticCache: true
useDownloadCache: true
Expand Down