Skip to content

Improve CI/CD for simcore and add staging/production stages #738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 165 commits into from
May 16, 2019
Merged

Improve CI/CD for simcore and add staging/production stages #738

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
165 commits
Select commit Hold shift + click to select a range
7e0ec29
fixed director in devel mode
sanderegg Mar 26, 2019
d88bd08
Merge remote-tracking branch 'upstream/master'
sanderegg Mar 28, 2019
994a94f
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 3, 2019
ac98a02
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 4, 2019
6d02a21
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 4, 2019
0714adc
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 8, 2019
9fa3e08
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 9, 2019
b7f9841
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 11, 2019
1875de4
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 17, 2019
56905cd
Merge branch 'master' of github.com:sanderegg/osparc-simcore
sanderegg Apr 17, 2019
8ff7d7b
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 25, 2019
683bd7b
refactor
sanderegg Apr 25, 2019
541b575
create deployment scripts for master, staging and production
sanderegg Apr 25, 2019
c581f98
changed default naming
sanderegg Apr 25, 2019
be49507
new scripts for deployment
sanderegg Apr 25, 2019
eeaa5d4
use inofficial rules for bash scripts
sanderegg Apr 25, 2019
12e745a
refactoring
sanderegg Apr 25, 2019
74cb99b
added simcore-sdk integration tests
sanderegg Apr 25, 2019
f1559ee
remove building for unit tests
sanderegg Apr 25, 2019
7c4243d
unit tests should not need the other components
sanderegg Apr 25, 2019
63ecf81
added die on error flags
sanderegg Apr 25, 2019
cae624e
auto replace if / is missing from image prefix
sanderegg Apr 25, 2019
1d25c0e
fixes pull
sanderegg Apr 26, 2019
d10d145
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 26, 2019
d754888
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 27, 2019
9f42016
Merge branch 'master' into is713/rename_staging_to_master
sanderegg Apr 27, 2019
ed0aa70
copy requirements in build/tmp folder so that cache does not get inva…
sanderegg Apr 29, 2019
8bc5cb0
removed unused recipe
sanderegg Apr 29, 2019
1f6e32d
push-staging renamed to more generic pushadded tag recipepull-staging…
sanderegg Apr 29, 2019
3656626
set scripts to strict mode
sanderegg Apr 29, 2019
f505191
Merge remote-tracking branch 'upstream/master' into fix_staging_overw…
sanderegg Apr 29, 2019
66f1041
Merge branch 'fix_staging_overwrite' into is713/rename_staging_to_master
sanderegg Apr 29, 2019
fd916e0
added x flag
sanderegg Apr 29, 2019
3891dcf
temp switch off jobs
sanderegg Apr 29, 2019
097519d
renaming scripts
sanderegg Apr 29, 2019
2c2c799
check env variables in travis
sanderegg Apr 29, 2019
fd1a931
added build scripts
sanderegg Apr 29, 2019
2fd7e92
set deployment for test
sanderegg Apr 29, 2019
d3ff9c2
removed unused exports
sanderegg Apr 29, 2019
29896d9
push cache now done in deployment phase
sanderegg Apr 29, 2019
7e315a7
added cache generation only on master branch
sanderegg Apr 29, 2019
f9ea3bc
typo
sanderegg Apr 29, 2019
b5d6980
allow deploying from all branches
sanderegg Apr 29, 2019
f05facb
minor
sanderegg Apr 29, 2019
9de35f1
use git to get branch name
sanderegg Apr 29, 2019
c96df5f
no need to tag, done by build
sanderegg Apr 29, 2019
54e7b3a
add script to slugify a branch name
sanderegg Apr 29, 2019
b0dff45
fixed called to non define env variable
sanderegg Apr 29, 2019
1d55857
pull images from registry or build if no registry
sanderegg Apr 29, 2019
d1e5656
enable simcore-sdk integration
sanderegg Apr 29, 2019
0fbfb38
added ignore failure flag
sanderegg Apr 29, 2019
9af57f6
Update test-images.sh
sanderegg Apr 30, 2019
0475b8b
Update Makefile
sanderegg Apr 30, 2019
cf36fe5
Merge pull request #29 from sanderegg/sanderegg-patch-1
sanderegg Apr 30, 2019
c1c108f
copy from webclient with name depending on env variable
sanderegg Apr 30, 2019
6f0ca7c
Merge remote-tracking branch 'upstream/master'
sanderegg Apr 30, 2019
b78056e
cannot use env variable in dockerfile COPY
sanderegg Apr 30, 2019
2066f5e
set default image prefix to services_
sanderegg Apr 30, 2019
b61a42d
check if travis variable are available
sanderegg Apr 30, 2019
3c8ff26
pull the cache before building
sanderegg Apr 30, 2019
34d4894
removed auto pulling cache
sanderegg Apr 30, 2019
7340d5f
correctly set the cache
sanderegg Apr 30, 2019
075d6a9
build the tests also if the travis script file changes
sanderegg Apr 30, 2019
32a02fa
fixes simcore-sdk tests
sanderegg Apr 30, 2019
54a18ce
get the color back?
sanderegg Apr 30, 2019
e00db68
move args after copies to not invalidate cache
sanderegg Apr 30, 2019
f37bbf8
also pull last build if possible
sanderegg Apr 30, 2019
f6048c5
change scripts to use .sh ending
sanderegg Apr 30, 2019
aa18f91
set unit tests back in
sanderegg Apr 30, 2019
c4d63f0
change script ending to be .sh
sanderegg Apr 30, 2019
e4efcdb
copy from settings template file
sanderegg Apr 30, 2019
8f914e7
removed cache script
sanderegg Apr 30, 2019
0270dfd
improve caching
sanderegg Apr 30, 2019
6ec7d05
improve size
sanderegg Apr 30, 2019
13b6946
wraps using unbuffer
sanderegg Apr 30, 2019
d48e7be
Merge remote-tracking branch 'upstream/master' into is713/rename_stag…
sanderegg Apr 30, 2019
e46590f
Merge branch 'master' into is713/rename_staging_to_master
sanderegg Apr 30, 2019
605836b
remove unbuffer
sanderegg Apr 30, 2019
f5aa14b
removed unbuffer for frontend unit-test
sanderegg Apr 30, 2019
3d02c09
use env variables instead of fixed service_ as service name
sanderegg May 2, 2019
2e2a842
do not replace service name from default
sanderegg May 2, 2019
0fae57b
remove import pdb
sanderegg May 2, 2019
a1bc14a
some of these libs are needed
sanderegg May 2, 2019
057e428
adding deployment stages
sanderegg May 2, 2019
9d9d8c1
script to generate build docker image tag
sanderegg May 2, 2019
025b5a6
slugify script only does that
sanderegg May 2, 2019
8168601
script creates docker image tag
sanderegg May 2, 2019
6c01e0c
tag recipe now takes additional variables
sanderegg May 2, 2019
ac4eb92
set execute flag on
sanderegg May 2, 2019
f626e2a
fixed deployment of test images
sanderegg May 2, 2019
5785c29
Merge branch 'is713/rename_staging_to_master' of github.com:sanderegg…
sanderegg May 2, 2019
d3897e1
typo
sanderegg May 2, 2019
03781be
no build stage for pull request, no credentials there
sanderegg May 2, 2019
d28976e
fuse build and unit testing
sanderegg May 2, 2019
3968047
use itisfoundation as default
sanderegg May 2, 2019
5b6a221
do not ignore failure and only pull the used services
sanderegg May 2, 2019
b2e4777
also pull the cache
sanderegg May 2, 2019
5be6f53
download cache if possible
sanderegg May 2, 2019
2152d79
minor
sanderegg May 2, 2019
b484f8e
download cache from default location to at least improve build time
sanderegg May 2, 2019
0efdfed
Merge pull request #30 from sanderegg/sanderegg-patch-1
sanderegg May 2, 2019
bc4b79c
no more ignoring faults
sanderegg May 6, 2019
d216496
created deployment scripts for staging, production
sanderegg May 6, 2019
b3b7a89
only build or test when not on staging/production branches
sanderegg May 6, 2019
fea923e
Merge remote-tracking branch 'upstream/master' into is713/rename_stag…
sanderegg May 7, 2019
6858aa3
Merge remote-tracking branch 'upstream/master'
sanderegg May 7, 2019
ead5902
deploy to staging if tag is present
sanderegg May 8, 2019
afcce01
disable git clone for deployment
sanderegg May 8, 2019
7f4264c
output message when Image prefix is missing
sanderegg May 8, 2019
1b1991c
removed DOCKER_IMAGE_TAG_PREFIX
sanderegg May 8, 2019
33e4b30
only deploy gh pages when in the main repo
sanderegg May 8, 2019
eecc701
minor
sanderegg May 8, 2019
6688611
minor
sanderegg May 8, 2019
310e707
minor
sanderegg May 8, 2019
ce62f6b
fixes travis?
sanderegg May 8, 2019
117bfaa
Merge remote-tracking branch 'upstream/master' into is713/rename_stag…
sanderegg May 8, 2019
8ab82d9
condition in if case instead of deploy
sanderegg May 8, 2019
1af7230
Merge remote-tracking branch 'upstream/master'
sanderegg May 9, 2019
2a2e569
Merge remote-tracking branch 'upstream/master'
sanderegg May 9, 2019
5443f15
added chosen git workflow for releasing
sanderegg May 13, 2019
ce4256b
added workflow svg
sanderegg May 13, 2019
018e27a
autodeploy on push to staging
sanderegg May 13, 2019
5375e8a
frontend testing takes more time, start it earlier
sanderegg May 13, 2019
5c6baae
Merge remote-tracking branch 'upstream/master'
sanderegg May 13, 2019
5694dd6
Merge branch 'master' of github.com:sanderegg/osparc-simcore
sanderegg May 13, 2019
b0d4ebf
Merge branch 'master' into is713/rename_staging_to_master
sanderegg May 13, 2019
be41169
always return true if branch is staging or master
sanderegg May 13, 2019
7a41533
deploying to production shall happen on staging branch when tag is pr…
sanderegg May 13, 2019
0ed27ad
fix version name
sanderegg May 13, 2019
90ec2d9
pass the correct TAG
sanderegg May 13, 2019
99f4678
clean up
sanderegg May 14, 2019
8d17ee8
add script to find docker tag from release version
sanderegg May 14, 2019
834c8cb
find the corresponding docker image from git tag
sanderegg May 14, 2019
eabd695
set travis tag to semantic version name
sanderegg May 14, 2019
c66ba90
fixes production deployment script
sanderegg May 14, 2019
676440a
regexp with \d
sanderegg May 14, 2019
14cc6ba
updated workflow diagram
sanderegg May 14, 2019
1b071e8
minor
sanderegg May 14, 2019
545a162
added CI to workflow
sanderegg May 14, 2019
d17b4a4
set execute permission on script
sanderegg May 14, 2019
38f1c50
production tag set to semantic version + compose
sanderegg May 14, 2019
a58877a
refactoring
sanderegg May 14, 2019
cb69825
more refactoring
sanderegg May 14, 2019
9c8626a
minor
sanderegg May 14, 2019
aef0ab8
Merge remote-tracking branch 'upstream/master'
sanderegg May 14, 2019
bbcef71
Merge branch 'master' into is713/rename_staging_to_master
sanderegg May 14, 2019
6a29e39
correcting merge: remove simcore-sdk tests from unit test suite
sanderegg May 14, 2019
cd4a217
refactor
sanderegg May 14, 2019
72902ff
renamed scripts to use no extension to follow
sanderegg May 14, 2019
bfc0069
set execution right on scripts
sanderegg May 14, 2019
0de79aa
lint
sanderegg May 14, 2019
8f9b25f
minor
sanderegg May 14, 2019
95e822b
add some documentation towards enabling travis
sanderegg May 14, 2019
ae4b7b1
fixes the docker_registry missing env
sanderegg May 15, 2019
7aaf4db
fixes tests with new environ variable names
sanderegg May 15, 2019
46683b5
fixes pytest warnings
sanderegg May 15, 2019
6234b0e
add error message when missing variable
sanderegg May 15, 2019
cb2eaf7
ensure DOCKER_REGISTRY and DOCKER_IMAGE_TAG are set when testing
sanderegg May 15, 2019
3bed11c
completed documentation
sanderegg May 15, 2019
789967d
moved release workflow in ops
sanderegg May 15, 2019
45e6e04
add link to mermaid live editor
sanderegg May 15, 2019
9765c72
set default in docker-compose to prevent having to modify tests
sanderegg May 15, 2019
fdf48c1
added unbuffer
sanderegg May 15, 2019
dc9c0f1
linting
sanderegg May 15, 2019
7ddc048
DOCKER_REGISTRY always contains something such that a / shall be adde…
sanderegg May 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .env-devel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
DOCKER_IMAGE_PREFIX=services_
DOCKER_IMAGE_TAG=latest

PUBLISHED_HOST_NAME=localhost
Expand Down
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,4 @@ itisfoundation.github.io/

# pylint-profile output
prof/
!ops/**/build
Loading