Allow parallel branch builds · JasonYangCode/docker-python@eb7c4fa · GitHub

JasonYangCode / docker-python Public

forked from Kaggle/docker-python

• Notifications You must be signed in to change notification settings
• Fork 0
• Star 1

• Code
• Pull requests
• Actions
• Projects
• Security and quality
• Insights

Additional navigation options

Commit eb7c4fa

Browse filesBrowse the repository at this point in the historyBrowse files

rosbo

committed

Allow parallel branch builds

1 parent 7780a6f commit eb7c4fa
Copy full SHA for eb7c4fa

4 files changed

+28-10Lines changed: 28 additions & 10 deletions

File tree

Expand file treeCollapse file tree

Open diff view settings

Filter options

• Dockerfile
• Jenkinsfile
• build
• gpu.Dockerfile

Expand file treeCollapse file tree

Open diff view settings

Collapse file

‎Dockerfile‎

Copy file name to clipboardExpand all lines: Dockerfile

+3-1Lines changed: 3 additions & 1 deletion

Original file line number	Diff line number	Diff line change
@@ -1,5 +1,7 @@
	1	+ ARG BASE_TAG=5.2.0
	2	+
1	3	FROM gcr.io/kaggle-images/python-tensorflow-whl:1.12.0-py36 as tensorflow_whl
2		- FROM continuumio/anaconda3:5.2.0
	4	+ FROM continuumio/anaconda3:${BASE_TAG}
3	5
4	6	ADD clean-layer.sh /tmp/clean-layer.sh
5	7	ADD patches/nbconvert-extensions.tpl /opt/kaggle/nbconvert-extensions.tpl

Collapse file

‎Jenkinsfile‎

Copy file name to clipboardExpand all lines: Jenkinsfile

+7-5Lines changed: 7 additions & 5 deletions

Original file line number	Diff line number	Diff line change
@@ -15,6 +15,8 @@ pipeline {
15	15	GIT_COMMIT_AUTHOR = sh(returnStdout: true, script:"git log --format='%an' -n 1 HEAD").trim()
16	16	GIT_COMMIT_SUMMARY = "`<https://github.com/Kaggle/docker-python/commit/${GIT_COMMIT}|${GIT_COMMIT_SHORT}>` ${GIT_COMMIT_SUBJECT} - ${GIT_COMMIT_AUTHOR}"
17	17	SLACK_CHANNEL = sh(returnStdout: true, script: "if [[ \"${GIT_BRANCH}\" == \"master\" ]]; then echo \"#kernelops\"; else echo \"#builds\"; fi").trim()
	18	+ PRETEST_TAG = sh(returnStdout: true, script: "if [[ \"${GIT_BRANCH}\" == \"master\" ]]; then echo \"ci-pretest\"; else echo \"ci-${GIT_BRANCH}-pretest\"; fi").trim()
	19	+ STAGING_TAG = sh(returnStdout: true, script: "if [[ \"${GIT_BRANCH}\" == \"master\" ]]; then echo \"staging\"; else echo \"${GIT_BRANCH}-staging\"; fi").trim()
18	20	}
19	21
20	22	stages {
@@ -34,7 +36,7 @@ pipeline {
34	36	set -exo pipefail
35	37
36	38	date
37		- ./push ci-pretest
	39	+ ./push ${PRETEST_TAG}
38	40	'''
39	41	}
40	42	}
@@ -56,7 +58,7 @@ pipeline {
56	58	set -exo pipefail
57	59
58	60	date
59		- ./push staging
	61	+ ./push ${STAGING_TAG}
60	62	'''
61	63	}
62	64	}
@@ -73,7 +75,7 @@ pipeline {
73	75	sh '''#!/bin/bash
74	76	set -exo pipefail
75	77	docker image prune -f # remove previously built image to prevent disk from filling up
76		- ./build --gpu | ts
	78	+ ./build --gpu --base-image-tag ${STAGING_TAG} | ts
77	79	'''
78	80	}
79	81	}
@@ -85,7 +87,7 @@ pipeline {
85	87	set -exo pipefail
86	88
87	89	date
88		- ./push --gpu ci-pretest
	90	+ ./push --gpu ${PRETEST_TAG}
89	91	'''
90	92	}
91	93	}
@@ -109,7 +111,7 @@ pipeline {
109	111	set -exo pipefail
110	112
111	113	date
112		- ./push --gpu staging
	114	+ ./push --gpu ${STAGING_TAG}
113	115	'''
114	116	}
115	117	}

Collapse file

‎build‎

Copy file name to clipboardExpand all lines: build

+15-3Lines changed: 15 additions & 3 deletions

Original file line number	Diff line number	Diff line change
@@ -7,14 +7,16 @@ Usage: $0 [OPTIONS]
7	7	Build a new Python Docker image.
8	8
9	9	Options:
10		- -g, --gpu Build an image with GPU support.
11		- -c, --use-cache Use layer cache when building a new image.
	10	+ -g, --gpu Build an image with GPU support.
	11	+ -c, --use-cache Use layer cache when building a new image.
	12	+ -b, --base-image-tag TAG Base image tag. Defaults to value defined in DOCKERFILE.
12	13	EOF
13	14	}
14	15
15	16	CACHE_FLAG='--no-cache'
16	17	DOCKERFILE='Dockerfile'
17	18	IMAGE_TAG='kaggle/python-build'
	19	+ BUILD_ARGS=''
18	20
19	21	while :; do
20	22	case "$1" in
@@ -29,6 +31,15 @@ while :; do
29	31	-c|--use-cache)
30	32	CACHE_FLAG=''
31	33	;;
	34	+ -b|--base-image-tag)
	35	+ if [[ -z $2 ]]; then
	36	+ usage
	37	+ printf 'ERROR: No TAG specified after the %s flag.\n' "$1" >&2
	38	+ exit
	39	+ fi
	40	+ BUILD_ARGS="--build-arg BASE_TAG=$2"
	41	+ shift # skip the flag value
	42	+ ;;
32	43	-?*)
33	44	usage
34	45	printf 'ERROR: Unknown option: %s\n' "$1" >&2
@@ -44,6 +55,7 @@ done
44	55	readonly CACHE_FLAG
45	56	readonly DOCKERFILE
46	57	readonly IMAGE_TAG
	58	+ readonly BUILD_ARGS
47	59
48	60	set -x
49		- docker build --rm --pull $CACHE_FLAG -t "$IMAGE_TAG" -f "$DOCKERFILE" .
	61	+ docker build --rm --pull $CACHE_FLAG -t "$IMAGE_TAG" -f "$DOCKERFILE" $BUILD_ARGS .

Collapse file

‎gpu.Dockerfile‎

Copy file name to clipboardExpand all lines: gpu.Dockerfile

+3-1Lines changed: 3 additions & 1 deletion

Original file line number	Diff line number	Diff line change
@@ -1,6 +1,8 @@
	1	+ ARG BASE_TAG=staging
	2	+
1	3	FROM nvidia/cuda:9.2-cudnn7-devel-ubuntu16.04 AS nvidia
2	4	FROM gcr.io/kaggle-images/python-tensorflow-whl:1.12.0-py36 as tensorflow_whl
3		- FROM gcr.io/kaggle-images/python:staging
	5	+ FROM gcr.io/kaggle-images/python:${BASE_TAG}
4	6
5	7	ADD clean-layer.sh /tmp/clean-layer.sh
6	8

0 commit comments

Comments

0 (0)