String cron_string = BRANCH_NAME == "master" ? "H 12 * * 1,3" : ""

pipeline {

agent { label 'ephemeral-linux' }

}

}

stage('Push CPU Pretest Image') {

steps {

slackSend color: 'none', message: "*<${env.BUILD_URL}console|${JOB_NAME} pushing pretest image>* ${GIT_COMMIT_SUMMARY}", channel: env.SLACK_CHANNEL

sh '''#!/bin/bash

}

}

}

}

stage('Push GPU Pretest Image') {

agent { label 'ephemeral-linux-gpu' }

steps {

slackSend color: 'none', message: "*<${env.BUILD_URL}console|${JOB_NAME} pushing pretest image>* ${GIT_COMMIT_SUMMARY}", channel: env.SLACK_CHANNEL

sh '''#!/bin/bash

}

}

[Kaggle Kernels](https://www.kaggle.com/kernels) allow users to run a Python Notebook in the cloud against our competitions and datasets without having to download data or set up their environment.

Our Python Docker images are stored on Google Container Registry at:

* CPU-only: [gcr.io/kaggle-images/python](https://gcr.io/kaggle-images/python)

* GPU: private for now, we will make it public soon.

Note: The base image for the GPU image is our CPU-only image. The [gpu.Dockerfile](gpu.Dockerfile) adds a few extra layers to install GPU related libraries and packages (cuda, libcudnn, pycuda etc.) and reinstall packages with specific GPU builds (torch, tensorflow and a few mores).

To get started with this image, read our [guide](http://blog.kaggle.com/2016/02/05/how-to-get-started-with-data-science-in-containers/) to using it yourself, or browse [Kaggle Kernels](https://www.kaggle.com/kernels) for ideas.

If you the first step above doesn't work for your use case, [open an issue](https://github.com/Kaggle/docker-python/issues/new) or a [pull request](https://github.com/Kaggle/docker-python/pulls).

1. Update the *Dockerfile*

1. For changes specific to the GPU image, update the [gpu.Dockerfile](gpu.Dockerfile).

1. Otherwise, update the [Dockerfile](Dockerfile).

1. Follow the instructions below to build a new image.

1. Add tests for your new package. See this [example](https://github.com/Kaggle/docker-python/blob/master/tests/test_fastai.py).

1. Follow the instructions below to test the new image.

1. Open a PR on this repo and you are all set!

./build

* `--gpu` to build an image for GPU.

* `--use-cache` for faster iterative builds.

A suite of tests can be found under the `/tests` folder. You can run the test using this command:

./test

Flags:

* `--gpu` to test the GPU image.

For the CPU-only image:

docker run --rm -it kaggle/python-build /bin/bash

docker run --rm -it gcr.io/kaggle-images/python /bin/bash

For the GPU image:

docker run --runtime nvidia --rm -it kaggle/python-gpu-build /bin/bash

To ensure your container can access the GPU, follow the instructions posted [here](https://github.com/Kaggle/docker-python/issues/361#issuecomment-448093930).

A Tensorflow custom pre-built wheel is used mainly for:

* Faster build time: Building tensorflow from sources takes ~1h. Keeping this process outside the main build allows faster iterations when working on our Dockerfiles.

Building Tensorflow from sources:

* Increase performance: When building from sources, we can leverage CPU specific optimizations

* Is required: Tensorflow with GPU support must be built from sources

The [Dockerfile](tensorflow-whl/Dockerfile) and the [instructions](tensorflow-whl/README.md) can be found in the [tensorflow-whl folder/](tensorflow-whl/).

RUN pip install pycuda && \

pip install cupy-cuda92 && \

pip install pynvrtc && \

/tmp/clean-layer.sh

import os

from google.auth import credentials

from google.cloud import bigquery

1. Add an entry in the [CHANGELOG](CHANGELOG.md) with an appropriate `LABEL`.

2. Push the new image using the `LABEL` you picked above.

FROM kaggle/python-tensorflow-whl as tensorflow_whl

FROM gcr.io/kaggle-images/python-tensorflow-whl:<LABEL> as tensorflow_whl

set -e

IMAGE_TAG='kaggle/python-build'

IMAGE_TAG_OVERRIDE=''

ADDITONAL_OPTS=''

PATTERN='test*.py'

usage() {

cat << EOF

}

while :; do

case "$1" in

-h|--help)

IMAGE_TAG_OVERRIDE=$2

shift # skip the flag value

;;

-p|--pattern)

if [[ -z $2 ]]; then

usage

printf 'ERROR: No PATTERN specified after the %s flag.\n' "$1" >&2

exit

fi

PATTERN="$2"

shift # skip the flag value

;;

-?*)

usage

printf 'ERROR: Unknown option: %s\n' "$1" >&2

readonly IMAGE_TAG

readonly ADDITONAL_OPTS

readonly PATTERN

set -x

rm -rf /tmp/python-build

-w=/working \

$ADDITONAL_OPTS \

"$IMAGE_TAG" \

/bin/bash -c "python -m unittest discover -s /input/tests -p $PATTERN"

import unittest

from pytext.config.field_config import FeatureConfig

from pytext.data.featurizer import InputRecord, SimpleFeaturizer

class TestPyText(unittest.TestCase):

def test_tokenize(self):

featurizer = SimpleFeaturizer.from_config(

SimpleFeaturizer.Config(), FeatureConfig()

)

tokens = featurizer.featurize(InputRecord(raw_text="At eight o'clock")).tokens

self.assertEqual(['at', 'eight', "o'clock"], tokens)

import unittest

import pandas as pd

import numpy as np

from tsfresh import extract_features

class TestTsFresh(unittest.TestCase):

def test_extract_feature(self):

ts = pd.DataFrame({

'id': np.array(['a', 'a', 'a', 'b', 'b', 'b']),

'time': np.array([0,1,2,0,1,2]),

'x': np.array([3,4,5,7,8,10])

})

extracted_features = extract_features(ts, column_id='id', column_sort='time')

self.assertEqual(2, len(extracted_features))