FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Invoke tensorflow-core-api deployment from script · Retror/java@7100e84 · GitHub

/ java Public
forked from tensorflow/java

Commit 7100e84

Browse files
karllessard
committed
Invoke tensorflow-core-api deployment from script
+remove support for Bintray (not used)
1 parent f184e23 commit 7100e84

8 files changed

Lines changed: 87 additions & 239 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ jobs:
8787
[[ "${{ github.event_name }}" == "push" ]] && MAVEN_PHASE=deploy || MAVEN_PHASE=install
8888
echo "<settings><servers><server><id>ossrh</id><username>${{ secrets.CI_DEPLOY_USERNAME }}</username><password>${{ secrets.CI_DEPLOY_PASSWORD }}</password></server></servers></settings>" > $HOME/.m2/settings.xml
8989
echo Executing Maven $MAVEN_PHASE
90-
mvn clean $MAVEN_PHASE -Possrh -B -U -e -Djavacpp.platform=linux-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl $NATIVE_BUILD_PROJECTS -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
90+
mvn clean $MAVEN_PHASE -B -U -e -Djavacpp.platform=linux-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl $NATIVE_BUILD_PROJECTS -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
9191
df -h
9292
macosx-x86_64:
9393
if: github.event_name == 'push' || contains(github.event.pull_request.labels.*.name, 'CI build')
@@ -117,7 +117,7 @@ jobs:
117117
echo "<settings><servers><server><id>ossrh</id><username>${{ secrets.CI_DEPLOY_USERNAME }}</username><password>${{ secrets.CI_DEPLOY_PASSWORD }}</password></server></servers></settings>" > $HOME/.m2/settings.xml
118118
df -h
119119
echo Executing Maven $MAVEN_PHASE
120-
mvn clean $MAVEN_PHASE -Possrh -B -U -e -Djavacpp.platform=macosx-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl $NATIVE_BUILD_PROJECTS -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
120+
mvn clean $MAVEN_PHASE -B -U -e -Djavacpp.platform=macosx-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl $NATIVE_BUILD_PROJECTS -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
121121
df -h
122122
windows-x86_64:
123123
if: github.event_name == 'push' || contains(github.event.pull_request.labels.*.name, 'CI build')
@@ -176,7 +176,7 @@ jobs:
176176
df -h
177177
wmic pagefile list /format:list
178178
echo Executing Maven %MAVEN_PHASE%
179-
call mvn clean %MAVEN_PHASE% -Possrh -B -U -e -Djavacpp.platform=windows-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl %NATIVE_BUILD_PROJECTS% -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
179+
call mvn clean %MAVEN_PHASE% -B -U -e -Djavacpp.platform=windows-x86_64 -Djavacpp.platform.extension=${{ matrix.ext }} -pl %NATIVE_BUILD_PROJECTS% -am -DstagingRepositoryId=${{ needs.prepare.outputs.stagingRepositoryId }}
180180
if ERRORLEVEL 1 exit /b
181181
df -h
182182
wmic pagefile list /format:list

‎RELEASE.md‎

Lines changed: 14 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,7 @@
33
The
44
[TensorFlow Java API](https://github.com/tensorflow/java) is available on Maven Central and JCenter
55
through artifacts uploaded to
6-
[OSS Sonatype](https://oss.sonatype.org/content/repositories/releases/org/tensorflow/) and/or
7-
[Bintray](https://bintray.com/google/tensorflow/tensorflow). This
6+
[OSS Sonatype](https://oss.sonatype.org/content/repositories/releases/org/tensorflow/). This
87
document describes the process of updating the release artifacts. It does _not_ describe how to use
98
the artifacts, for which the reader is referred to the
109
[TensorFlow for Java installation instructions](https://github.com/tensorflow/java/blob/master/README.md).
@@ -36,14 +35,6 @@ a [Docker](https://www.docker.com) container for a hermetic release process.
3635
account does not have permissions, then you'll need to ask someone who does
3736
to [file a ticket](https://issues.sonatype.org/) to add to the permissions
3837
([sample ticket](https://issues.sonatype.org/browse/MVNCENTRAL-1637)).
39-
- Optionally, an account at [bintray.com](https://bintray.com) that has permissions to
40-
update the [tensorflow repository](https://bintray.com/google/tensorflow).
41-
If your account does not have permissions, then you'll need to ask one of
42-
the [organization administrators](https://bintray.com/google) to give you
43-
permissions to update the `tensorflow` repository. Please keep the
44-
[repository option](https://bintray.com/google/tensorflow/edit?tab=general)
45-
to *"GPG sign uploaded files using Bintray's public/private key pair"*
46-
**unchecked**, otherwise it will conflict with locally signed artifacts.
4738
- A GPG signing key, required
4839
[to sign the release artifacts](http://central.sonatype.org/pages/apache-maven.html#gpg-signed-components).
4940

@@ -52,7 +43,7 @@ a [Docker](https://www.docker.com) container for a hermetic release process.
5243
#### Major or minor release
5344

5445
1. Get a clean version of the source code by cloning the
55-
[TensorFlow Java GitHub repository](https://github.com/tensorflow-java)
46+
[TensorFlow Java GitHub repository](https://github.com/tensorflow/java)
5647
```
5748
git clone https://github.com/tensorflow/java
5849
```
@@ -74,7 +65,7 @@ a [Docker](https://www.docker.com) container for a hermetic release process.
7465
#### Patch release
7566
7667
1. Get a clean version of the source code by cloning the
77-
[TensorFlow Java GitHub repository](https://github.com/tensorflow-java)
68+
[TensorFlow Java GitHub repository](https://github.com/tensorflow/java)
7869
```
7970
git clone https://github.com/tensorflow/java
8071
```
@@ -111,20 +102,17 @@ Some platforms cannot be build successfully on GitHub Actions, due to some limit
111102
(e.g. max 6 hours for a job). For this reasons, we need to build manually some of our artifacts on
112103
private servers.
113104
114-
To do so, follow the same steps as the [CI build](https://github.com/tensorflow/java/.github/workflows/ci.yml)
105+
To do so, follow the same steps as the [CI build](https://github.com/tensorflow/java/blob/master/.github/workflows/ci.yml)
115106
for the same platform and make sure to checkout the release branch and to provide your Sonatype credentials
116107
for temporary staging.
117108
118-
### Building and deploying all artifacts to Sonatype and Bintray
109+
### Building and deploying all artifacts to Sonatype
119110
120111
1. At the root of your TensorFlow Java copy, create a Maven settings.xml file with your OSSRH credentials and
121-
[Bintray API key](https://bintray.com/docs/usermanual/interacting/interacting_interacting.html#anchorAPIKEY),
122-
plus your GPG key passphrase:
112+
your GPG key passphrase:
123113
```sh
124114
SONATYPE_USERNAME="your_sonatype.org_username_here"
125115
SONATYPE_PASSWORD="your_sonatype.org_password_here"
126-
BINTRAY_USERNAME="your_bintray_username_here"
127-
BINTRAY_API_KEY="your_bintray_api_key_here"
128116
GPG_PASSPHRASE="your_gpg_passphrase_here"
129117
cat > settings.xml <<EOF
130118
<settings>
@@ -135,9 +123,9 @@ for temporary staging.
135123
<password>${SONATYPE_PASSWORD}</password>
136124
</server>
137125
<server>
138-
<id>bintray</id>
139-
<username>${BINTRAY_USERNAME}</username>
140-
<password>${BINTRAY_API_KEY}</password>
126+
<id>ossrh-staging</id>
127+
<username>${SONATYPE_USERNAME}</username>
128+
<password>${SONATYPE_PASSWORD}</password>
141129
</server>
142130
</servers>
143131
<profiles>
@@ -154,10 +142,8 @@ for temporary staging.
154142
</settings>
155143
EOF
156144
```
157-
2. Execute the `release.sh` script. This will deploy only on OSSRH by default. To enable Bintray
158-
deployment as well, pass the environment variable DEPLOY_BINTRAY=true. All native artifacts
159-
previously temporarily staged by GitHub Actions will be fetched, signed and redeployed with
160-
all the other artifacts.
145+
2. Execute the `release.sh` script. This will deploy artifacts on OSS Sonatype. All native artifacts
146+
previously temporarily staged by GitHub Actions will be fetched, signed and redeployed as well.
161147
162148
The script takes in paramater the sequence number of the staging repository created in OSSRH
163149
by the GitHub Actions workflow. You can retrieve this ID by looking in the staging repositories
@@ -168,33 +154,16 @@ for temporary staging.
168154
sh release.sh 1100
169155
```
170156
3. If the script above succeeds then the artifacts would have been uploaded to
171-
the private staging repository in Sonatype, and as unpublished artifacts in
172-
Bintray. After verifying the release, you should finalize or abort the
173-
release on both sites.
174-
175-
4. Visit https://oss.sonatype.org/#stagingRepositories, find the `org.tensorflow`
157+
the private staging repository in Sonatype. After verifying the release, you should finalize or
158+
abort the release. Visit https://oss.sonatype.org/#stagingRepositories, find the `org.tensorflow`
176159
release and click on either `Release` to finalize the release, or `Drop` to
177160
abort.
178161
179-
5. Visit https://bintray.com/google/tensorflow/tensorflow, and select the
180-
version you just uploaded. Notice there's a message about unpublished
181-
artifacts. Click on either `Publish` to finalize the release, or `Discard`
182-
to abort.
183-
184-
6. Some things of note:
162+
4. Some things of note:
185163
- For details, look at the [Sonatype guide](http://central.sonatype.org/pages/releasing-the-deployment.html).
186164
- Syncing with [Maven Central](http://repo1.maven.org/maven2/org/tensorflow/)
187165
can take 10 minutes to 2 hours (as per the [OSSRH
188166
guide](http://central.sonatype.org/pages/ossrh-guide.html#releasing-to-central)).
189-
- For Bintray details, refer to their guide on
190-
[managing uploaded content](https://bintray.com/docs/usermanual/uploads/uploads_managinguploadedcontent.html#_publishing).
191-
192-
#### Skip deploying to a repository
193-
194-
Should you need, setting environment variables `DEPLOY_OSSRH=false` or
195-
`DEPLOY_BINTRAY=false` when calling `release.sh` will skip deploying to OSSRH or
196-
Bintray respectively. Note that snapshots are only uploaded to OSSRH, so you
197-
cannot skip deploying to OSSRH for a `-SNAPSHOT` version.
198167
199168
### Finishing a release
200169
@@ -218,4 +187,3 @@ Go to GitHub and create a release tag at the release branch with a summary of wh
218187
- [Sonatype guide](http://central.sonatype.org/pages/ossrh-guide.html) for
219188
hosting releases.
220189
- [Ticket that created the `org/tensorflow` configuration](https://issues.sonatype.org/browse/OSSRH-28072) on OSSRH.
221-
- The [Bintray User Manual](https://bintray.com/docs/usermanual/index.html)

‎deploy.sh‎

Lines changed: 44 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/usr/bin/env bash
2-
# Copyright 2017 The TensorFlow Authors. All Rights Reserved.
2+
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -19,19 +19,15 @@
1919

2020
set -e
2121

22-
# By default we deploy to both ossrh and bintray. These two
23-
# environment variables can be set to skip either repository.
24-
DEPLOY_OSSRH="${DEPLOY_OSSRH:-true}"
25-
DEPLOY_BINTRAY="${DEPLOY_BINTRAY:-false}"
2622
IN_CONTAINER="${IN_CONTAINER:-false}"
2723

28-
if [[ "${DEPLOY_BINTRAY}" != "true" && "${DEPLOY_OSSRH}" != "true" ]]; then
29-
echo "Must deploy to at least one of Bintray, OSSRH or local" >&2
30-
exit 2
31-
fi
32-
3324
MVN_OPTIONS="-B -e --settings ${PWD}/settings.xml -Pdeploying"
3425

26+
mvn_property() {
27+
local property="$1"
28+
mvn exec:exec $MVN_OPTIONS -q -N -Dexec.executable='echo' -Dexec.args="\${${property}}"
29+
}
30+
3531
#
3632
# Clean the working directory before doing anything
3733
#
@@ -42,42 +38,39 @@ mvn clean $MVN_OPTIONS -q -U
4238
# Extract deployed version from POM
4339
#
4440
echo "Parsing Maven configuration..."
45-
TF_VERSION=`mvn exec:exec $MVN_OPTIONS -q -N -Dexec.executable='echo' -Dexec.args="\\${project.version}"`
41+
TF_VERSION=`mvn_property "project.version"`
4642
if [ -z "${TF_VERSION}" ]; then
4743
echo "Fail to extract TF_VERSION from POM"
4844
exit 1
4945
fi
50-
echo
51-
echo "Deploying TensorFlow Java, version ${TF_VERSION}, on:"
52-
if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
53-
echo "* OSSRH"
54-
fi
55-
if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
56-
echo "* Bintray"
57-
fi
58-
echo
5946

6047
#
6148
# Validate the environment based on if we are deploying a snapshot or a release version
6249
#
63-
NATIVE_ARTIFACTS_REPO=ossrh
64-
if ! [[ $TF_VERSION = *-SNAPSHOT ]]; then
50+
echo
51+
if [[ $TF_VERSION = *-SNAPSHOT ]]; then
52+
echo "===> Deploying TensorFlow Java, version ${TF_VERSION} <==="
53+
echo
54+
DEPLOY_FILE_GOAL=deploy:deploy-file
55+
DEPLOY_REPOSITORY_URL=`mvn_property "project.distributionManagement.snapshotRepository.url"`
56+
57+
else
58+
echo "===> Releasing TensorFlow Java, version ${TF_VERSION} <==="
59+
echo
6560
if [[ "${IN_CONTAINER}" != "true" ]]; then
6661
echo "Release must be executed from a Docker container, please make sure to invoke release.sh"
6762
exit 1
6863
fi
69-
if [[ -z "${STAGING_SEQ}" ]]; then
64+
if [[ -z "${STAGING_SEQ}" ]]; then
7065
echo "Staging sequence is required for release"
7166
exit 1
7267
fi
68+
DEPLOY_FILE_GOAL=gpg:sign-and-deploy-file
69+
DEPLOY_REPOSITORY_URL=`mvn_property "project.distributionManagement.repository.url"`
7370

74-
# In release mode, fetch the native artifacts from the staging repository created by the CI build
7571
MVN_OPTIONS="$MVN_OPTIONS -Preleasing -DstagingRepositoryId=orgtensorflow-${STAGING_SEQ}"
76-
NATIVE_ARTIFACTS_REPO=ossrh-staging
7772

78-
# gnupg2 is required for signing releases
79-
apt-get -qq update
80-
apt-get -qq install -y gnupg2
73+
apt-get -qq update && apt-get -qq install -y gnupg2
8174
fi
8275

8376
#
@@ -92,7 +85,7 @@ for p in `find tensorflow-core -name tensorflow-core-platform* -type d -exec bas
9285
if [[ -n $PLATFORM_EXT ]]; then
9386
[[ -n $PLATFORM_EXTS ]] && PLATFORM_EXTS="$PLATFORM_EXTS $PLATFORM_EXT" || PLATFORM_EXTS=$PLATFORM_EXT
9487
fi
95-
mvn dependency:copy-dependencies $MVN_OPTIONS -q -P$NATIVE_ARTIFACTS_REPO \
88+
mvn dependency:copy-dependencies $MVN_OPTIONS -q \
9689
-Djavacpp.platform.extension=$PLATFORM_EXT -DincludeArtifactIds=tensorflow-core-api \
9790
-DoutputDirectory=../../tensorflow-core/tensorflow-core-api/target -pl tensorflow-core/$p
9891
fi
@@ -105,48 +98,40 @@ done
10598
for f in tensorflow-core/tensorflow-core-api/target/tensorflow-core-api-$TF_VERSION-*.jar; do
10699
echo "Found native artifact: $f"
107100
if [[ $f =~ tensorflow-core-api-$TF_VERSION-(.*).jar ]]; then
108-
[[ -n $FILES ]] && FILES=$FILES,$f || FILES=$f
109-
[[ -n $TYPES ]] && TYPES=$TYPES,jar || TYPES=jar
110-
[[ -n $CLASSIFIERS ]] && CLASSIFIERS=$CLASSIFIERS,${BASH_REMATCH[1]} || CLASSIFIERS=${BASH_REMATCH[1]}
101+
[[ -n $NATIVE_FILES ]] && NATIVE_FILES=$NATIVE_FILES,$f || NATIVE_FILES=$f
102+
[[ -n $NATIVE_FILE_TYPES ]] && NATIVE_FILE_TYPES=$NATIVE_FILE_TYPES,jar || NATIVE_FILE_TYPES=jar
103+
[[ -n $NATIVE_CLASSIFIERS ]] && NATIVE_CLASSIFIERS=$NATIVE_CLASSIFIERS,${BASH_REMATCH[1]} || NATIVE_CLASSIFIERS=${BASH_REMATCH[1]}
111104
fi
112105
done
113106

114-
MVN_DEPLOY_OPTIONS="$MVN_OPTIONS -Dtensorflow.core.api.files=$FILES -Dtensorflow.core.api.types=$TYPES -Dtensorflow.core.api.classifiers=$CLASSIFIERS"
115-
116107
#
117-
# Build and deploy the artifacts on OSSRH and/or Bintray
108+
# Build and deploy the artifacts on OSSRH
118109
# We need to do it manually for all non-default platforms, as they are not automatically included as
119-
# modules in the POM and depends on the javacpp.platform.extension property
120-
#
121-
mvn_all_platforms() {
122-
local options="$1"
123-
mvn $options
124-
for p in $PLATFORM_EXTS; do
125-
mvn $options -Djavacpp.platform.extension=$p -pl tensorflow-core/tensorflow-core-platform$p
126-
done
127-
}
110+
# modules in the POM and depends on the javacpp.platform.extension property.
111+
# Note that the tensorflow-core-api, which needs special care, won't be deployed yet, see below.
112+
#
113+
mvn deploy $MVN_OPTIONS
114+
for p in $PLATFORM_EXTS; do
115+
mvn deploy $MVN_OPTIONS -Djavacpp.platform.extension=$p -pl tensorflow-core/tensorflow-core-platform$p
116+
done
128117

129-
if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
130-
mvn_all_platforms "deploy $MVN_DEPLOY_OPTIONS -Possrh"
131-
fi
132-
# Deploy artifacts to bintray if requested.
133-
if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
134-
mvn_all_platforms "deploy $MVN_DEPLOY_OPTIONS -Pbintray"
135-
fi
118+
# Now deploy manually the tensorflow-core-api with all its native artifacts.
119+
mvn $DEPLOY_FILE_GOAL $MVN_OPTIONS -pl tensorflow-core/tensorflow-core-api \
120+
-DgroupId=org.tensorflow -DartifactId=tensorflow-core-api -Dversion=$TF_VERSION -Dpackaging=jar \
121+
-Dfile=target/tensorflow-core-api-$TF_VERSION.jar \
122+
-Dfiles=$NATIVE_FILES -Dtypes=$NATIVE_FILE_TYPES -Dclassifiers=$NATIVE_CLASSIFIERS \
123+
-Dsources=target/tensorflow-core-api-$TF_VERSION-sources.jar \
124+
-Djavadoc=target/tensorflow-core-api-$TF_VERSION-javadoc.jar \
125+
-DpomFile=pom.xml \
126+
-DrepositoryId=ossrh -Durl=$DEPLOY_REPOSITORY_URL
136127

137128
echo
138129
if [[ $TF_VERSION = *-SNAPSHOT ]]; then
139130
echo "Uploaded to snapshot repository"
140131
else
141132
echo "Uploaded to the staging repository"
142133
echo "After validating the release: "
143-
if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
144-
echo "* Login to https://oss.sonatype.org/#stagingRepositories"
145-
echo "* Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort"
146-
fi
147-
if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
148-
echo "* Login to https://bintray.com/google/tensorflow/tensorflow"
149-
echo "* Either 'Publish' unpublished items to release, or 'Discard' to abort"
150-
fi
134+
echo "* Login to https://oss.sonatype.org/#stagingRepositories"
135+
echo "* Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort"
151136
fi
152137
echo

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL