Commit 843eec8
committed
Update README.md for Beta
1 parent 5adcf85 commit 843eec8
1 file changed
Lines changed: 49 additions & 33 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -1,20 +1,39 @@ | |||
| 1 | - # Cloud Dataflow Java SDK (Alpha) | ||
| 1 | + # Google Cloud Dataflow SDK for Java (Beta) | ||
| 2 | 2 | ||
| 3 | - [Google Cloud Dataflow](https://cloud.google.com/dataflow/) | ||
| 4 | - provides a simple, powerful programming model for building both batch | ||
| 5 | - and streaming parallel data processing pipelines. | ||
| 3 | + [Google Cloud Dataflow](https://cloud.google.com/dataflow/) provides a simple, | ||
| 4 | + powerful programming model for building both batch and streaming parallel data | ||
| 5 | + processing pipelines. This repository hosts the open-sourced Cloud Dataflow SDK | ||
| 6 | + for Java, which can be used to run pipelines against the Google Cloud Dataflow | ||
| 7 | + Service. | ||
| 8 | + | ||
| 9 | + The contents of this repository are also available as released artifacts in the | ||
| 10 | + [Maven Central Repository](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.google.cloud.dataflow%22). | ||
| 11 | + You can bypass this GitHub repository and depend directly on the released | ||
| 12 | + artifacts from Maven Central by adding the following dependency to development | ||
| 13 | + environments like Eclipse or Apache Maven: | ||
| 14 | + | ||
| 15 | + <dependency> | ||
| 16 | + <groupId>com.google.cloud.dataflow</groupId> | ||
| 17 | + <artifactId>google-cloud-dataflow-java-sdk-all</artifactId> | ||
| 18 | + <version>version_number</version> | ||
| 19 | + </dependency> | ||
| 20 | + | ||
| 21 | + Please replace `version_number` with one of the supported versions from our | ||
| 22 | + [Release Notes](https://cloud.google.com/dataflow/release-notes/java). | ||
| 6 | 23 | ||
| 7 | 24 | ## Status [](https://travis-ci.org/GoogleCloudPlatform/DataflowJavaSDK) | |
| 8 | 25 | ||
| 9 | - The Cloud Dataflow SDK is used to access the Google Cloud Dataflow | ||
| 10 | - service, which is currently in Alpha and restricted to whitelisted users. | ||
| 26 | + The SDK is publicly available as a Beta release, and might be changed in | ||
| 27 | + backward-incompatible ways. | ||
| 28 | + | ||
| 29 | + The Google Cloud Dataflow Service is also publicly available in Beta under the | ||
| 30 | + following conditions: | ||
| 11 | 31 | ||
| 12 | - The SDK is publicly available and can be used for local execution by anyone. | ||
| 13 | - Note, however, that the SDK is also an Alpha release and may change | ||
| 14 | - significantly over time. The SDK is built to be extensible and support | ||
| 15 | - additional execution environments ("runners") beyond local execution and the | ||
| 16 | - Google Cloud Dataflow service. As the product matures, we look forward to | ||
| 17 | - working with you to improve Cloud Dataflow. | ||
| 32 | + * Your use of Google Cloud Dataflow is governed by the Google Cloud Platform | ||
| 33 | + Terms of Service. The foregoing notwithstanding, Google Cloud Dataflow is | ||
| 34 | + currently in Beta release and might be changed in backward-incompatible ways. | ||
| 35 | + It is not subject to any SLA or deprecation policy and is not recommended for | ||
| 36 | + production use. | ||
| 18 | 37 | ||
| 19 | 38 | ## Overview | |
| 20 | 39 | ||
@@ -38,28 +57,30 @@ runs the pipeline on your local machine. | |||
| 38 | 57 | 2. The [`DataflowPipelineRunner`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/sdk/src/main/java/com/google/cloud/dataflow/sdk/runners/DataflowPipelineRunner.java) | |
| 39 | 58 | submits the pipeline to the Dataflow Service, where it runs using managed | |
| 40 | 59 | resources in the [Google Cloud Platform](https://cloud.google.com) (GCP). | |
| 41 | - 3. The | ||
| 42 | - [`BlockingDataflowPipelineRunner`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/sdk/src/main/java/com/google/cloud/dataflow/sdk/runners/BlockingDataflowPipelineRunner.java) | ||
| 60 | + 3. The [`BlockingDataflowPipelineRunner`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/sdk/src/main/java/com/google/cloud/dataflow/sdk/runners/BlockingDataflowPipelineRunner.java) | ||
| 43 | 61 | submits the pipeline to the Dataflow Service via the `DataflowPipelineRunner` | |
| 44 | 62 | and then prints messages about the job status until the execution is complete. | |
| 45 | 63 | ||
| 46 | - _The Dataflow Service is currently in the Alpha phase of development and | ||
| 47 | - access is limited to whitelisted users._ | ||
| 48 | - | ||
| 49 | - Additionally, in partnership with [Cloudera](https://www.cloudera.com/), you can | ||
| 50 | - run Dataflow pipelines on an [Apache Spark](https://spark.apache.org/) backend. | ||
| 51 | - The relevant runner code is hosted in | ||
| 52 | - [this](https://github.com/cloudera/spark-dataflow) repository. | ||
| 64 | + The SDK is built to be extensible and support additional execution environments | ||
| 65 | + beyond local execution and the Google Cloud Dataflow Service. In partnership | ||
| 66 | + with [Cloudera](https://www.cloudera.com/), you can run Dataflow pipelines on | ||
| 67 | + an [Apache Spark](https://spark.apache.org/) backend using the | ||
| 68 | + [SparkPipelineRunner](https://github.com/cloudera/spark-dataflow). | ||
| 69 | + Additionally, you can run Dataflow pipelines on an | ||
| 70 | + [Apache Flink](https://flink.apache.org/) backend using the | ||
| 71 | + [FlinkPipelineRunner](https://github.com/dataArtisans/flink-dataflow). | ||
| 53 | 72 | ||
| 54 | 73 | ## Getting Started | |
| 55 | 74 | ||
| 56 | - This repository consists of two modules: | ||
| 75 | + This repository consists of three parts: | ||
| 57 | 76 | ||
| 58 | - * [`SDK`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/sdk) | ||
| 77 | + * The [`SDK`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/sdk) | ||
| 59 | 78 | module provides a set of basic Java APIs to program against. | |
| 60 | - * [`Examples`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples) | ||
| 79 | + * The [`Examples`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples) | ||
| 61 | 80 | module provides a few samples to get started. We recommend starting with the | |
| 62 | - WordCount example. | ||
| 81 | + `WordCount` example. | ||
| 82 | + * The [`Contrib`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/contrib) | ||
| 83 | + directory hosts community-contributed Dataflow modules. | ||
| 63 | 84 | ||
| 64 | 85 | The following command will build both modules and install them in your local | |
| 65 | 86 | Maven repository: | |
@@ -81,18 +102,13 @@ You can speed up the build and install process by using the following options: | |||
| 81 | 102 | from the local repository (or Maven Central) even if you have changed it | |
| 82 | 103 | locally. | |
| 83 | 104 | ||
| 84 | - 3. To run Maven using multiple threads, run: | ||
| 85 | - | ||
| 86 | - mvn -T 4 install | ||
| 87 | - | ||
| 88 | 105 | If you are using [Eclipse](https://eclipse.org/) integrated development | |
| 89 | 106 | environment (IDE), please additionally review our | |
| 90 | 107 | [Eclipse integration instructions](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/eclipse/README.md). | |
| 91 | 108 | ||
| 92 | - ## Running the Examples | ||
| 93 | - | ||
| 94 | - After building and installing, you can execute the `WordCount` and other example | ||
| 95 | - pipelines by following the instructions in this [README](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/README.md). | ||
| 109 | + After building and installing, you can execute the `WordCount` and other | ||
| 110 | + example pipelines by following the instructions in this | ||
| 111 | + [README](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/README.md). | ||
| 96 | 112 | ||
| 97 | 113 | ## Contact Us | |
| 98 | 114 | ||
0 commit comments