@dhermes qq: did we resolve the namespace/import issue? Will users be able to do;

from google.cloud import storage

storage_client = storage.Client()

?

Other than the services in #2258 (logging and pubsub) everything is a normal package (not a namespace package)

Can I be a jerk and block the release on that issue? I would prefer not to have to update sample code twice.

@bjwatson Thoughts on timing?

@jonparrott We are just in a state of flux until a release is cut.

@dhermes how bad is the state of flux? is it better to push this out quickly or wait for #2258 to be resolved? Are they any other issues that would make upgrading more trivial than just updating imports?

(Apologies for the slew of questions, I just want to make sure I understand the full picture)

@dhermes also, do we have a changelog?

I have yet to make the changelog. I'm a bit weary of doing it, it'll be big. Also some people may have started doing squash merges so my typical script won't see those PRs?

@dhermes @jonparrott I would like to resolve the following first if possible:

• Support Cloud Client Libraries on Google App Engine standard #1893 (comment) (issue Jacob raised about gRPC dependency remaining in google/longrunning/operations_pb2.py)
• Remove pubsub and logging as namespace packages #2258 (I'll work on this next; I think we have agreement on the path structure, and just need to make sure it works)
• Remove references to grpc.beta gax-python#127 (LGTM'd, but need to be merged and released, and the gax and gcloud dependencies updated)

I can be talked out of any of these if there are urgent overriding needs to get this release out.

@bjwatson Do you have a timeframe on this? e.g. why hasn't the LGTM'd PR been merged?

@dhermes

• Support Cloud Client Libraries on Google App Engine standard #1893 (comment): this will be done today; it's taken much longer than expected due to the many (albeit valuable) discussions about how best to manage the gRPC dependencies
• Remove pubsub and logging as namespace packages #2258: realisitically, I won't be able to get this done before Tuesday or Wednesday of next week, unless @geigerj can take this sooner
• Remove references to grpc.beta gax-python#127: @geigerj can you merge and publish this today?

@bjwatson gax-python 0.13.0 has been published, although until we refresh codegen, the gax-google-... packages will still be restricted to 0.12.x.

We can refresh the packages as part of #2258, unless @dhermes needs this done sooner.

It'd be nice to have it in today, but I trust you're going as quick as you can.

@dhermes I am. Regarding #2258, we can reduce risk and maybe get it done sooner by just focusing on the GAX namespace.

The other part of the problem is fixing the gRPC namespace, mainly to be more consistent, but also to avoid conflicts for the ML libraries (the gRPC packages currently use google.cloud.{api} as a namespace for language, vision, and speech). We can hand-fix the ML gRPC packages ASAP, and then fix the other gRPC packages a little later.

That SGTM.

This would be great to get a 0.19.0 onto PyPy PyPI since the Datastore v1beta2 API is being deprecated on 9/30

https://cloud.google.com/datastore/release-notes

This is a follow-up to our recent email to remind you that Google Cloud Datastore is turning down the v1beta1 and v1beta2 API on September 30, 2016.

The previous 0.18.1 release had v1beta2.

@overset Thanks for the heads up. FWIW, we haven't supported v1beta2 for a very long time.

• We upgraded from v1beta3 to v1 on August 16, 2016 (c52d363).
• We upgraded from v1beta2 to v1beta3 on January 20, 2016 (64c28fd)

Check out the release history for relevant releases near those dates

@dhermes My previous comment was flat-out wrong and thanks for confirming 0.18.1 having upgdaded to Datastore v1beta3 via 64c28fd. We haven't seen a turn-down notice for v1beta3 but it is marked as deprecated. (Thanks for the s/PyPy/PyPI/ on the previous comment as well)

Sure thing. Not sure when v1beta3 will be turned off.

@dhermes @tseaver @bjwatson, unless I'm missing something, I think we have all the items for this done?

We might want to have a PR to switch the naming back from #2264 or add it to this PR.

Time to make release notes. Yay for 130+ PRs!

$ git log -1 --pretty=%H
6fb2d73ff6350590aa061bcd902422828340f337
$ git log 0.18.1..HEAD | grep 'Merge pull request ' |  awk -F ' ' '{print $4}' | sort

yields

#2035
#2039
#2050
#2051
#2052
#2054
#2056
#2058
#2060
#2062
#2065
#2071
#2078
#2081
#2082
#2085
#2086
#2087
#2088
#2089
#2090
#2094
#2095
#2096
#2097
#2099
#2100
#2103
#2106
#2107
#2109
#2110
#2115
#2122
#2124
#2129
#2135
#2139
#2141
#2147
#2148
#2149
#2150
#2151
#2153
#2155
#2161
#2162
#2164
#2165
#2167
#2171
#2172
#2174
#2175
#2178
#2179
#2181
#2187
#2188
#2189
#2191
#2195
#2197
#2199
#2202
#2203
#2210
#2211
#2212
#2213
#2214
#2215
#2219
#2223
#2224
#2227
#2235
#2236
#2237
#2240
#2241
#2244
#2245
#2247
#2248
#2249
#2250
#2251
#2252
#2254
#2257
#2260
#2261
#2262
#2265
#2267
#2272
#2273
#2276
#2279
#2280
#2283
#2284
#2286
#2288
#2290
#2292
#2293
#2295
#2299
#2302
#2303
#2304
#2306
#2307
#2308
#2310
#2311
#2319
#2320
#2321
#2323
#2331
#2333
#2334
#2335
#2336
#2338
#2339
#2345

I also used the UI to look through merged PRs since August 9. I realized part way through that 0.18.1 wasn't a real release, it was a patch release due to broken PyPI deps, so I wanted to be extra careful.

Here is the release notes with stuff left out after. @daspecster Feel free to edit the vision parts inline as much as you like

Core

• Rename gcloud package as google.cloud and move on
  PyPI from gcloud to google-cloud (Change gcloud package to google.cloud #2223, Removing pubsub and logging as namespace packages. #2334, fix README installation command and PyPI links #2264, Reverting user-agent change that came with rename. #2284)
• Drop support for Python 2.6 (Drop support for Python 2.6. #2051)
• Upgrade to GAX 0.13.0 (Upgrade dependencies to gapic-* 0.9.x packages #2324, Fixing broken exception handling after GAX 0.13.0 upgrade. #2336)
• Remove gRPC / GAX restriction to Python 2.7 and removing the
  gcloud[grpc] extra. (Remove gRPC/GAX conditional checks. #2308, Remove stale gRPC warning from Bigtable docs. #2203)
• Add missing exception class for '502 Bad Gateway' response code (Add missing exception class for '502 Bad Gateway' response code. #2078)
• Bulk spelling / style / grammar docs fix (Fix Numerous Spelling, Style, English Issues #2139)
• Upgrade all usage of gRPC from beta to stable (Removing usage of the gRPC beta subpackage. #2149)
• Add shared Operation class for long-running operations (Factor out shared 'Operation' class and helpers. #2165)
• Rename core docs: gcloud-api to google-cloud-api, gcloud-auth
  to google-cloud-auth and gcloud-config to google-cloud-config (Updating RTD links and renaming RST docs with gcloud-* prefix #2280)
• Modify user agent metadata for gRPC (Reverting user-agent change that came with rename. #2284, Upgrading gRPC user agent from secondary to primary. #2286)

Resource Manager

• Add alpha warning to docs (Add note about resource_manager projects being an alpha feature. #2345)

Datastore

• Upgrade from v1beta3 to v1 (Updating datastore to use v1 #2107)
• Add gRPC support (Factor out datastore API surface. #2097, Enable gRPC for datastore API. #2099)
• BREAKING CHANGE: Making batches and transactions more robust to
  failure (Making datastore batch/transaction more robust to failure. #2303, fixes Transaction is left open when error happens in Datastore API #2297). Now a Batch must call begin()
  before adding mutations.
• Add emulator support for gRPC implementation (Upgrading datastore emulator to work with gRPC #2244)

BigQuery

• Add partition support (Add partition support to BigQuery #2058)
• Fix docs error about starting a job (Replace 'job.submit' w/ 'job.begin' in usage docs. #2338)
• Add QueryJob.results method (Add 'QueryJob.results' method #2321)
• Add desiganted schema module (Slightly more explicit coverage for partition-related methods. #2060)
• Fix docs typo about destination table (Fix 'destination_table' attribute typo in BQ usage docs. #2250)
• Update state values in usage doc (Poll async jobs using correct state logic in BQ usage docs. #2251)
• Add list_projects method (Add 'Client.list_projects'. #2254)
• Add configuration properties for maximumBillingTier and
  maximumBytesBilled for QueryJobs (Bigquery: Adding configuration properties for maximumBillingTier and maximumBytesBilled for QueryJobs #2276)
• Support parsing DATE, INT64, and FLOAT64 columns from Standard SQL tables (Support parsing DATE columns from Standard SQL tables #2265, Add support for Standard SQL dialect 'INT64' and 'FLOAT64' types. #2347)

Vision

• Initial usage documentation and discussion Add Vision API usage documentation. #2148
• Connection and Client Add Vision API connection. #2167, Add Vision Client #2179
• Face detection API Add Vision API Face detection. #2224
• Label detection API Add vision label detection #2237
• Logo detection API Vision logo detection #2235
• Landmark detection API Add vision landmark detection #2236
• Text detection API Vision text detection #2299
• Safe search detection API Add Vision safe search detection. #2319
• Image properties detection API Vision image properties #2320

Pub / Sub

• Re-enable gRPC / GAX by default if it is importable. (Re-enable GAX by default if it is importable. #2085)
• Fix bug mapping publish timestamp to result in
  gRPC / GAX implementation (Map pb 'publish_time' -> JSON 'publishTimestamp' for received messges. #2262)
• Add emulator support for gRPC / GAX implementation (Updating Pub / Sub GAX calls to work with emulator. #2245)

Logging

• Re-enable gRPC / GAX by default if it is importable. (Re-enable GAX by default if it is importable. #2085)
• Map gRPC exceptions onto exceptions from
  google.cloud.exceptions (logger_delete: map gRPC 'NOT_FOUND' error onto our 'NotFound' exception. #2094)
• Rename product (Stackdriver) (Stackdriver product rename #2273)
• Docs bug: Rename filter to filter_ (Update 'filter' -> 'filter_' in logging examples. #2279)

Bigtable

• Move HappyBase into its own project (Excise 'gcloud.bigtable.happybase'. #2109)
• Remove Bigtable undelete (not present in v2) (Remove all traces of 'undelete'. #2210)
• Add support for Bigtable emulator (Updating Bigtable stub creation to work with emulator. #2249)

Monitoring

• Add Group support (Add Group support to the Monitoring API #2035)
• Add Factory Methods for Metric/Resource/Timeseries (Add Factory Methods for Metric/Resource/Timeseries #2071)
• Fix ValueType link in docs (Fix value type link #2150)
• Add support for comparison filters (Add support for the comparison filters of Google Monitoring API #2234)
• Add docs in README (Add documentation for Stackdriver Monitoring in the README. #2054)

Natural Language

• Initial implementation (Adding usage doc for Natural Language API. #2062, Implement basic connection and client for Natural Language API #2162, Implement Document factory constructors on language client #2164, Adding Document.analyze_entities() in language package #2172, Updating language implementation after some review comments #2187, Implementing analyze sentiment method and helpers in language #2189, Adding annotate_text() method in language and helper classes needed for it #2211)

Not in notes.
Misc: #2339, #2335, #2311, #2065, #2067, #2085, #2088, #2090, #2096, #2097, #2135, #2141, #2151, #2212, #2227, #2241, #2257, #2272, #2288, #2292, #2304, #2056
Docs stuff (some dead code): #2052, #2103, #2110, #2039, #2115, #2129, #2122, #2155, #2147, #2171, #2175, #2178, #2188, #2191, #2174, #2199, #2153, #2219, #2247, #2267, #2269, #2282, #2281, #2302
Refactor private helpers: #2082, #2124, #2240
Misc. System test changes: #2313, #2050, #2087, #2202
Generated code: #2081
System test robustness: #2333, #2293, #2331, #2248, #2323, #2307, #2310, #2306, #2089, #2100, #2086, #2106, #2095, #2161, #2181, #2195, #2197, #2214, #2213, #2215, #2260, #2261, #2252, #2283, #2290, #2295

I've tweaked the entries all into imperative mode.

Thanks @tseaver

I updated vision to be more verbose so it's easier to reference later if needed.

Content moved to #2336.

@tseaver @daspecster Are we good to merge here? (Don't know why Travis is being fast on "push" builds to master but not "pr" builds.)

Green!

LGTM

Pushed the tag and added the notes: https://github.com/GoogleCloudPlatform/google-cloud-python/releases/tag/0.19.0 (may need #2347 update)

Push to PyPI should come from: https://travis-ci.org/GoogleCloudPlatform/google-cloud-python/builds/161121136

I updated the release notes to include the changes from #2347.