FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: add `BigQueryWriteClient` where `append_rows` returns a helper … · googleapis/google-cloud-python@90fd7c6 · GitHub

/ google-cloud-python Public

Commit 90fd7c6

Browse files
authored and
parthea
committed
feat: add BigQueryWriteClient where append_rows returns a helper for writing rows (#284)
* WIP: write client sample * add sample with nullable types * add schema for all supported types * add complex types to code sample * refactor sample so that it can be tested * make test assertions more thorough * fix lint error * remove done TODO * address reviewer comments * fix tag mismatch * test on multiple regions * correct comments about why offset exists * upgrade g-c-b * WIP: invert stream using BiDi class * WIP: attempt to use Future for send instead * WIP: use futures, populated by background consumer * make sure stream is actually open before returning from open * copy close implementation from pub/sub * support extra metadata * process exceptions, add open timeout * sort imports * WIP: unit tests * drain futures when stream closes * update docs * add callbacks to detect when a stream fails * add unit tests * add sleep to loop waiting for RPC to be active * don't freeze if initial RPC fails * add needed initializations so done() functions * fail fast when there is a problem with the initial request * don't inherit concurrent.futures It's unnecessary and kept resulting in stuff getting stuck. * add unit test for open timeout * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * add manual client to docs * typo in sample comments * force timeout and metadata to be kwargs * unify interface for sending row data * pull stream name from merged request * require newer proto-plus for copy_from method Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent 683c769 commit 90fd7c6

24 files changed

Lines changed: 1943 additions & 41 deletions

File tree

‎packages/google-cloud-bigquery-storage/docs/bigquery_storage_v1beta2/library.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,7 @@ Bigquery Storage v1beta2 API Library
44
.. automodule:: google.cloud.bigquery_storage_v1beta2.client
55
:members:
66
:inherited-members:
7+
8+
.. automodule:: google.cloud.bigquery_storage_v1beta2.writer
9+
:members:
10+
:inherited-members:

‎packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1beta2/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,15 @@ class BigQueryReadClient(client.BigQueryReadClient):
3030
__doc__ = client.BigQueryReadClient.__doc__
3131

3232

33+
class BigQueryWriteClient(client.BigQueryWriteClient):
34+
__doc__ = client.BigQueryWriteClient.__doc__
35+
36+
3337
__all__ = (
3438
# google.cloud.bigquery_storage_v1beta2
3539
"__version__",
3640
"types",
3741
# google.cloud.bigquery_storage_v1beta2.client
3842
"BigQueryReadClient",
43+
"BigQueryWriteClient",
3944
)

‎packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1beta2/client.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,14 @@
1919
This is the base from which all interactions with the API occur.
2020
"""
2121

22-
from __future__ import absolute_import
23-
2422
import google.api_core.gapic_v1.method
23+
import google.api_core.retry
2524

2625
from google.cloud.bigquery_storage_v1 import reader
27-
from google.cloud.bigquery_storage_v1beta2.services import big_query_read
26+
from google.cloud.bigquery_storage_v1beta2.services import (
27+
big_query_read,
28+
big_query_write,
29+
)
2830

2931

3032
_SCOPES = (
@@ -135,3 +137,7 @@ def read_rows(
135137
offset,
136138
{"retry": retry, "timeout": timeout, "metadata": metadata},
137139
)
140+
141+
142+
class BigQueryWriteClient(big_query_write.BigQueryWriteClient):
143+
__doc__ = big_query_write.BigQueryWriteClient.__doc__

‎packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1beta2/exceptions.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
16+
class StreamClosedError(Exception):
17+
"""Operation not supported while stream is closed."""

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL