Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
If you are still having issues, please be sure to include as much information as possible:
Environment details
- OS type and version: MacOS Darwin Kernel Version 22.1.0
- Python version: 3.11
- pip version: 23.2.1
- google-cloud-bigquery version: 3.15.0
Steps to reproduce
- Create an Arrow backed dataframe with a large list field.
- Create a google.cloud.bigquery Client
- call Client.load_table_from_dataframe on this dataframe
Code example
# example
import pandas as pd
import pyarrow as pa
from google.cloud import bigquery as gbq
client= gbq.Client(
project=<project_id>,
credentials=<credentials>,
location=<location>,
)
df = pd.DataFrame({"x":pa.array(pd.Series([[2.2]*5]*10000000)).to_pandas(types_mapper=pd.ArrowDtype)})
client.load_table_from_dataframe(
df, 'temporary_tables.chunked_array_error')Stack trace
File "/Users/<redacted>", line 250, in create_table_from_dataframe
load_job = self.client.load_table_from_dataframe(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<redacted>/lib/python3.11/site-packages/google/cloud/bigquery/client.py", line 2671, in load_table_from_dataframe
new_job_config.schema = _pandas_helpers.dataframe_to_bq_schema(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<redacted>/lib/python3.11/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 465, in dataframe_to_bq_schema
bq_schema_out = augment_schema(dataframe, bq_schema_out)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<redacted>lib/python3.11/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 500, in augment_schema
arrow_table.values.type.id
^^^^^^^^^^^^^^^^^^
AttributeError: 'pyarrow.lib.ChunkedArray' object has no attribute 'values'
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!
Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
If you are still having issues, please be sure to include as much information as possible:
Environment details
Steps to reproduce
Code example
Stack trace
File "/Users/<redacted>", line 250, in create_table_from_dataframe load_job = self.client.load_table_from_dataframe( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/<redacted>/lib/python3.11/site-packages/google/cloud/bigquery/client.py", line 2671, in load_table_from_dataframe new_job_config.schema = _pandas_helpers.dataframe_to_bq_schema( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/<redacted>/lib/python3.11/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 465, in dataframe_to_bq_schema bq_schema_out = augment_schema(dataframe, bq_schema_out) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/<redacted>lib/python3.11/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 500, in augment_schema arrow_table.values.type.id ^^^^^^^^^^^^^^^^^^ AttributeError: 'pyarrow.lib.ChunkedArray' object has no attribute 'values'Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!