Fix new partitioned table - #3381
Conversation
Sorry, something went wrong.
…w-partitioned-table
[This is a re-do of the pull request googleapis#3366 to fix the author email address to use my google.com email] I moved the partitioning_type check into the if/elif block to avoid a ValueError that should not have been thrown. See: https://cloud.google.com/bigquery/docs/creating-partitioned-tables#creat ing_a_partitioned_table and the "API" example. Before my change, the following code failed with the ValueError thrown by this function, but with my change, I was able to successfully create the partitioned table: destination_table = dataset.table(table_shortname) destination_partition = dataset.table(table_shortname+'$'+partition_date) if not destination_table.exists(): destination_table.partitioning_type = 'DAY' destination_table.create()
|
@dhermes @jonparrott |
Sorry, something went wrong.
|
Thanks @richkadel. This LGTM. @tswast Any issues? |
Sorry, something went wrong.
|
@richkadel Your code snippet above never uses destination_partition. Is there a typo somewhere? |
Sorry, something went wrong.
| raise ValueError("Set either 'view_query' or 'schema'.") | ||
| elif self.partitioning_type is None: | ||
| raise ValueError( | ||
| "Set either 'view_query' or 'schema' or 'partitioning_type'.") |
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
|
@dhermes re: @richkadel Your code snippet above never uses destination_partition. Is there a typo somewhere? No typo. The code that uses the partition itself is not shown. I left the line in to show that the table name used in the "create" call is the table name without the partition suffix. Once the table is created then you can use destination_partition value as the "job_destination" setting to materialize data into just that partition. |
Sorry, something went wrong.
|
@richkadel Gotcha, I just made an edit to make that clearer. |
Sorry, something went wrong.
As discussed with team, I removed the ValueError and any assumptions about the legitimate combinations of values in the request. ValueError unit test was removed, and the unused schema in `test_create_w_alternate_client` that was suppressed due to the `elif` in `_build_resource`, was removed so the expected value still matches the remaining inputs.
|
Updates have been pushed and all tests pass. |
Sorry, something went wrong.
tswast
left a comment
There was a problem hiding this comment.
Thank you!
Sorry, something went wrong.
|
OK @jonparrott also thinks this looks just fine. |
Sorry, something went wrong.
[This is a re-do of the pull request #3366 to fix the author email address to use my google.com email]
I moved the partitioning_type check into the if / elif block to avoid a ValueError that should not have been thrown.
See:
https://cloud.google.com/bigquery/docs/creating-partitioned-tables#creating_a_partitioned_table and the "API" example.
Before my change, the following code failed with the ValueError thrown by this function, but with my change, I was able to successfully create the partitioned table: