FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: Feature/list tables page size by OmriBromberg · Pull Request #174 · googleapis/python-bigquery-sqlalchemy · GitHub

This repository was archived by the owner on May 16, 2026. It is now read-only.

feat: Feature/list tables page size - #174

Merged
jimfulton merged 10 commits into
googleapis:masterfrom
OmriBromberg:feature/list_tables_page_size
Jul 6, 2021
Merged

feat: Feature/list tables page size#174
jimfulton merged 10 commits into
googleapis:masterfrom
OmriBromberg:feature/list_tables_page_size

Conversation

OmriBromberg commented May 12, 2021
edited by jimfulton
Loading

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • [V] Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • [V] Ensure the tests and linter pass
  • [V] Code coverage does not decrease (if any source code was changed)
  • [V] Appropriate docs were updated (if necessary)

Fixes #173 🦕

Hey guys, I explained in the issue a bit but I wanted to implement it so users would be able to change max_results @ dataset.list_tables like the arraysize parameter so I copied how it is treated in the parse_url area and updated the tests and docs for it, the new parameter for it is called list_tables_page_size

also I saw that on line 606
there is this expression self.arraysize = self.arraysize or arraysize which always takes the default arraysize instead of the parsed querystring arraysize because self.arraysize is always initialized to a default value, so I also reversed it to be as so self.arraysize = arraysize or self.arraysize, now the querystring arraysize will take precedence over the default value

OmriBromberg requested a review from a team May 12, 2021 21:12
product-auto-label Bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label May 12, 2021

google-cla Bot commented May 12, 2021

Copy link
Copy Markdown

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

google-cla Bot added the cla: no This human has *not* signed the Contributor License Agreement. label May 12, 2021

Copy link
Copy Markdown
Contributor Author

@googlebot I signed it!

google-cla Bot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels May 12, 2021
OmriBromberg force-pushed the feature/list_tables_page_size branch from 5a97fa7 to 085df0f Compare May 12, 2021 21:28
…peration as list_tables_page_size

fix: arraysize default always takes priority over querystring
OmriBromberg force-pushed the feature/list_tables_page_size branch from 085df0f to 5331bcd Compare May 12, 2021 21:31
jimfulton changed the title Feature/list tables page size feat: Feature/list tables page size May 14, 2021
jimfulton added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels May 16, 2021
yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels May 16, 2021
jimfulton self-requested a review May 16, 2021 20:32

jimfulton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is a great start. In addition to adding a feature, you found two bugs!

This needs test of the logic that fetches lists of tables. A unit test would be fine and should be straightforward to write.

OmriBromberg commented May 19, 2021
edited
Loading

Copy link
Copy Markdown
Contributor Author

hey, regarding this can you explain this logic to me? why shouldn't it always take the "full_table_id" property? gives us inconsistencies

Copy link
Copy Markdown
Contributor

hey, regarding this can you explain this logic to me? why shouldn't it always take the "full_table_id" property? gives us inconsistencies

That's a good question.

If you look at: https://github.com/sqlalchemy/sqlalchemy/blob/b20b6f8fe7ea0198f819a0fd68ca076b6c760054/lib/sqlalchemy/testing/suite/test_reflection.py#L537-L579

SQLAlchemy expects the results to be unqualified, which we satisfy if we have a schema.

Looking at other dialects:

MS SQL requires a schema: https://github.com/sqlalchemy/sqlalchemy/blob/b20b6f8fe7ea0198f819a0fd68ca076b6c760054/lib/sqlalchemy/dialects/mssql/base.py#L2877

Oracle assumes there's a default: https://github.com/sqlalchemy/sqlalchemy/blob/b20b6f8fe7ea0198f819a0fd68ca076b6c760054/lib/sqlalchemy/dialects/oracle/base.py#L1759-L1760

as does Postgres: https://github.com/sqlalchemy/sqlalchemy/blob/b20b6f8fe7ea0198f819a0fd68ca076b6c760054/lib/sqlalchemy/dialects/postgresql/base.py#L3488-L3490

As does sqlite, sort of: https://github.com/sqlalchemy/sqlalchemy/blob/b20b6f8fe7ea0198f819a0fd68ca076b6c760054/lib/sqlalchemy/dialects/sqlite/base.py#L1977-L1981

and so on.

So BQ is a little different because there might not be a default.

Amongst the options:

  1. Error if no schema is provided and there's no default.
  2. Qualify the results if we don't have a schema. I prefer using just the data set id in this case. full_table_id adds clutter IMO.

Playing with this some more, you can list schemas in other projects by specifying project_id.schema_id (e.g. "bigquery-public-data.census_bureau_acs"). That makes the use of a colon in full_table_id a little weird.

jimfulton marked this pull request as draft May 24, 2021 20:21

Copy link
Copy Markdown
Contributor

The bigquery client now accepts a page_size parameter, so this PR wants to use that.

When your;re ready for me to review, please merge master to get rid of many spurious changes. :)

OmriBromberg marked this pull request as ready for review July 4, 2021 06:29
jimfulton added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Jul 6, 2021
yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jul 6, 2021

Copy link
Copy Markdown
Contributor

Thanks!

jimfulton merged commit e0f1496 into googleapis:master Jul 6, 2021
jimfulton pushed a commit to jimfulton/python-bigquery-sqlalchemy that referenced this pull request Jul 22, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dataset.list_tables max_results parameter

3 participants


Back | FazBrowse Home | New Git URL