| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
|
If we aren't going to marshall the value, then we should probably be preventing assignment of it in __setitem__() and update(). Otherwise, we might need to fabricate a custom entity_value to be used for this case. |
Sorry, something went wrong.
|
@tseaver entity = Entity(dataset, 'test')
entity['a'] = []
entity['a'].append(1) |
Sorry, something went wrong.
|
Silently dropping the attribute-with-empty-list during save() will mean that it is missing when loaded again. |
Sorry, something went wrong.
|
Is this PR dead? It's certainly silent. |
Sorry, something went wrong.
|
@dhermes how would you envision dealing with an entity where the property value (in Python) is an empty list? We can't marshall that to the protobuf "normally", becase an empty list_value field doesn't satisfy the criterion that the property have a value set. I posted in #403 (comment) a reference to an alternate implementation, which creates an entity_value for the property with an empty list, but didn't create a PR from that branch. |
Sorry, something went wrong.
|
@tseaver Sorry I wasn't trying to claim #403 is dead / fixed, just this PR. I looked at the alternate implementation, but am not a big fan of things like '_gcloud:empty_list'. Has anyone been able to send an empty list using just proto objects? |
Sorry, something went wrong.
|
@tseaver I commented on the original issue as well, but I think the "correct" / simplest fix would be to change the code you focus on in your solution: elif attr == 'list_value':
if len(val) == 0:
continue
l_pb = value_pb.list_value
for item in val: |
Sorry, something went wrong.
|
We would need to document that some values (empty lists, None, etc.) cannot be saved to the back-end. |
Sorry, something went wrong.
|
I agree. I wonder if the datastore folks also document that it's not possible to do? |
Sorry, something went wrong.
|
If we document this behavior,
I guess we don't need to have '_gcloud:empty_list' ? |
Sorry, something went wrong.
|
Just a tiny code change which never adds the property and documents that empty lists aren't stored. |
Sorry, something went wrong.
|
@dhermes elif attr == 'list_value':
if len(val) == 0:
return
l_pb = value_pb.list_value
for item in val:
i_pb = l_pb.add()
_set_protobuf_value(i_pb, item)
else: # scalar, just assign
setattr(value_pb, attr, val)
it will still raise the following exception Traceback (most recent call last):
File "test.py", line 26, in <module>
entity.save()
File "/Users/david/repos/gcloud-python/gcloud/datastore/entity.py", line 245, in save
exclude_from_indexes=self.exclude_from_indexes())
File "/Users/david/repos/gcloud-python/gcloud/datastore/connection.py", line 482, in save_entity
result = self.commit(dataset_id, mutation)
File "/Users/david/repos/gcloud-python/gcloud/datastore/connection.py", line 382, in commit
datastore_pb.CommitResponse)
File "/Users/david/repos/gcloud-python/gcloud/datastore/connection.py", line 95, in _rpc
data=request_pb.SerializeToString())
File "/Users/david/anaconda/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 813, in SerializeToString
self.DESCRIPTOR.full_name, ','.join(self.FindInitializationErrors())))
google.protobuf.message.EncodeError: Message api.services.datastore.CommitRequest is missing required fields: mutation.insert_auto_id[0].property[0].value
so I check the list value in connection.py https://github.com/GoogleCloudPlatform/gcloud-python/pull/404/files |
Sorry, something went wrong.
|
Right, adding the property should never happen. My bad for indicating otherwise. |
Sorry, something went wrong.
|
The test case is added :) |
Sorry, something went wrong.
…list or none value
|
You should also add a test for save_entities. This makes me think the change should happen in the to_protobuf method. Also, for such a small change, we have quite a few do-nothing commits in this PR. Do you mind squashing into a single commit and re-writing your history? I'm happy to help if you like. |
Sorry, something went wrong.
|
@dhermes elif attr == 'list_value':
l_pb = value_pb.list_value
if len(val) == 0:
x = l_pb.add()
l_pb.remove(x)or if len(val) == 0:
l_pb.extend([]) |
Sorry, something went wrong.
|
@lucemia That's great! I'm a little confused as to why it works. Wouldn't we be better clearing value_pb from its parent? Also, provided we can agree on this proposed workaround, want to just close this PR and open a new one (so as not to have to worry about re-writing history)? Make sure to polish up and run tox locally before committing / submitting the PR. |
Sorry, something went wrong.
Thx @dhermes I am now very sure which parent you mention, would you mind to provide more detail therefore I can add it to my new PR? I will open a new PR with new workaround and close current one. |
Sorry, something went wrong.
|
Well the issue is that the only thing _set_protobuf_value receives is an instance of datastore_v1_pb2.Value (via value_pb). However, we actually want to remove a datastore_v1_pb2.Property (which owns value_pb) from a datastore_v1_pb2.Entity object such as an auto insert in a mutation: >>> insert_auto = mutation.insert_auto_id.add()
>>> prop = insert_auto.property.add()
>>> # Do some stuff
>>> # Oops didn't mean to add the property
>>> insert_auto.property.remove(prop)Unfortunately, it appears that there is no public API for retrieving a parent, so our best way forward may be a re-thinking of which data gets passed to _set_protobuf_value. I took a stab with the non-public API provided by protobuf and it is not very pleasant. (Though it's also irrelevant since we can't rely on unsupported behavior.) No need to close this one until we get the right solution actually sorted out. |
Sorry, something went wrong.
|
I created two branch and did some test with the following code. conn = datastore.connection.Connection(credentials)
result = conn.save_entity(DATASET_ID, key_pb, {
'foo': u'Foo',
'bar': None,
'bar1': []
})While saving a empty list to cloud datastore: method1 fix the _set_protobuf_valuelucemia@5e4d5b9 method2 fix the save_entitylucemia@2c3defa I also test the same condition on ndb (with remote_api_shell), the ndb will drop repeated property with empty list (like method2). So I think the original approach (fix the save_entity) works better. If it sounds make sense, I will rebase the branch property-with-empty-list. |
Sorry, something went wrong.
* chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * revert Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* feat: added font_family to document.proto feat: added ImageQualityScores message to document.proto feat: added PropertyMetadata and EntityTypeMetadata to document_schema.proto PiperOrigin-RevId: 486975621 Source-Link: googleapis/googleapis@398c9f9 Source-Link: googleapis/googleapis-gen@7cd1f5f Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiN2NkMWY1ZjRlNDM1Nzc3Y2I4MjRhZjI2OGRjOGQzNzEzNDYxM2U2YSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update constraints-3.7.txt Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Holt Skinner <13262395+holtskinner@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: https://togithub.com/googleapis/synthtool/commit/25083af347468dd5f90f69627420f7d452b6c50e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e6cbd61f1838d9ff6a31436dfc13717f372a7482a82fc1863ca954ec47bff8c8
Source-Link: googleapis/synthtool@d2871d9 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:b2dc5f80edcf5d4486c39068c9fa11f7f851d9568eea4dcba130f994ea9b5e97 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* feat: support per-entity search and autocomplete feat: add model get API feat: support new filter syntax for recommendation feat: expose A/B experiment info in search response docs: keep the API doc up-to-date with recent changes PiperOrigin-RevId: 522675546 Source-Link: googleapis/googleapis@81b0808 Source-Link: googleapis/googleapis-gen@e950439 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZTk1MDQzOWNkMGQ0ODZhYjhkMmIzMjY3MmI1NGMwYTZiNjQ1MTQyMCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add model service feat: support per-entity search and autocomplete feat: support new filter syntax for recommendation feat: expose A/B experiment info in search response docs: keep the API doc up-to-date with recent changes PiperOrigin-RevId: 522675951 Source-Link: googleapis/googleapis@f149e91 Source-Link: googleapis/googleapis-gen@c4538a8 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzQ1MzhhODg4ZDJlYzEzY2U3MTljMWExYWE5ZGM3ZGU1YjE3Mzc1YSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add merchant center link service feat: support per-entity search and autocomplete feat: expose facets and product counts in autocomplete feat: add model get API feat: allow cascaded deletion on primary product feat: support new filter syntax for recommendation feat: expose A/B experiment info in search response docs: keep the API doc up-to-date with recent changes PiperOrigin-RevId: 523140513 Source-Link: googleapis/googleapis@4a9f933 Source-Link: googleapis/googleapis-gen@ffb754a Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZmZiNzU0YTdkMmNkNzkzODBmYjVjY2ZkMzNhMWMwOTA4ZGJiNjJhNSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* docs(samples): adds training phrases sample * docs(samples): fixed lint * docs(samples): ignored auto-foratting Co-authored-by: Anthonios Partheniou <partheniou@google.com>
🤖 I have created a release \*beep\* \*boop\* --- ### [2.9.1](https://www.github.com/googleapis/python-dialogflow/compare/v2.9.0...v2.9.1) (2021-10-04) ### Documentation * **samples:** adds training phrases sample ([#404](https://www.github.com/googleapis/python-dialogflow/issues/404)) ([9d98f9b](https://www.github.com/googleapis/python-dialogflow/commit/9d98f9b47208cbbdee13f678000c2970387c716e)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
* chore: update to gapic-generator-python 1.5.0 feat: add support for `google.cloud.<api>.__version__` PiperOrigin-RevId: 484665853 Source-Link: googleapis/googleapis@8eb249a Source-Link: googleapis/googleapis-gen@c8aa327 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzhhYTMyN2I1ZjQ3ODg2NWZjM2ZkOTFlM2MyNzY4ZTU0ZTI2YWQ0NCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * update version in gapic_version.py * add .release-please-manifest.json with correct version * add owlbot.py to exclude generated gapic_version.py * set manifest to true in .github/release-please.yml * add release-please-config.json * chore: Update to gapic-generator-python 1.6.0 feat(python): Add typing to proto.Message based class attributes feat(python): Snippetgen handling of repeated enum field PiperOrigin-RevId: 487326846 Source-Link: googleapis/googleapis@da380c7 Source-Link: googleapis/googleapis-gen@61ef576 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjFlZjU3NjJlZTY3MzFhMGNiYmZlYTIyZmQwZWVjZWU1MWFiMWM4ZSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: new APIs added to reflect updates to the filestore service - Add ENTERPRISE Tier - Add snapshot APIs: RevertInstance, ListSnapshots, CreateSnapshot, DeleteSnapshot, UpdateSnapshot - Add multi-share APIs: ListShares, GetShare, CreateShare, DeleteShare, UpdateShare - Add ConnectMode to NetworkConfig (for Private Service Access support) - New status codes (SUSPENDED/SUSPENDING, REVERTING/RESUMING) - Add SuspensionReason (for KMS related suspension) - Add new fields to Instance information: max_capacity_gb, capacity_step_size_gb, max_share_count, capacity_gb, multi_share_enabled PiperOrigin-RevId: 487492758 Source-Link: googleapis/googleapis@5be5981 Source-Link: googleapis/googleapis-gen@ab0e217 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWIwZTIxN2Y1NjBjYzJjMWFmYzExNDQxYzJlYWI2YjY5NTBlZmQyYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * update path to snippet metadata json * chore: Update gapic-generator-python to v1.6.1 PiperOrigin-RevId: 488036204 Source-Link: googleapis/googleapis@08f275f Source-Link: googleapis/googleapis-gen@555c094 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTU1YzA5NDVlNjA2NDllMzg3MzlhZTY0YmM0NTcxOWNkZjcyMTc4ZiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: added Location API methods docs: updated comments PiperOrigin-RevId: 489094434 Source-Link: googleapis/googleapis@71673d8 Source-Link: googleapis/googleapis-gen@1017723 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTAxNzcyMzJkNDIwMDA2NjEwYWU5ZmI3NDBkOWYxM2VmZTk3NWNiOSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* docs: Fix formatting of request arg in docstring chore: Update gapic-generator-python to v1.9.1 PiperOrigin-RevId: 518604533 Source-Link: googleapis/googleapis@8a085ae Source-Link: googleapis/googleapis-gen@b2ab4b0 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjJhYjRiMGEwYWUyOTA3ZTgxMmMyMDkxOThhNzRlMDg5OGFmY2IwNCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: https://togithub.com/googleapis/synthtool/commit/eaef28efd179e6eeb9f4e9bf697530d074a6f3b9 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f8ca7655fa8a449cadcabcbce4054f593dcbae7aeeab34aa3fcc8b5cf7a93c9e
* chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * revert Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
…le (#404) Source-Link: googleapis/synthtool@a7ed11e Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:effb79ef0525b02611cada94df8d7a248758928579c03d522f26ed37f9867668 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@7e1f6da Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:a1a891041baa4ffbe1a809ac1b8b9b4a71887293c9101c88e8e255943c5aec2d Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: omair <omairn@google.com>
Source-Link: googleapis/synthtool@7197a00 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@dd05f9d Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:aea14a583128771ae8aefa364e1652f3c56070168ef31beb203534222d842b8b
- [ ] Regenerate this pull request now. fix: add 'dict' annotation type to 'request' Committer: @busunkim96 PiperOrigin-RevId: 398509016 Source-Link: googleapis/googleapis@b224dfa Source-Link: googleapis/googleapis-gen@63a1db7 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjNhMWRiN2EzOGQ3NGI5NjM5NTkyZjUyMWVkMWRhYWY3Mjk5YWQ5YSJ9
@gaborfeher pointed at that while we were trying to use a `keys_only` query for `Query.count()` we were failing to actually do so. This is @gaborfeher's proposed fix from PR #400, fleshed out with unit tests. Fixes #404.
- [ ] Regenerate this pull request now. fix: add 'dict' annotation type to 'request' Committer: @busunkim96 PiperOrigin-RevId: 398509016 Source-Link: googleapis/googleapis@b224dfa Source-Link: googleapis/googleapis-gen@63a1db7 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjNhMWRiN2EzOGQ3NGI5NjM5NTkyZjUyMWVkMWRhYWY3Mjk5YWQ5YSJ9
This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/1cb8ad84-81d7-41ae-80cc-f275d38cb08c/targets - [ ] To automatically regenerate this PR, check this box. (May take up to 24 hours.) PiperOrigin-RevId: 383459959 Source-Link: googleapis/googleapis@fe3cc5e PiperOrigin-RevId: 382318016 Source-Link: googleapis/googleapis@e9b2b61 PiperOrigin-RevId: 382142900 Source-Link: googleapis/googleapis@513440f PiperOrigin-RevId: 381898347 Source-Link: googleapis/googleapis@2bf1e2a PiperOrigin-RevId: 380641501 Source-Link: googleapis/googleapis@076f7e9 PiperOrigin-RevId: 373649163 Source-Link: googleapis/googleapis@7e1b14e PiperOrigin-RevId: 373400747 Source-Link: googleapis/googleapis@162641c PiperOrigin-RevId: 372197450 Source-Link: googleapis/googleapis@83a7e1c PiperOrigin-RevId: 371362703 Source-Link: googleapis/googleapis@5a04154 PiperOrigin-RevId: 370989216 Source-Link: googleapis/googleapis@4e82555 PiperOrigin-RevId: 370926454 Source-Link: googleapis/googleapis@382ed8d PiperOrigin-RevId: 370525906 Source-Link: googleapis/googleapis@60e129d This PR includes the following features/fixes: fix: add async client to %name_%version/init.py chore: add autogenerated snippets chore: remove auth, policy, and options from the reserved names list feat: support self-signed JWT flow for service accounts chore: enable GAPIC metadata generation chore: sort subpackages in %namespace/%name/init.py feat: add always_use_jwt_access fix: disable always_use_jwt_access feat: add subscription properties to streaming pull response feat: add method signature for Subscriber.Pull without the deprecated return_immediately field. fix(deps): require google-api-core >= 1.26.0 fix(deps): add packaging requirement
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* test: add tests for using json array Adds test to verify that using JSON with both an array and a dict work as expected. Fixes #404 * chore: remove GetSession checks in tests --------- Co-authored-by: Sanjeev Bhatt <bhatt4982@gmail.com>
Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.17.1 to 20.26.6. - [Release notes](https://github.com/pypa/virtualenv/releases) - [Changelog](https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst) - [Commits](pypa/virtualenv@20.17.1...20.26.6) --- updated-dependencies: - dependency-name: virtualenv dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
| Back | FazBrowse Home | New Git URL |
for issue #403
for test case. I am curious about whether to create a new test case test_save_entity_w_transaction_list_value and included #399.