| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
As I noted on #215, I don't believe this change is necessary. Current releases of protobuf do return the (potentially coerced) value from the checker. |
Sorry, something went wrong.
|
Ah you're right. Didn't realize that. However, after some digging... I'm on Ubuntu 13.10, with the python-protobuf package installed (via apt, not pip) and it returns None :( This makes me think the patch might be a good idea... (master)jj@jjg-gigabyte:~/projects/gcloud$ sudo apt-get install python-protobuf Reading package lists... Done Building dependency tree Reading state information... Done python-protobuf is already the newest version. The following packages were automatically installed and are no longer required: libgetopt-argvfile-perl libmodule-scandeps-perl libmodule-signature-perl libpar-dist-perl libpar-perl python-keyring python-launchpadlib python-lazr.restfulclient python-lazr.uri python-oauth python-problem-report python-secretstorage python-simplejson python-wadllib Use 'apt-get autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded. (master)jj@jjg-gigabyte:~/projects/gcloud$ dpkg -s python-protobuf Package: python-protobuf Status: install ok installed Priority: optional Section: python Installed-Size: 458 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Architecture: amd64 Source: protobuf Version: 2.4.1-3ubuntu2 Provides: python2.7-protobuf Depends: libc6 (>= 2.4), libgcc1 (>= 1:4.1.1), libprotobuf7, libstdc++6 (>= 4.1.1), python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2) Recommends: protobuf-compiler Description: Python bindings for protocol buffers Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data - similar to XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format. . Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats. . This package contains the Python bindings for the protocol buffers. You will need the protoc tool (in the protobuf-compiler package) to compile your definition to Python classes, and then the modules in this package will allow you to use those classes in your programs. . This package contains both the traditional Python-based implementation and the new C++-based one, and you can select at runtime between the two. Homepage: http://code.google.com/p/protobuf/ Original-Maintainer: Iustin Pop <iustin@debian.org> (master)jj@jjg-gigabyte:~/projects/gcloud$ python Python 2.7.5+ (default, Feb 27 2014, 19:37:08) [GCC 4.8.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from google import protobuf >>> from google.protobuf.internal.type_checkers import Int64ValueChecker >>> print Int64ValueChecker().CheckValue(123) None |
Sorry, something went wrong.
|
Ubuntu 13.10 is not an LTS release (12.04 and 14.04 are nearest). That is possibly the reason for out of date packages. UPDATE: 13.10 was EOL'd in July - https://wiki.ubuntu.com/Releases UPDATE (Again): The versions on the LTS releases are:
But at the end of the day, the version in PyPI is up to date, so we need not really worry (other than the version conflict with dateutil I previously mentioned). |
Sorry, something went wrong.
|
Centos / RHEL 6: python-protobuf-2.3.0 As I noted on #215, looks like we either need to pin our dependency ('protobuf >= 2.6.0') or else apply the patch to handle older protobuf versions. |
Sorry, something went wrong.
|
I'm inclined to go with pinning the dependency. Using the system Python site library is not the way to run an app on any platform. The question we should ask is: does 2.6 run on RHEL6 etc? With one caveat, we make it very clear what the problem is when an earlier library version is installed. Returning None unexpectedly is the most malicious outcome a piece of Python code can do. Suggest: check library version somewhere in the code and explicitly give an error. |
Sorry, something went wrong.
|
I'm +1 on pinning the dependency, and +1 on being compatible with the old CheckValue method... I have a hunch this will be a really subtle error, and since Google did some weird stuff with the protobuf package namespacing (google.protobuf), it's just extra headache IMO. For reference, here's what happens when I try to install the newer version of protobuf (and therefore what will happen if I do pip install -r requirements.txt based on a pinned version): (master)jj@jjg-gigabyte:~/projects/gcloud$ sudo pip install protobuf --upgrade
Downloading/unpacking protobuf from https://pypi.python.org/packages/source/p/protobuf/protobuf-2.6.0.tar.gz#md5=3a25a1efda4feba5427b96e9b34bc94a
Downloading protobuf-2.6.0.tar.gz (187kB): 187kB downloaded
Running setup.py egg_info for package protobuf
zip_safe flag not set; analyzing archive contents...
google.__init__: module references __path__
google.apputils.setup_command: module references __file__
google.apputils.run_script_module: module references __file__
google.apputils.__init__: module references __path__
Installed /tmp/pip_build_root/protobuf/google_apputils-0.4.0-py2.7.egg
Searching for python-gflags>=1.4
Reading http://pypi.python.org/simple/python-gflags/
Reading http://code.google.com/p/python-gflags
Best match: python-gflags 2.0
Downloading https://pypi.python.org/packages/source/p/python-gflags/python-gflags-2.0.tar.gz#md5=23c9a793959a54971b1f094b0c6d03b1
Processing python-gflags-2.0.tar.gz
Writing /tmp/easy_install-sBhHBF/python-gflags-2.0/setup.cfg
Running python-gflags-2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-sBhHBF/python-gflags-2.0/egg-dist-tmp-2iR1Tn
zip_safe flag not set; analyzing archive contents...
Installed /tmp/pip_build_root/protobuf/python_gflags-2.0-py2.7.egg
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip_build_root/protobuf/setup.py", line 199, in <module>
"Protocol Buffers are Google's data interchange format.",
File "/usr/lib/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 221, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 245, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 598, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (python-dateutil 2.2 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('python-dateutil>=1.4,<2'))
Complete output from command python setup.py egg_info:
zip_safe flag not set; analyzing archive contents...
google.__init__: module references __path__
google.apputils.setup_command: module references __file__
google.apputils.run_script_module: module references __file__
google.apputils.__init__: module references __path__
Installed /tmp/pip_build_root/protobuf/google_apputils-0.4.0-py2.7.egg
Searching for python-gflags>=1.4
Reading http://pypi.python.org/simple/python-gflags/
Reading http://code.google.com/p/python-gflags
Best match: python-gflags 2.0
Downloading https://pypi.python.org/packages/source/p/python-gflags/python-gflags-2.0.tar.gz#md5=23c9a793959a54971b1f094b0c6d03b1
Processing python-gflags-2.0.tar.gz
Writing /tmp/easy_install-sBhHBF/python-gflags-2.0/setup.cfg
Running python-gflags-2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-sBhHBF/python-gflags-2.0/egg-dist-tmp-2iR1Tn
zip_safe flag not set; analyzing archive contents...
Installed /tmp/pip_build_root/protobuf/python_gflags-2.0-py2.7.egg
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip_build_root/protobuf/setup.py", line 199, in <module>
"Protocol Buffers are Google's data interchange format.",
File "/usr/lib/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 221, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 245, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 598, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (python-dateutil 2.2 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('python-dateutil>=1.4,<2'))
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/protobuf
Storing complete log in /home/jj/.pip/pip.log
I had to do.... $ sudo pip uninstall python-dateutil $ sudo pip install "python-dateutil < 2" $ sudo pip install protobuf to get things to work.. this is far from a good experience just to get the right version of protobuf on a machine :( |
Sorry, something went wrong.
|
@jgeewax This dependency is a problem with protobuf and IMO should be raised with the authors. I brought this up in #91 and also pointed out a thread on their forum. Also, if we are pinning the dependency, then why worry about backwards compatibility? |
Sorry, something went wrong.
|
@dhermes we worry, because setup.py is not the only way to install things. |
Sorry, something went wrong.
|
This is my current feeling(s) on the matter. @aliafshar How does one control the other ways to install things in a new release? (i.e. some of the broken things are already out there and not easy to pull back in / revise.) |
Sorry, something went wrong.
|
I've updated setup.py to pin protobuf at 2.5.0 per our discussion on #215. I think this is the best "solution" to the problem given where we are today. |
Sorry, something went wrong.
|
@aliafshar what other ways do we care about? .deb and .rpm packagers will use the 'install_requires' from setup.py to compute dependencies on the relevant packaged libraries. If we are worrying about developers who have older versions of protobuf installed in their system site-pacakges (e.g., via .deb or .rpm), then ISTM we need to teach them about working in a virtualenv, isolating any cruft in /usr/lib. |
Sorry, something went wrong.
|
@jgeewax Pinning protobuf at 2.5.0 can't be the right thing: even if we merge your patch, the code still works fine with 2.6.0. |
Sorry, something went wrong.
|
So how about protobuf >= 2.5.0 ? That way, older Ubuntu versions don't require a newer version (which gives lots of errors while installing) and newer versions are happy. Sound good? |
Sorry, something went wrong.
|
protobuf==2.6.0 (i.e. latest) will not install in its current state, which makes this library uninstallable. For devs who have no idea how to hack around the protobuf install, this is a problem. |
Sorry, something went wrong.
|
@dhermes I can't replicate that in a clean virtualenv (tox wouldn't be passing, either): $ /opt/Python-2.7.6/bin/virtualenv /tmp/protobuf
New python executable in /tmp/protobuf/bin/python
Installing setuptools, pip...done.
$ /tmp/protobuf/bin/pip install protobuf
Downloading/unpacking protobuf
Downloading protobuf-2.6.0.tar.gz (187kB): 187kB downloaded
Running setup.py (path:/tmp/protobuf/build/protobuf/setup.py) egg_info for package protobuf
zip_safe flag not set; analyzing archive contents...
google.__init__: module references __path__
google.apputils.__init__: module references __path__
google.apputils.run_script_module: module references __file__
google.apputils.setup_command: module references __file__
Installed /tmp/protobuf/build/protobuf/google_apputils-0.4.0-py2.7.egg
Searching for pytz>=2010
Reading https://pypi.python.org/simple/pytz/
Best match: pytz 2014.7
Downloading https://pypi.python.org/packages/2.7/p/pytz/pytz-2014.7-py2.7.egg#md5=316ea05e4fd43ae88b2b559cacacf81b
Processing pytz-2014.7-py2.7.egg
Moving pytz-2014.7-py2.7.egg to /tmp/protobuf/build/protobuf
Installed /tmp/protobuf/build/protobuf/pytz-2014.7-py2.7.egg
Searching for python-gflags>=1.4
Reading https://pypi.python.org/simple/python-gflags/
Reading http://code.google.com/p/python-gflags
Best match: python-gflags 2.0
Downloading https://pypi.python.org/packages/source/p/python-gflags/python-gflags-2.0.tar.gz#md5=23c9a793959a54971b1f094b0c6d03b1
Processing python-gflags-2.0.tar.gz
Writing /tmp/easy_install-MbrsQ0/python-gflags-2.0/setup.cfg
Running python-gflags-2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-MbrsQ0/python-gflags-2.0/egg-dist-tmp-mY68Te
zip_safe flag not set; analyzing archive contents...
Installed /tmp/protobuf/build/protobuf/python_gflags-2.0-py2.7.egg
Searching for python-dateutil>=1.4,<2
Reading https://pypi.python.org/simple/python-dateutil/
Reading http://labix.org/python-dateutil
Best match: python-dateutil 1.5
Downloading https://pypi.python.org/packages/source/p/python-dateutil/python-dateutil-1.5.tar.gz#md5=0dcb1de5e5cad69490a3b6ab63f0cfa5
Processing python-dateutil-1.5.tar.gz
Writing /tmp/easy_install-Uu_HAL/python-dateutil-1.5/setup.cfg
Running python-dateutil-1.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Uu_HAL/python-dateutil-1.5/egg-dist-tmp-fCQCzQ
Installed /tmp/protobuf/build/protobuf/python_dateutil-1.5-py2.7.egg
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/protobuf/lib/python2.7/site-packages (from protobuf)
Installing collected packages: protobuf
Running setup.py install for protobuf
Skipping installation of /tmp/protobuf/lib/python2.7/site-packages/google/__init__.py (namespace package)
Installing /tmp/protobuf/lib/python2.7/site-packages/protobuf-2.6.0-py2.7-nspkg.pth
Successfully installed protobuf
Cleaning up...
$ /tmp/protobuf/bin/python
Python 2.7.6 (default, Nov 15 2013, 11:35:43)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import google.protobuf
>>>
|
Sorry, something went wrong.
|
I copied and pasted the output of installing on Ubuntu 13.10 (no virtual environment which unfortunately is how many of our users will install this...). I think the best bet is to do protobuf >= 2.5.0... |
Sorry, something went wrong.
|
So the versions in the virtualenv output from @tseaver are: protobuf-2.6.0.tar.gz google_apputils-0.4.0-py2.7.egg pytz-2014.7-py2.7.egg python_gflags-2.0-py2.7.egg python_dateutil-1.5-py2.7.egg So what is happening is that setuptools respects python-dateutil>=1.4,<2 but aptitude / apt-get installs Python dependencies in a different way. It seems the issue happens in the python-google-apputils package: That is the only hard dependency of the protobuf package: UPDATE: I completely wiped all my installed dependencies on Ubuntu 14.04 and apt-get install python-protobuf worked just fine. I think output from an already sunset version (13.10) is not so relevant to us. However, this didn't actually install the dateutil dependency so I'm now feeling like |
Sorry, something went wrong.
See discussion on Issue googleapis#215 for details.
|
Making our code work with protobuf 2.5 and 2.6 isn't a bad thing IMO. I've updated the path with two things:
Problem solved? |
Sorry, something went wrong.
|
Yes, it seems in protocolbuffers/protobuf#36 that the install issue will "go away" very shortly so IMO this is enough. LGTM (pending my tiny "typecast" comment). |
Sorry, something went wrong.
* chore(deps): update all dependencies to v2.56.0 * revert Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
) Source-Link: googleapis/synthtool@c1dd87e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:2d13c2172a5d6129c861edaa48b60ead15aeaf58aa75e02d870c4cbdfa63aaba Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google-cloud-storage](https://togithub.com/googleapis/python-storage) | `==1.39.0` -> `==1.40.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>googleapis/python-storage</summary> ### [`v1.40.0`](https://togithub.com/googleapis/python-storage/blob/master/CHANGELOG.md#​1400-httpswwwgithubcomgoogleapispython-storagecomparev1390v1400-2021-06-30) [Compare Source](https://togithub.com/googleapis/python-storage/compare/v1.39.0...v1.40.0) ##### Features - add preconditions and retry configuration to blob.create_resumable_upload_session ([#​484](https://www.github.com/googleapis/python-storage/issues/484)) ([0ae35ee](https://www.github.com/googleapis/python-storage/commit/0ae35eef0fe82fe60bc095c4b183102bb1dabeeb)) - add public access prevention to bucket IAM configuration ([#​304](https://www.github.com/googleapis/python-storage/issues/304)) ([e3e57a9](https://www.github.com/googleapis/python-storage/commit/e3e57a9c779d6b87852063787f19e27c76b1bb14)) ##### Bug Fixes - replace default retry for upload operations ([#​480](https://www.github.com/googleapis/python-storage/issues/480)) ([c027ccf](https://www.github.com/googleapis/python-storage/commit/c027ccf4279fb05e041754294f10744b7d81beea)) </details> --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-dataproc).
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@7ff4aad Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:462782b0b492346b2d9099aaff52206dd30bc8e031ea97082e6facecc2373244
Source-Link: googleapis/synthtool@c5026b3 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:0e18b9475fbeb12d9ad4302283171edebb6baf2dfca1bd215ee3b34ed79d95d7 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* feat: Add support for python 3.11 chore: Update gapic-generator-python to v1.8.0 PiperOrigin-RevId: 500768693 Source-Link: googleapis/googleapis@190b612 Source-Link: googleapis/googleapis-gen@7bf29a4 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiN2JmMjlhNDE0YjllY2FjMzE3MGYwYjY1YmRjMmE5NTcwNWMwZWYxYSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@0941ef3 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:2f90537dd7df70f6b663cd654b1fa5dee483cf6a4edcfd46072b2775be8a23ec Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@f15cc72 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bc5eed3804aec2f05fad42aacf973821d9500c174015341f721a984a0825b6fd
Source-Link: googleapis/synthtool@06e8279 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:b3500c053313dc34e07b1632ba9e4e589f4f77036a7cf39e1fe8906811ae0fce
Source-Link: googleapis/synthtool@dd05f9d Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:aea14a583128771ae8aefa364e1652f3c56070168ef31beb203534222d842b8b
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* chore: add aap-dpes to codeowners Removing cicd and adding aap-dpes. * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Source-Link: https://togithub.com/googleapis/synthtool/commit/7197a001ffb6d8ce7b0b9b11c280f0c536c1033a Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320
* fix(deps): allow protobuf 3.19.5 * explicitly exclude protobuf 4.21.0
fix(deps): require proto-plus >= 1.22.0
Source-Link: googleapis/synthtool@dd05f9d Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:aea14a583128771ae8aefa364e1652f3c56070168ef31beb203534222d842b8b
Source-Link: https://togithub.com/googleapis/synthtool/commit/d0f51a0c2a9a6bcca86911eabea9e484baadf64b Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:240b5bcc2bafd450912d2da2be15e62bc6de2cf839823ae4bf94d4f392b451dc
* [TST] Avoid listing datasets and tables in system tests Most tests only use a single dataset, so it doesn't make sense to list datasets in the clean-up method. Also, listing datasets is eventually-consistent, so avoiding listing when it isn't necessary makes the tests less flaky. Removes unused _Dataset.tables() and _Dataset.datasets() methods. * Remove unused randint import. * Update changelog.
🤖 I have created a release \*beep\* \*boop\* --- ## [1.18.0](https://www.github.com/googleapis/proto-plus-python/compare/v1.17.0...v1.18.0) (2021-03-16) ### Features * add copy_from method for field assignment ([#215](https://www.github.com/googleapis/proto-plus-python/issues/215)) ([11c3e58](https://www.github.com/googleapis/proto-plus-python/commit/11c3e58a9ba59f0d7d808a26597dab735ca982ba)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
) Source-Link: googleapis/synthtool@0c7b033 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:08e34975760f002746b1d8c86fdc90660be45945ee6d9db914d1508acdf9a547 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
| Back | FazBrowse Home | New Git URL |
No description provided.