Environment details
- OS type and version:
- Python version: Python 3.11.4
- pip version: pip 23.1.2 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
- google-cloud-storage version: 2.14.0
Steps to reproduce
- Usage of Blob.upload_from_filename() seems to sometimes encounter a 503 response
- This seems to result in an immediate failure, despite indications in
|
_RETRYABLE_TYPES = ( |
|
api_exceptions.TooManyRequests, # 429 |
|
api_exceptions.InternalServerError, # 500 |
|
api_exceptions.BadGateway, # 502 |
|
api_exceptions.ServiceUnavailable, # 503 |
|
api_exceptions.GatewayTimeout, # 504 |
|
ConnectionError, |
|
requests.ConnectionError, |
|
requests_exceptions.ChunkedEncodingError, |
|
requests_exceptions.Timeout, |
|
) |
that api_exceptions.ServiceUnavailable is a 503 and should be retried.
Code example
https://github.com/google/osv.dev/blob/b705d0d0b7450ce94137624118a2b54a7f719147/docker/exporter/exporter.py#L57-L64
def upload_single(self, bucket, source_path, target_path):
"""Upload a single file to a bucket."""
logging.info('Uploading %s', target_path)
try:
blob = bucket.blob(target_path)
blob.upload_from_filename(source_path)
except Exception as e:
logging.error('Failed to export: %s', e)
Stack trace
Failed to export: 503 POST https://storage.googleapis.com/upload/storage/v1/b/osv-test-vulnerabilities/o?uploadType=multipart: {
"error": {
"code": 503,
"message": "We encountered an internal error. Please try again.",
"errors": [
{
"message": "We encountered an internal error. Please try again.",
"domain": "global",
"reason": "backendError"
}
]
}
}
: ('Request failed with status code', 503, 'Expected one of', <HTTPStatus.OK: 200>)
Perhaps 503 needs to be added to _ADDITIONAL_RETRYABLE_STATUS_CODES also?
Reactions are currently unavailable
Environment details
Steps to reproduce
python-storage/google/cloud/storage/retry.py
Lines 28 to 38 in ae9a53b
Code example
https://github.com/google/osv.dev/blob/b705d0d0b7450ce94137624118a2b54a7f719147/docker/exporter/exporter.py#L57-L64
Stack trace
Failed to export: 503 POST https://storage.googleapis.com/upload/storage/v1/b/osv-test-vulnerabilities/o?uploadType=multipart: { "error": { "code": 503, "message": "We encountered an internal error. Please try again.", "errors": [ { "message": "We encountered an internal error. Please try again.", "domain": "global", "reason": "backendError" } ] } } : ('Request failed with status code', 503, 'Expected one of', <HTTPStatus.OK: 200>)Perhaps 503 needs to be added to _ADDITIONAL_RETRYABLE_STATUS_CODES also?