I'm seeing frequent 503 and 504 responses using the blob.upload_from_filename() method. An example trace looks like this:
503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: {
"error": {
"code": 503,
"message": "Backend Error",
"errors": [
{
"message": "Backend Error",
"domain": "global",
"reason": "backendError"
}
]
}
}
: (u'Request failed with status code', 503, u'Expected one of', 200)' - Calling rollback() ...
Traceback (most recent call last):
File "/my/app/module.py", line 112, in upload
blob.upload_from_filename(file_path)
File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2471, in upload_from_filename
checksum=checksum,
File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2359, in upload_from_file
_raise_from_invalid_response(exc)
File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 3886, in _raise_from_invalid_response
raise exceptions.from_http_status(response.status_code, message, response=response)
ServiceUnavailable: 503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: {
"error": {
"code": 503,
"message": "Backend Error",
"errors": [
{
"message": "Backend Error",
"domain": "global",
"reason": "backendError"
}
]
}
}
: (u'Request failed with status code', 503, u'Expected one of', 200)
It looks like this method calls blob.upload_from_file() with num_retries=None, so there is no retry strategy applied. I also see that the num_retries argument is documented as deprecated.
Does this library provide an alternative way to retry blob.upload_from_filename() or blob.upload_from_file()? I can wrap these calls in retry logic of my own, but I see that retries are enabled by default for other functions.
I'm using google-cloud-storage==1.34.0. I've already pulled in google-resumable-media==1.20.0 so I should be getting the recent resumable media retry fixes that came with #342.
Thanks for any tips on retrying uploads.
I'm seeing frequent 503 and 504 responses using the blob.upload_from_filename() method. An example trace looks like this:
503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: { "error": { "code": 503, "message": "Backend Error", "errors": [ { "message": "Backend Error", "domain": "global", "reason": "backendError" } ] } } : (u'Request failed with status code', 503, u'Expected one of', 200)' - Calling rollback() ... Traceback (most recent call last): File "/my/app/module.py", line 112, in upload blob.upload_from_filename(file_path) File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2471, in upload_from_filename checksum=checksum, File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2359, in upload_from_file _raise_from_invalid_response(exc) File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 3886, in _raise_from_invalid_response raise exceptions.from_http_status(response.status_code, message, response=response) ServiceUnavailable: 503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: { "error": { "code": 503, "message": "Backend Error", "errors": [ { "message": "Backend Error", "domain": "global", "reason": "backendError" } ] } } : (u'Request failed with status code', 503, u'Expected one of', 200)It looks like this method calls blob.upload_from_file() with num_retries=None, so there is no retry strategy applied. I also see that the num_retries argument is documented as deprecated.
Does this library provide an alternative way to retry blob.upload_from_filename() or blob.upload_from_file()? I can wrap these calls in retry logic of my own, but I see that retries are enabled by default for other functions.
I'm using google-cloud-storage==1.34.0. I've already pulled in google-resumable-media==1.20.0 so I should be getting the recent resumable media retry fixes that came with #342.
Thanks for any tips on retrying uploads.