FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Retry and failure-handling logic for http-json by andreamlin · Pull Request #363 · googleapis/gax-java · GitHub

This repository was archived by the owner on Sep 26, 2023. It is now read-only.

Repository navigation

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
ce8351f
ApiMessage as an interface
andreamlin Aug 9, 2017
89dc19f
comments
andreamlin Aug 10, 2017
8480a90
basic constructors for ApiMethodDescriptor
andreamlin Aug 12, 2017
c8d3a88
adding gson to build.gradle and made ApiMethodDescriptor an AutoValue…
andreamlin Aug 14, 2017
c242d4c
comments
andreamlin Aug 22, 2017
5423b69
default HttpJsonCallOptions
andreamlin Aug 23, 2017
d319c88
using global instance of NetHttpTransport as httptransport object
andreamlin Aug 23, 2017
5b28bf9
uncompileable changes to ApiMethodDescriptor
andreamlin Aug 23, 2017
f2d6e34
uncompileable changes to ApiMethodDescriptor
andreamlin Aug 23, 2017
3ed61de
cleaner ApiMethodDescriptor
andreamlin Aug 23, 2017
aa219b6
using methoddescriptor in issuing unarycall
andreamlin Aug 23, 2017
63478c0
managedhttpjsonchannel actually marshalls objects
andreamlin Aug 23, 2017
f34e3fc
integrating ApiMessage into ManagedHttpJsonChannel
andreamlin Aug 23, 2017
e5051b5
made ApiMethodDescriptor AutoValue, more work in ManagedHttpJsonChannel
andreamlin Aug 23, 2017
a6ff124
AbstractPage will turn null nextPageTokens into empty strings
andreamlin Aug 24, 2017
181db07
comments
andreamlin Aug 24, 2017
25a4d41
writeRequestBody writes only the body
andreamlin Aug 24, 2017
54ccd16
apimethoddescriptor takes sets of param names
andreamlin Aug 24, 2017
95bb1d3
formatting and comments on HttpMethod
andreamlin Aug 24, 2017
c181a75
requests that have non-empty bodies now work, like addresses.insert
andreamlin Aug 24, 2017
1e1032d
uncommented ApiResultRetryAlgorithm
andreamlin Aug 24, 2017
8d3c679
s/HttpRequestBuilder/HttpRequestFormatter/g
andreamlin Aug 25, 2017
e71074f
moving httpTransport and httprequestfactory to be private members of …
andreamlin Aug 25, 2017
0a92f38
adding shutdown methods to ManagedHttpJsonChannel
andreamlin Aug 25, 2017
10da2e5
putting request body in httprequest
andreamlin Aug 25, 2017
69f076d
merge with origin/branch
andreamlin Aug 25, 2017
e0d6c0e
finally found the right maven dependency path for GoogleNetHttpTransport
andreamlin Aug 25, 2017
c65b1ab
betaApi
andreamlin Aug 25, 2017
318abd5
another BetaApi
andreamlin Aug 25, 2017
0d0c427
creating separate class for HttpRequestRunnable
andreamlin Aug 25, 2017
e839d03
parameterize settableapifuture
andreamlin Aug 25, 2017
3c6ab2f
javadoc style comments
andreamlin Aug 25, 2017
2f52e05
grammar
andreamlin Aug 25, 2017
1dbaf02
removing GoogleNetHttpTransport b/c NetHttpTransport works fine
andreamlin Aug 25, 2017
d9282c1
using String for HttpMethod instead of something typed
andreamlin Aug 25, 2017
1d0e3da
HttpRequestRunnable is package-private
andreamlin Aug 26, 2017
aeb9d59
private HttpRequestRunnable constructor
andreamlin Aug 26, 2017
d0e4323
only one try/catch block in HttpRequestRunnable.run()
andreamlin Aug 26, 2017
ece328d
removing HttpMethod.java
andreamlin Aug 26, 2017
c68586f
remove ApiMethodDescriptor.toBuilder()
andreamlin Aug 26, 2017
929fd72
default values in ApiMethodDescriptor.toBuilder()
andreamlin Aug 26, 2017
6db8201
adding back setHttpTransport in ManagedHttpJsonChannel
andreamlin Aug 26, 2017
f372174
retry codes in ImmutableSet
andreamlin Aug 30, 2017
4dbab5d
moving RecordingScheduler to common gax-core
andreamlin Aug 30, 2017
9415291
prettying
andreamlin Aug 30, 2017
0ded1f5
fixed some onFailure() logic in HttpJsonExceptionCallable
andreamlin Aug 30, 2017
6da4961
fixed RetryTest
andreamlin Aug 30, 2017
0a03509
cleaning up
andreamlin Aug 30, 2017
064a321
Merge branch 'gax-httpjson' of github.com:googleapis/gax-java into retry
andreamlin Aug 31, 2017
f18a013
removed finished TODO
andreamlin Aug 31, 2017
File filter

Filter by extension

Filter by extension .java  (5) All 1 file type selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
Prev Previous commit
Next Next commit
moving RecordingScheduler to common gax-core
  • Loading branch information
andreamlin committed Aug 30, 2017
commit 4dbab5d9ba7e57dbc6c0e19cf9238ca38191aad8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import com.google.api.core.ApiFuture;
import com.google.api.core.SettableApiFuture;
import com.google.api.gax.core.FakeApiClock;
import com.google.api.gax.core.RecordingScheduler;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ClientContext;
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import com.google.api.core.ApiFuture;
import com.google.api.core.ListenableFutureToApiFuture;
import com.google.api.gax.core.FakeApiClock;
import com.google.api.gax.core.RecordingScheduler;
import com.google.api.gax.grpc.testing.FakeMethodDescriptor;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.retrying.TimedAttemptSettings;
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutures;
import com.google.api.gax.core.FakeApiClock;
import com.google.api.gax.core.RecordingScheduler;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ApiException;
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,309 @@
/*
* Copyright 2017, Google Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Google Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package com.google.api.gax.httpjson;

import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutures;
import com.google.api.gax.core.FakeApiClock;
import com.google.api.gax.core.RecordingScheduler;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ApiException;
import com.google.api.gax.rpc.ClientContext;
import com.google.api.gax.rpc.SimpleCallSettings;
import com.google.api.gax.rpc.StatusCode;
import com.google.api.gax.rpc.UnaryCallable;
import com.google.common.collect.ImmutableSet;
import com.google.common.truth.Truth;
import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.UncheckedExecutionException;
import io.grpc.Status;
import java.util.Set;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
import org.mockito.Mockito;
import org.threeten.bp.Duration;

@RunWith(JUnit4.class)
public class RetryingTest {
@SuppressWarnings("unchecked")
private UnaryCallable<Integer, Integer> callInt = Mockito.mock(UnaryCallable.class);

private RecordingScheduler executor;
private FakeApiClock fakeClock;
private ClientContext clientContext;

private static final RetrySettings FAST_RETRY_SETTINGS =
RetrySettings.newBuilder()
.setInitialRetryDelay(Duration.ofMillis(2L))
.setRetryDelayMultiplier(1)
.setMaxRetryDelay(Duration.ofMillis(2L))
.setInitialRpcTimeout(Duration.ofMillis(2L))
.setRpcTimeoutMultiplier(1)
.setMaxRpcTimeout(Duration.ofMillis(2L))
.setTotalTimeout(Duration.ofMillis(10L))
.build();

@Before
public void resetClock() {
fakeClock = new FakeApiClock(System.nanoTime());
executor = RecordingScheduler.create(fakeClock);
clientContext = ClientContext.newBuilder().setExecutor(executor).setClock(fakeClock).build();
}

@After
public void teardown() {
executor.shutdownNow();
}

@Rule public ExpectedException thrown = ExpectedException.none();

static <V> ApiFuture<V> immediateFailedFuture(Throwable t) {
return ApiFutures.<V>immediateFailedFuture(t);
}

@Test
public void retry() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Throwable throwable = Status.UNAVAILABLE.asException();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));

SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
Truth.assertThat(callable.call(1)).isEqualTo(2);
}

@Test(expected = ApiException.class)
public void retryTotalTimeoutExceeded() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Throwable throwable = Status.UNAVAILABLE.asException();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));

RetrySettings retrySettings =
FAST_RETRY_SETTINGS
.toBuilder()
.setInitialRetryDelay(Duration.ofMillis(Integer.MAX_VALUE))
.setMaxRetryDelay(Duration.ofMillis(Integer.MAX_VALUE))
.build();
SimpleCallSettings<Integer, Integer> callSettings = createSettings(retryable, retrySettings);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
}

@Test(expected = ApiException.class)
public void retryMaxAttemptsExeeded() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Throwable throwable = Status.UNAVAILABLE.asException();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));

RetrySettings retrySettings = FAST_RETRY_SETTINGS.toBuilder().setMaxAttempts(2).build();
SimpleCallSettings<Integer, Integer> callSettings = createSettings(retryable, retrySettings);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
}

@Test
public void retryWithinMaxAttempts() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Throwable throwable = Status.UNAVAILABLE.asException();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));

RetrySettings retrySettings = FAST_RETRY_SETTINGS.toBuilder().setMaxAttempts(3).build();
SimpleCallSettings<Integer, Integer> callSettings = createSettings(retryable, retrySettings);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
Truth.assertThat(callable.call(1)).isEqualTo(2);
}

@Test
public void retryOnStatusUnknown() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNKNOWN));
Throwable throwable = Status.UNKNOWN.asException();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));
SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
Truth.assertThat(callable.call(1)).isEqualTo(2);
}

@Test
public void retryOnUnexpectedException() {
thrown.expect(ApiException.class);
thrown.expectMessage("foobar");
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNKNOWN));
Throwable throwable = new RuntimeException("foobar");
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(throwable));
SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
}

@Test
public void retryNoRecover() {
thrown.expect(ApiException.class);
thrown.expectMessage("foobar");
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(
RetryingTest.<Integer>immediateFailedFuture(
Status.FAILED_PRECONDITION.withDescription("foobar").asException()))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));
SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
}

@Test
public void retryKeepFailing() {
thrown.expect(UncheckedExecutionException.class);
thrown.expectMessage("foobar");
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(
RetryingTest.<Integer>immediateFailedFuture(
Status.UNAVAILABLE.withDescription("foobar").asException()));
SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
// Need to advance time inside the call.
ApiFuture<Integer> future = callable.futureCall(1);
Futures.getUnchecked(future);
}

@Test
public void noSleepOnRetryTimeout() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(
GrpcStatusCode.of(Status.Code.UNAVAILABLE),
GrpcStatusCode.of(Status.Code.DEADLINE_EXCEEDED));
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(
RetryingTest.<Integer>immediateFailedFuture(
Status.DEADLINE_EXCEEDED.withDescription("DEADLINE_EXCEEDED").asException()))
.thenReturn(ApiFutures.<Integer>immediateFuture(2));

SimpleCallSettings<Integer, Integer> callSettings =
createSettings(retryable, FAST_RETRY_SETTINGS);
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
callable.call(1);
Truth.assertThat(executor.getSleepDurations().size()).isEqualTo(1);
Truth.assertThat(executor.getSleepDurations().get(0))
.isEqualTo(ApiResultRetryAlgorithm.DEADLINE_SLEEP_DURATION);
}

@Test
public void testKnownStatusCode() {
ImmutableSet<StatusCode> retryable =
ImmutableSet.<StatusCode>of(GrpcStatusCode.of(Status.Code.UNAVAILABLE));
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(
RetryingTest.<Integer>immediateFailedFuture(
Status.FAILED_PRECONDITION.withDescription("known").asException()));
SimpleCallSettings<Integer, Integer> callSettings =
SimpleCallSettings.<Integer, Integer>newBuilder().setRetryableCodes(retryable).build();
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
try {
callable.call(1);
} catch (GrpcApiException exception) {
Truth.assertThat(exception.getStatusCode().getCode())
.isEqualTo(Status.Code.FAILED_PRECONDITION);
Truth.assertThat(exception.getMessage())
.isEqualTo("io.grpc.StatusException: FAILED_PRECONDITION: known");
}
}

@Test
public void testUnknownStatusCode() {
ImmutableSet<StatusCode> retryable = ImmutableSet.<StatusCode>of();
Mockito.when(callInt.futureCall((Integer) Mockito.any(), (ApiCallContext) Mockito.any()))
.thenReturn(RetryingTest.<Integer>immediateFailedFuture(new RuntimeException("unknown")));
SimpleCallSettings<Integer, Integer> callSettings =
SimpleCallSettings.<Integer, Integer>newBuilder().setRetryableCodes(retryable).build();
UnaryCallable<Integer, Integer> callable =
GrpcCallableFactory.create(callInt, callSettings, clientContext);
try {
callable.call(1);
} catch (GrpcApiException exception) {
Truth.assertThat(exception.getStatusCode().getCode()).isEqualTo(Status.Code.UNKNOWN);
Truth.assertThat(exception.getMessage()).isEqualTo("java.lang.RuntimeException: unknown");
}
}

public static SimpleCallSettings<Integer, Integer> createSettings(
Set<StatusCode> retryableCodes, RetrySettings retrySettings) {
return SimpleCallSettings.<Integer, Integer>newBuilder()
.setRetryableCodes(retryableCodes)
.setRetrySettings(retrySettings)
.build();
}
}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,12 @@
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package com.google.api.gax.grpc;
package com.google.api.gax.core;

import static org.mockito.Matchers.any;
import static org.mockito.Matchers.anyLong;
import static org.mockito.Mockito.when;

import com.google.api.gax.core.FakeApiClock;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ScheduledExecutorService;
Expand Down

Back | FazBrowse Home | New Git URL