Support async profiler feature by zhengziyi0117 · Pull Request #720 · apache/skywalking-java · GitHub

Integrate async profiler performance analysis function in Java

If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the CHANGES log.

Pls fix CI

…/apm/agent/core/conf/Config.java Co-authored-by: Jiajing LU <lujiajing1126@gmail.com>

@wu-sheng In the current design, the Java Agent has to wait after the first request in order to determine whether the OAP accepts the following data transfer. This is a kind of negotiation.

WDYT?

If you need to wait for the server, you should change AsyncProfilerTask service to dual streaming, and the response needs to be a kind of ack

500ms can't guarantee anything AFAIK.

If you need to wait for the server, you should change AsyncProfilerTask service to dual streaming, and the response needs to be a kind of ack

Sure. Let's use dual streaming @zhengziyi0117

Note, in this case, Command should not be the response anymore.

What does this comment mean?

Please move this the last one rather than the first.

LGTM if all the current comments are resolved

you have to check NULL!

You should check service not implemented error of gRPC, you could try that by connecting v < 10.1 OAP.

Once it is not implemented, the agent should shutdown this part automatically.

This comment is not accurate. You hard coded this value as true, so, the default value is TRUE, not disabled. Please update your default value in the codes.

This should move into config. Don't have to expose through agent.config, but we don't need to hard code this.

When onError, we should output logs.

Please use lazy initialization as much as possible. I am not sure how much effects would this getInstance takes, but as service list is loaded before agent instrumentation, we should load as less code as possible. Otherwise, the instrumentation could fail due to classes have been loaded.

Same here, please initialize later in the process.

…/apm/agent/core/asyncprofiler/AsyncProfilerDataSender.java Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

…/apm/agent/core/asyncprofiler/AsyncProfilerTaskExecutionService.java Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

LGTM

Please use Status.Code.UNIMPLEMENTED.equals. It is safer.

lujiajing1126 mentioned this pull request Sep 30, 2024

[OSPP] Async Profiler integration with Java Agent apache/skywalking#12673

Closed

7 tasks

feat: support async-profiler

5adec9b

zhengziyi0117 force-pushed the async-profiler branch from 4f73937 to 5adec9b Compare September 30, 2024 05:35

zhengziyi added 2 commits October 11, 2024 14:50

fix: reduce the default max_duration of async-profiler

5a88a8f

fix: Cooperate with proto modification

e5243b2

lujiajing1126 requested changes Oct 25, 2024

View reviewed changes

Comment thread apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking/apm/agent/core/conf/Config.java

lujiajing1126 requested changes Oct 25, 2024

View reviewed changes

Comment thread ...core/src/main/java/org/apache/skywalking/apm/agent/core/asyncprofiler/AsyncProfilerTask.java Outdated

lujiajing1126 requested changes Oct 25, 2024

View reviewed changes

Comment thread ...rc/main/java/org/apache/skywalking/apm/agent/core/asyncprofiler/AsyncProfilerDataSender.java Outdated

lujiajing1126 added the enhancement New feature or request label Oct 25, 2024

lujiajing1126 assigned zhengziyi0117 Oct 25, 2024

lujiajing1126 added this to the 9.4.0 milestone Oct 25, 2024

zhengziyi0117 and others added 7 commits October 25, 2024 15:12

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

c236129

…/apm/agent/core/conf/Config.java Co-authored-by: Jiajing LU <lujiajing1126@gmail.com>

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

7417103

…/apm/agent/core/conf/Config.java Co-authored-by: Jiajing LU <lujiajing1126@gmail.com>

fix: proto git sha

d327dd2

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

1a417ac

…/apm/agent/core/conf/Config.java Co-authored-by: Jiajing LU <lujiajing1126@gmail.com>

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

3b59da7

…/apm/agent/core/conf/Config.java Co-authored-by: Jiajing LU <lujiajing1126@gmail.com>

Merge remote-tracking branch 'origin/async-profiler' into async-profiler

fdbb15c

fix: comment

faa8732

lujiajing1126 self-requested a review October 28, 2024 02:12

zhengziyi added 2 commits October 28, 2024 10:25

fix: comment & data sender

fbc24ef

fix: data sender and server communication

dbb7bff

lujiajing1126 reviewed Oct 28, 2024

View reviewed changes

Merge branch 'main' into async-profiler

9aa2ec4

wu-sheng reviewed Oct 28, 2024

View reviewed changes

zhengziyi added 2 commits October 28, 2024 23:06

fix: data sender

2f1cdd8

fix: data sender

bcc5f30

lujiajing1126 requested changes Oct 30, 2024

View reviewed changes

fix: some issue

e66546a

wu-sheng reviewed Oct 30, 2024

View reviewed changes

Comment thread ...java/org/apache/skywalking/apm/network/trace/component/command/AsyncProfilerTaskCommand.java

zhengziyi added 4 commits October 30, 2024 11:56

fix: unit test

2f28e5f

fix: comment

17990e0

fix: unit test

d6dcbff

fix: comment

e8aef4c

wu-sheng reviewed Oct 30, 2024

View reviewed changes

Comment thread ...rc/main/java/org/apache/skywalking/apm/agent/core/asyncprofiler/AsyncProfilerDataSender.java Outdated

wu-sheng reviewed Oct 30, 2024

View reviewed changes

zhengziyi and others added 2 commits October 30, 2024 15:06

fix: lazy init && config

b3fe849

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

4a6d7f5

…/apm/agent/core/asyncprofiler/AsyncProfilerDataSender.java Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

wu-sheng reviewed Oct 30, 2024

View reviewed changes

Comment thread ...va/org/apache/skywalking/apm/agent/core/asyncprofiler/AsyncProfilerTaskExecutionService.java Outdated

wu-sheng reviewed Oct 30, 2024

View reviewed changes

Comment thread ...va/org/apache/skywalking/apm/agent/core/asyncprofiler/AsyncProfilerTaskExecutionService.java Outdated

zhengziyi and others added 6 commits October 30, 2024 15:13

fix: volatile

599fb98

Merge remote-tracking branch 'origin/async-profiler' into async-profiler

2ccbd23

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

0435607

…/apm/agent/core/asyncprofiler/AsyncProfilerTaskExecutionService.java Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

Update apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking…

efa9ce9

…/apm/agent/core/asyncprofiler/AsyncProfilerTaskExecutionService.java Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

fix: log grammar

bbd987f

Merge remote-tracking branch 'origin/async-profiler' into async-profiler

071d062

lujiajing1126 previously approved these changes Oct 30, 2024

View reviewed changes

wu-sheng reviewed Oct 30, 2024

View reviewed changes

fix: log && code format

d09acd4

zhengziyi0117 dismissed lujiajing1126’s stale review via d09acd4 October 30, 2024 08:25

wu-sheng approved these changes Oct 30, 2024

View reviewed changes

wu-sheng merged commit 2027a98 into apache:main Oct 30, 2024

+                       * Wait briefly to see if the server can receive the jfr. If the server cannot receive it, onError will be triggered.
+                       * Then we wait for a while (waiting for the server to send onError) and then decide whether to send the jfr file.
+                       */
+                      Thread.sleep(500);

+                          <dependency>
+                              <groupId>tools.profiler</groupId>
+                              <artifactId>async-profiler</artifactId>
+                              <version>${async-profiler.version}</version>
+                          </dependency>

               profile.dump_max_stack_depth=${SW_AGENT_PROFILE_DUMP_MAX_STACK_DEPTH:500}
               # Snapshot transport to backend buffer size
               profile.snapshot_transport_buffer_size=${SW_AGENT_PROFILE_SNAPSHOT_TRANSPORT_BUFFER_SIZE:4500}
+              # If true, Async Profiler will be enabled when user creates a new async profiler task, Otherwise it is disabled.

+              @DefaultImplementor
+              public class AsyncProfilerDataSender implements BootService, GRPCChannelListener {
+                  private static final ILog LOGGER = LogManager.getLogger(ProfileSnapshotSender.class);
+                  private static final int DATA_CHUNK_SIZE = 1024 * 1024;

                       <!-- core lib dependency -->
                       <bytebuddy.version>1.14.9</bytebuddy.version>
+                      <async-profiler.version>3.0</async-profiler.version>

+                  @Override
+                  public void shutdown() throws Throwable {
+                      scheduledFuture.cancel(true);

+                          Commands commands = asyncProfilerTaskBlockingStub.withDeadlineAfter(GRPC_UPSTREAM_TIMEOUT, TimeUnit.SECONDS)
+                                  .getAsyncProfilerTaskCommands(query);
+                          ServiceManager.INSTANCE.findService(CommandService.class).receiveCommand(commands);

		status.finished();
		ServiceManager.INSTANCE.findService(GRPCChannelManager.class).reportError(t);


		private static final ILog LOGGER = LogManager.getLogger(AsyncProfilerTaskChannelService.class);

		private static final AsyncProfiler ASYNC_PROFILER = AsyncProfiler.getInstance();

+                  private static final String SUCCESS_RESULT = "Profiling started\n";
+                  // profile executor thread pool, only running one thread
+                  private static final ScheduledExecutorService ASYNC_PROFILER_EXECUTOR = Executors.newSingleThreadScheduledExecutor(

+                                  return;
+                              }
+                              final StatusRuntimeException statusRuntimeException = (StatusRuntimeException) t;
+                              if (statusRuntimeException.getStatus().getCode() == Status.Code.UNIMPLEMENTED) {

Support async profiler feature by zhengziyi0117 · Pull Request #720 · apache/skywalking-java · GitHub

Uh oh!

Support async profiler feature - #720

Support async profiler feature#720wu-sheng merged 32 commits intoapache:mainapache/skywalking-java:mainfrom zhengziyi0117:async-profilerzhengziyi0117/skywalking-java:async-profilerCopy head branch name to clipboard

Conversation

zhengziyi0117 commented Sep 29, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lujiajing1126 commented Oct 25, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lujiajing1126 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lujiajing1126 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Support async profiler feature#720
wu-sheng merged 32 commits into
apache:mainfrom
zhengziyi0117:async-profiler