FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Add X-Ray daemon by dominikschubert · Pull Request #14 · localstack/lambda-runtime-init · GitHub

/ lambda-runtime-init Public
forked from aws/aws-lambda-runtime-interface-emulator

Add X-Ray daemon - #14

Merged
dominikschubert merged 6 commits into
localstackfrom
feat_xray-integration
Mar 21, 2023
ConversationCommits (6)ChecksFiles changed
Merged

Add X-Ray daemon#14
dominikschubert merged 6 commits into
localstackfrom
feat_xray-integration

Conversation

dominikschubert commented Mar 17, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Adds the xray daemon as a dependency and starts the daemon asynchronously.

The daemon is stopped and recorded segments are flushed when init shuts down (see signal handler / shutdown funcs in sandbox).

It's not yet ideal, but it provides parity with the old xray implementation. For proper active & passive tracing with additional generated (sub-)segments we'll need to add a few more things

  • Generate & publish traces in init. There's actually already some things we can re-use here
  • Properly use provided credentials for service calls
  • Generate the root trace segment in the lambda service.

Other changes:

  • Added a basic LOCALSTACK_INIT_LOG_LEVEL internal variable for us to be able to configure the log level in the init binary without re-compiling. I don't think this will be the final setup, since I still want to rework the general log setup here, but it's kinda useful in the meantime.

dominikschubert requested a review from joe4dev March 17, 2023 08:47
dominikschubert self-assigned this Mar 17, 2023
dominikschubert marked this pull request as ready for review March 17, 2023 09:27
dominikschubert mentioned this pull request Mar 17, 2023
Merged
1 task
dominikschubert changed the title Start x-ray daemon when the request is sampled Add X-Ray daemon Mar 19, 2023
dominikschubert requested a review from dfangl March 20, 2023 18:39
joe4dev approved these changes Mar 20, 2023
View reviewed changes

joe4dev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM 👍

just minor comments but nothing blocking.

Comment thread cmd/localstack/awsutil.go Outdated
Comment thread cmd/localstack/main.go
Comment thread cmd/localstack/main.go
Comment thread cmd/localstack/xraydaemon.go Outdated
xrayConfig.NoVerifySSL = util.Bool(true) // obvious
xrayConfig.LocalMode = util.Bool(true) // skip EC2 metadata check
xrayConfig.Region = GetEnvOrDie("AWS_REGION")
xrayConfig.Logging.LogLevel = "dev" // TODO

joe4dev Mar 20, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

When do we adjust that to avoid log spamming?
The daemon itself should be stable, this might just be interesting for debugging.

joe4dev Mar 20, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It periodically logs Skipped telemetry data as no segments found or the number of traces sent.

joe4dev commented Mar 20, 2023

Copy link
Copy Markdown
Member

The PR description is not up-to-date anymore after the change we introduced for updating the X-Ray TraceId with every request.

joe4dev commented Mar 20, 2023

Copy link
Copy Markdown
Member

Could it be that the runtime starts before the X-Ray daemon due to async runDaemon(d) given the warning: Subsegment lambda discarded due to Lambda worker still initializing 🤔 (probably not blocking now but good to follow up later)

2023-03-20 23:02:07 2023-03-20T22:02:07Z [Debug] Batch size: 10
2023-03-20 23:02:07 2023-03-20T22:02:07Z [Info] Starting proxy http server on 127.0.0.1:2000
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=debug msg="Hot reloading disabled." func=main.RunHotReloadingListener file="/Users/joe/Projects/LocalStack/lambda-runtime-init/cmd/localstack/awsutil.go:150"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=debug msg="Runtime API Server listening on 127.0.0.1:9001" func="go.amzn.com/lambda/rapi.(*Server).Listen" file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapi/server.go:93"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=info msg="Configure environment for Init Caching." func="go.amzn.com/lambda/rapid.(*rapidContext).acceptStartRequestForInitCaching" file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapid/start.go:391"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=info msg="extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory" func=go.amzn.com/lambda/rapid.extensionsDisabledByLayer file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapid/start.go:363"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=debug msg="Received RUNNING" func="go.amzn.com/lambda/rapidcore.(*Server).Init" file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapidcore/server.go:533"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=warning msg="Cannot list external agents" func=go.amzn.com/lambda/agents.ListExternalAgentPaths file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/agents/agent.go:71" error="open /opt/extensions: no such file or directory"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=debug msg="Preregister runtime" func=go.amzn.com/lambda/rapid.doInit file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapid/start.go:189"
2023-03-20 23:02:07 time="2023-03-20T22:02:07Z" level=debug msg="Start runtime" func=go.amzn.com/lambda/rapid.doInit file="/Users/joe/Projects/LocalStack/lambda-runtime-init/lambda/rapid/start.go:222"
2023-03-20 23:02:10 [INFO]      2023-03-20T22:02:10.899Z                successfully patched module requests
2023-03-20 23:02:10 [INFO]      2023-03-20T22:02:10.925Z                successfully patched module botocore
2023-03-20 23:02:10 [INFO]      2023-03-20T22:02:10.948Z                successfully patched module sqlite3
2023-03-20 23:02:11 [INFO]      2023-03-20T22:02:11.032Z                Found credentials in environment variables.
2023-03-20 23:02:11 [WARNING]   2023-03-20T22:02:11.290Z                Subsegment lambda discarded due to Lambda worker still initializing
2023-03-20 23:02:12 [WARNING]   2023-03-20T22:02:12.044Z                No subsegment to end.

dominikschubert commented Mar 21, 2023

Copy link
Copy Markdown
Member Author

Subsegment lambda discarded due to Lambda worker still initializing

Seems to be related to this issue here: aws/aws-xray-sdk-python#185

dominikschubert merged commit bede0a0 into localstack Mar 21, 2023
dominikschubert deleted the feat_xray-integration branch March 21, 2023 07:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

joe4dev joe4dev approved these changes
dfangl Awaiting requested review from dfangl dfangl is a code owner

Assignees

dominikschubert

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL