FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

GH-6: Add Linux test CI by lidavidm · Pull Request #339 · apache/arrow-java · GitHub

/ arrow-java Public

GH-6: Add Linux test CI - #339

Merged
kou merged 3 commits into
apache:mainfrom
lidavidm:gh-6
Nov 28, 2024
ConversationCommits (3)ChecksFiles changed
Merged

GH-6: Add Linux test CI#339
kou merged 3 commits into
apache:mainfrom
lidavidm:gh-6

Conversation

lidavidm commented Nov 27, 2024

Copy link
Copy Markdown
Member

Fixes #6.

lidavidm marked this pull request as ready for review November 27, 2024 02:59
kou reviewed Nov 27, 2024
View reviewed changes
Comment thread .gitmodules Outdated
Comment thread .github/workflows/java.yml

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

How about test.yml instead of java.yml?

Comment thread .env Outdated
# coredump generation set it to 0
ULIMIT_CORE=-1

# Default versions for platforms

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can we remove this?

Comment thread .env Outdated
ARCH_SHORT=amd64

# Default repository to pull and push images from
REPO=apache/arrow-dev

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

How about using ghcr.io/apache/arrow-java-dev?
If we use Docker Hub, we need to set Docker Hub secrets.

Comment thread .github/workflows/java.yml Outdated
uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
with:
path: .docker
key: maven-${{ hashFiles('java/**') }}

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Could you update the hashFiles() argument?

Comment thread .github/workflows/java.yml Outdated
Comment thread .github/workflows/java.yml Outdated
Comment thread .github/workflows/java.yml
restore-keys: maven-
- name: Execute Docker Build
env:
DEVELOCITY_ACCESS_KEY: ${{ secrets.GE_ACCESS_TOKEN }}

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Do we need to set this secret?

lidavidm Nov 27, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Yes, we'll need to ask Infra to set it up again - this implements build caching (I'll have to figure out the details). It's not strictly required, though

Comment thread arrow-format/README.rst Outdated
Comment on lines +21 to +25
This folder contains binary protocol definitions for the Arrow columnar format
and other parts of the project, like the Flight RPC framework.

For documentation about the Arrow format, see the `docs/source/format`
directory.

kou Nov 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

How about mentioning that this is a copy of https://github.com/apache/arrow/tree/main/format ?
We may want to mention the revision of the copy.

lidavidm and others added 2 commits November 27, 2024 01:32
Apply suggestions from code review
deb9400 
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Review feedback
432056f
kou approved these changes Nov 28, 2024
View reviewed changes

kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

+1

Comment thread .github/workflows/test.yml
Comment on lines +71 to +74
docker compose run \
-e CI=true \
-e "DEVELOCITY_ACCESS_KEY=$DEVELOCITY_ACCESS_KEY" \
${{ matrix.image }}

kou Nov 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We need docker compose pull/docker compose push to cache built images but let's work on it as a separated task.

kou Nov 28, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Sorry. We don't need this. Because we use existing Docker images...

We can remove REPO from .env.

kou reviewed Nov 28, 2024
View reviewed changes
Comment thread .github/workflows/test.yml
uses: actions/cache@6849a6489940f00c2f30c0fb92c6274307ccb58a # v4.1.2
with:
path: .docker
key: maven-${{ matrix.jdk }}-${{ matrix.maven }}-${{ hashFiles('**/docker-compose.yml') }}

kou Nov 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We may want to update cache by .java changes. Let's work on it as a separated task.

lidavidm Nov 28, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hmm, since the docker image is just jdk/maven, shouldn't it not depend on the actual Java sources?

kou Nov 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think that this is used to cache mvn ... results. So the cache will depend on *.java too.

lidavidm Nov 28, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hmm, the build script starts by wiping the build directory, though. (And we have separate build caching support.)

kou Nov 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Oh, I'm not familiar with Maven cache...

This .docker path caches ~/.m2/ directory. Do we need to cache ~/.m2/? If we don't need to cache ~/.m2/, we can remove this cache.

lidavidm Nov 28, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ah, this is confusing then. If it caches .m2 then it'll cache downloaded dependencies, which is useful, and that depends on pom.xml (not .java though)

lidavidm Nov 28, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'll update this to hash pom.xml in the new PR

kou merged commit 717db82 into apache:main Nov 28, 2024
lidavidm deleted the gh-6 branch November 28, 2024 02:04
lidavidm added this to the 18.2.0 milestone Jan 30, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

kou kou approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

18.2.0

Development

Successfully merging this pull request may close these issues.

Add test CI: Docker based

2 participants

Back | FazBrowse Home | New Git URL