Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discussion: Is it better to remove OS from the cache key? #219

Closed
wants to merge 2 commits into from

Conversation

KengoTODA
Copy link
Contributor

Description:

This PR is for discussion about the RUNNER_OS in the cache key.

I found an opinion at Twitter: it's needless because Java's package are portable, and I agree with it.
In my understanding, the Maven repository provides no feature to serve different files to different OSs.

I prepared necessary changes to hold discussion with you. How do you think about this change?
It may make the build efficient in some case, especially when we run builds on multiple OSs in serial.

Related issue:

Check list:

@KengoTODA KengoTODA force-pushed the remove-os-from-cache-key branch from fe37d8c to 7b921a1 Compare September 6, 2021 02:13
const hash = await glob.hashFiles(packageManager.pattern.join('\n'));
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${hash}`;
return `${CACHE_KEY_PREFIX}-${packageManager.id}-${hash}`;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess it should be configurable.
As mentionned on twitter, it is rare but sometimes needed (native like jansi/netty/...).
Default shouldnt use it IMHO but should be possible to have it.
Alternatively, cache creation can check binary (.so) in jar presence maybe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is rare but sometimes needed (native like jansi/netty/...).

ah I got it, Maven repo always serves same files but Maven project may use build profile to add dependencies only to specific OS. Gradle project can do similar build too.

In my current opinion, it's hard to provide a smart and high-speed feature detecting such dependency on OS. 😢
Then next consideration is how to make it configurable; but it may make the workflow file complex and nonintuitive. Here I imagine new settings like cache: os-dependent-maven and cache-key-strategy: os-dependent.

So I suggest making cache key OS independent for major use cases, and ask rare case users keep using actions/cache to make the deps cache OS dependent. Any thought?

Thanks again for making the product better with me 🙌

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as an user, I would expect the composite actions to expose the underlying composed actions and just - optionally - provide nice defaults if possible.
So overall it would be something like cache: <actions/cache configuration to merge with defaults of this action (lodash like merge)>.
If done, the key can stay OS dependent by default while the doc explicitly state that and shows how to drop the os specific part IMHO.
It will also enable to exclude or not part of the files (a common example is to exclude snapshots) quite easily.
So at the end, the feature can also be, to not have the cache by default until cache subaction is enabled by setting at least a key.
A warning can be logged if not done saying the build can be slow.
Something along those lines wouldn't shock me to be honest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So overall it would be something like cache: <actions/cache configuration to merge with defaults of this action (lodash like merge)>.

Like below?

- uses: actions/setup-java@v2
   with:
     java-version: 11
     distribution: temurin
     cache:
       path: ~/.m2/repository
       key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
       restore-keys: |
         ${{ runner.os }}-maven-

I don't know that the Actions support nested (object type) input, though.
It's surely intuitive but brings limited benefit to shorten the workflow file, I think.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes
The benefit is to override the defaults properly (typically path including the repo but NOT *-SNAPSHOT for ex and key using pom.xml properly).
Overall point is that if the benefit is low, the benefit to include the cache should be the same - but I strongly think it is good to encourage it.
Maybe just checking the yaml has cache action defined it a good outcome instead of trying to merge them.
Will also be saner for all hybrid builds (we have more and more npm+mvn builds these days).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have plans to add any customization of caching in scope of actions/setup-java. If you need customization, it is recomended to use actions/cache directly.
We had discussion about it when implemented caching for Node.js: https://github.com/actions/setup-node/blob/main/docs/adrs/0000-caching-dependencies.md#goals--anti-goals

We don't pursue the goal to provide wide customization of caching in scope of actions/setup-node action. The purpose of this integration is covering ~90% of basic use-cases. If user needs flexible customization, we should advice them to use actions/cache directly.

As for the removing platform from key, we will investigate it deeper

@KengoTODA
Copy link
Contributor Author

This PR is stale for months, and there is no more discussions on this topic so I will close this PR. Thanks all for your help! :)

@KengoTODA KengoTODA closed this Mar 16, 2022
@KengoTODA KengoTODA deleted the remove-os-from-cache-key branch March 16, 2022 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants