Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi-module releases in go-control-plane #714

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alecholmez
Copy link
Contributor

@alecholmez alecholmez commented Jun 5, 2023

This PR introduces support for multi-module releases in go-control-plane following the plan of action outlined in issue #391.

A few things have happened:

  • Bumped modules to Go 1.20 (to inherit support for workspaces)
  • Introduced a go workspace that orchestrates all the subcomponents of this repo
  • Initialized new modules for ./envoy and ./contrib containing the generated go stubs
  • Renamed build/ to scripts/ so we have a bucket for more general repo automation

When this PR is merged, a tag will be cut at v0.11.2 but a new tag will be introduced with envoy/v1.26.2. closes #391

valerian-roche
valerian-roche previously approved these changes Jun 5, 2023
Copy link
Contributor

@valerian-roche valerian-roche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with go workspaces, but overall lgtm
Nice to see independent versioning through this though


go 1.20

replace github.com/envoyproxy/go-control-plane/envoy => ../envoy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding from go.work is that it should no longer be needed to add those replace commands
Is that actually not the case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually couldn't get go mod tidy to play nice without them. I can mess around with it a bit more but figured this was fine for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to drop the replace from ./envoy but that one still complained so I left it in.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is an indication that we don't have the right structure? Since the "envoy" and "contrib" protobufs are bound to the same Envoy release, does it make sense to allow the possibility for a program to import different versions of them?

How hard would it be to move the contrib protobufs to "envoy/contrib"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I suppose that a similar question arises for the "./ratelimit" directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heres a branch of Contour using the above branch: https://github.com/sunjayBhatia/contour/tree/use-go-control-plane-multi-module

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the above go-control-plane branch also copies the ratelimit module to a package in the new api module)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this branch of Contour works with this PR's branch (plus this commit sunjayBhatia@4be6f62): sunjayBhatia/contour@dc8fa81

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think either approach will work just fine, though the upgrade/transition UX might be cleaner (no possibility for users complaining about ambiguous package paths if they are requiring mismatched versions) if the new modules have a new package path

Copy link
Contributor

@jpeach jpeach Jun 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the above go-control-plane branch also copies the ratelimit module to a package in the new api module)

I'll check later, but me recollection is that the ratelimit protos don't come from the envoy repository. So they probably need to be versioned differently and it might not make sense for them to be in the same module as the envoy api.

.gitignore Outdated Show resolved Hide resolved
contrib/go.mod Outdated
@@ -0,0 +1,23 @@
module github.com/envoyproxy/go-control-plane/contrib

go 1.20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JFYI, we should be prepared for some pushback on this. It's possible that some consumer of this module will be using earlier Go versions (see #271 for example).

Copy link
Contributor Author

@alecholmez alecholmez Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try and roll back to 1.18 but that is a bare minimum requirement for workspaces. Wondering if we should see if it's that problematic and roll back if people complain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 please don't use 1.20. In grpc-go, which uses this, we support 1.18+ and also haven't made any changes AFAIK that would prevent even 1.17 or maybe 1.16 from working (e.g. no use of any or generics). IMO this (and the use of new language features) should stay at the oldest version you can reasonably handle supporting.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alecholmez it sounds like this is the most contentious point within this change set.

@jpeach
Copy link
Contributor

jpeach commented Jun 6, 2023

  • Initialized new modules for ./envoy and ./contrib containing the generated go stubs

We probably want to do the same for ./ratelimit too?

@alecholmez alecholmez force-pushed the multi-module-release branch 2 times, most recently from 0ab1e02 to 1b350d3 Compare June 6, 2023 15:00
go.work Outdated
@@ -0,0 +1,10 @@
go 1.20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the dumb question...what do go workspaces actually do for you? Someone wanted to add a go.work file to our repo, but I didn't fully understand how it would help. We have several modules and don't use a workspace, and things generally work fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they are more useful in local development etc. when you have modules side-by-side rather than a module hierarchy like we have here, you should mainly be able to remove the replace directives but it doesn't seem (from my playing around with it as well) that we get that here because of the package/module structure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. to get the other modules in to actually use the toher modules in this repo all the way, had to do this: sunjayBhatia@4be6f62

so I'm not sure how much the workspace is getting us

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the decision to ultimately remove the workspace? I'm fine with that

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added the stale label Jul 10, 2023
@github-actions
Copy link

This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot closed this Jul 17, 2023
@alecholmez alecholmez reopened this Jul 17, 2023
@alecholmez
Copy link
Contributor Author

Hey all, been away for quite a while. Starting to catch back up on this project. Did we ever come to a consensus on the way forward here? I have some cycles I can spend to push this across so we can get onto a more regular release cycle.

@mmorel-35 mmorel-35 force-pushed the multi-module-release branch 2 times, most recently from d3be1f4 to 671b920 Compare November 24, 2024 10:39
mmorel-35
mmorel-35 previously approved these changes Nov 24, 2024
go.work Outdated
@@ -0,0 +1,10 @@
go 1.21
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this was discussed before but not resolved, and I see this PR was approved:

https://go.dev/ref/mod#go-work-file

It is generally inadvisable to commit go.work files into version control systems, for two reasons:

  • A checked-in go.work file might override a developer’s own go.work file from a parent directory, causing confusion when their use directives don’t apply.
  • A checked-in go.work file may cause a continuous integration (CI) system to select and thus test the wrong versions of a module’s dependencies. CI systems should generally not be allowed to use the go.work file so that they can test the behavior of the module as it would be used when required by other modules, where a go.work file within the module has no effect.

That said, there are some cases where committing a go.work file makes sense. For example, when the modules in a repository are developed exclusively with each other but not together with external modules, there may not be a reason the developer would want to use a different combination of modules in a workspace. In that case, the module author should ensure the individual modules are tested and released properly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the related issue that ultimately led to this addition:

golang/go#53502

Copy link
Contributor

@mmorel-35 mmorel-35 Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t mind seeing them removed.
By the way, help on multimodule release is welcome here

@mmorel-35 mmorel-35 force-pushed the multi-module-release branch 3 times, most recently from 3325c95 to d1ca98d Compare December 12, 2024 07:14
@mmorel-35
Copy link
Contributor

Concerning the multimodule publishing mecanism, has anyone considered using opentelemetry/multimod

This seems to fit this use case

@mmorel-35 mmorel-35 force-pushed the multi-module-release branch 4 times, most recently from f72de1d to 3ad429a Compare December 14, 2024 19:40
Signed-off-by: Alec Holmes <[email protected]>
Signed-off-by: Valerian Roche <[email protected]>
Signed-off-by: Matthieu MOREL <[email protected]>
@mmorel-35 mmorel-35 force-pushed the multi-module-release branch 7 times, most recently from 77e73b7 to 264890a Compare December 16, 2024 07:03
Signed-off-by: Matthieu MOREL <[email protected]>
@mmorel-35 mmorel-35 force-pushed the multi-module-release branch from 264890a to 952bc4c Compare December 16, 2024 07:27
@mmorel-35 mmorel-35 assigned dfawley and unassigned dfawley Dec 16, 2024
@mmorel-35 mmorel-35 marked this pull request as draft December 16, 2024 23:09
Copy link
Contributor

@valerian-roche valerian-roche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, can we set this PR as ready for review?
I believe we should be able to push on this now.
Thanks for all the work on this!

@mmorel-35 mmorel-35 marked this pull request as ready for review December 17, 2024 05:01
@mmorel-35
Copy link
Contributor

There is one last part to be added which is the prepare-release
See https://github.com/open-telemetry/opentelemetry-collector/blob/main/Makefile#L355-L393

Or https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/Makefile#L413-L426
For examples , if you'd like to take it from here as that's the crucial part that releasers will need to manipulate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Develop release policy
8 participants