Skip to content

Conversation

blampe
Copy link
Contributor

@blampe blampe commented Dec 19, 2024

We have a few goals with ci-mgmt:

  • Consolidate native and bridged provider workflows.
  • Make workflows easier to run locally as well as in CI.
  • Eliminate provider-specific special-casing, as it's not sustainable for hundreds of providers.

One of the special cases to address is p-k's cluster setup, which currently injects special jobs/steps into the normal workflow.

Instead, we can setup (and optionally tear down) the cluster as part of the test suite.

This PR revives some of @rquitales' work in master...rquitales/tune-presubmits. The setup is simplified somewhat to more closely mirror what we do today -- spin up a single KinD cluster on CI, or a GKE cluster on releases.

GKE is handled via API calls instead of as a Pulumi stack, which is (arguably) simpler and lets us initiate cluster deletion without needing to wait for it to be completely gone.

When run locally, we will continue to use existing ~/.kube/config. We will only create a new cluster locally if you happen to run tests without any cluster at all -- for example if you're a new developer.

The setup is slightly suboptimal because TestMain only applies at a package level, and our nodejs e2e have two packages. We'll also eventually spin up a GKE cluster per test shard, instead of one for the entire run. We can consider ways to optimize this later.

The KinD support works great. GKE is still a WIP and the PR's tests are failing while I debug that.

The ci-mgmt changes will be upstreamed.

@blampe blampe marked this pull request as draft December 19, 2024 23:54
@blampe blampe added the impact/no-changelog-required This issue doesn't require a CHANGELOG update label Dec 19, 2024
Copy link

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

Copy link

codecov bot commented Dec 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.08%. Comparing base (82406b3) to head (e565a46).
Report is 68 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3386      +/-   ##
==========================================
- Coverage   41.14%   41.08%   -0.07%     
==========================================
  Files          85       87       +2     
  Lines       12764    12900     +136     
==========================================
+ Hits         5252     5300      +48     
- Misses       7122     7205      +83     
- Partials      390      395       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blampe blampe changed the title Handle cluster setup Remove cluster setup from CI Dec 20, 2024
@blampe blampe removed their assignment May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cicd impact/no-changelog-required This issue doesn't require a CHANGELOG update
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants