pulumi/.github/workflows/on-pr.yml

152 lines
4.9 KiB
YAML
Raw Normal View History

2022-09-02 05:40:13 +00:00
name: Pull Request
permissions:
# To create a draft release.
2022-09-02 05:40:13 +00:00
contents: write
# To comment on PRs.
2022-09-02 05:40:13 +00:00
pull-requests: write
# To sign artifacts.
id-token: write
2022-09-02 05:40:13 +00:00
on:
pull_request:
paths-ignore:
- "sdk/.version"
2022-09-02 05:40:13 +00:00
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# The jobs in this workflow are only run on branches. The `on-community-pr.yml` job provides
# commands for running workflows from forks.
2022-09-02 05:40:13 +00:00
jobs:
changelog-comment:
name: changelog preview
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
2022-09-02 05:40:13 +00:00
permissions:
contents: read
pull-requests: write
uses: ./.github/workflows/on-pr-changelog.yml
with:
ref: ${{ github.ref }}
base-ref: origin/${{ github.base_ref }}
pr-number: ${{ github.event.pull_request.number }}
changelog-required: ${{ !contains(github.event.pull_request.labels.*.name, 'impact/no-changelog-required') }}
secrets: inherit
info:
name: info
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
2022-09-02 05:40:13 +00:00
uses: ./.github/workflows/ci-info.yml
permissions:
contents: read
with:
ref: ${{ github.ref }}
is-snapshot: true
secrets: inherit
# Determines which files have changed so we can avoid running expensive tests
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
# if they're not necessary.
inspect:
name: Inspect changed files
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
runs-on: ubuntu-latest
steps:
- uses: dorny/paths-filter@v3
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
id: changes
with:
token: ${{ secrets.GITHUB_TOKEN }}
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
filters: |
# If files matching any of these patterns change,
# we will run codegen tests for pull requests.
test-codegen:
- 'pkg/codegen/docs/**'
- 'pkg/codegen/dotnet/**'
- 'pkg/codegen/go/**'
- 'pkg/codegen/nodejs/**'
- 'pkg/codegen/python/**'
outputs:
# Add an entry here for every named pattern
# defined in filters.
test-codegen: ${{ steps.changes.outputs.test-codegen }}
2022-09-02 05:40:13 +00:00
ci:
name: CI
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
needs: [info, inspect]
2022-09-02 05:40:13 +00:00
uses: ./.github/workflows/ci.yml
permissions:
contents: read
# To sign artifacts.
id-token: write
2022-09-02 05:40:13 +00:00
with:
ref: ${{ github.ref }}
version: ${{ needs.info.outputs.version }}
lint: true
build-all-targets: ${{ contains(github.event.pull_request.labels.*.name, 'ci/test') }}
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
# codegen tests take quite a while to run.
# Run them only if ci/test is set,
# or if one of the codegen files changed.
2023-08-10 23:19:12 +00:00
test-codegen: >- # No newlines or trailing newline.
ci: Don't run codegen tests for PRs We currently run all codegen tests in pkg/codegen/$lang for every PR. These tests take quite a while to run and lock up many GitHub workers for this entire duration. This change attempts to address this issue by running codegen tests only for those PRs that touch the codegen directories. The machinery to make this work is roughly as follows: - In the on-pr workflow, when we're figuring out what we're doing, we check if we've changed codegen files. We use [paths-filter] to do this. - We decide whether we want to run codegen tests based on those files, and pass that onto the test matrix generator. - The test matrix generator filters out these packages and their subpackages from the list of tests under consideration. - Everything else proceeds as normal. [paths-filter]: https://github.com/dorny/paths-filter Things to note: - The test-codegen input defaults to true. All other invocations will run with codegen tests so these will continue to run on merge. Only PRs (from on-pr.yml) set it to false. - Since the number of tests is remarkably smaller without these tests, we can significantly reduce the number of partitions we use for pkg/ unit tests. This should alleviate pressure on GitHub workers further. This is a pretty blunt approach to the problem. If we wanted to be more targeted, instead of filtering at the get-job-matrix.py level, we could instead set an environment variable and add t.Skips in {program,sdk,type}_driver if that environment variable is set. And we can still do that in the future if we decide that maintaining this list is too much. Resolves #12334
2023-03-02 22:33:45 +00:00
${{
contains(github.event.pull_request.labels.*.name, 'ci/test') ||
(needs.inspect.outputs.test-codegen == 'true')
}}
test-version-sets: >- # No newlines or trailing newline.
${{
contains(github.event.pull_request.labels.*.name, 'ci/test')
&& 'minimum current'
|| 'current'
}}
integration-test-platforms: ubuntu-latest
acceptance-test-platforms: >- # No newlines or trailing newline.
${{
contains(github.event.pull_request.labels.*.name, 'ci/test')
&& 'macos-latest windows-latest'
|| ''
}}
# We'll only upload coverage artifacts with the periodic-coverage cron workflow.
enable-coverage: false
2022-09-02 05:40:13 +00:00
secrets: inherit
prepare-release:
name: prepare
if: >- # No newlines or trailing newline.
${{
github.event.pull_request.head.repo.full_name == github.repository
&& contains(github.event.pull_request.labels.*.name, 'ci/test')
}}
2022-09-02 05:40:13 +00:00
needs: [info, ci]
uses: ./.github/workflows/ci-prepare-release.yml
permissions:
contents: write
# To sign artifacts.
id-token: write
2022-09-02 05:40:13 +00:00
with:
ref: ${{ github.ref }}
version: ${{ needs.info.outputs.version }}
release-notes: ${{ needs.info.outputs.release-notes }}
project: ${{ github.repository }}
draft: true
prerelease: true
secrets: inherit
ci-ok:
name: ci-ok
ci: Remove Bors (#13715) # Description This PR removes Bors in favor of native GitHub merge queues. A very similar cutover was tested and verified in https://github.com/pulumi/watchutil-rs/pull/39. # Deployment plan * [x] Find some time where PRs aren't being merged. This is mostly to minimize confusion while we're futzing with branch protections. * [x] Update the repo's branch protection rules to match https://github.com/pulumi/watchutil-rs/pull/38: * "Allow specified actors to bypass required pull requests": "bors" → Unchecked * "Status checks that are required": "bors" → "ci-ok" * "Require merge queue": Unchecked → Checked * "Only merge non-failing pull requests" → Unchecked * "Merge method" → Squash and merge * [ ] Merge this PR using the "Merge when ready" button. * [ ] If it merges successfully: confirm the prepare-release step ran and published pre-release artifacts. * [ ] If the merge fails: it's possible we'll need to tweak steps slightly (e.g. if the step's name doesn't match the branch protection rule). The end goal is a "ci-ok" step in both the PR and merge workflows that captures whether everything is green. * [ ] If the issues aren't immediately resolvable, revert the changes made to branch protection settings. Folks will continue to merge with Bors. If all goes well, we should be able to continue merging with MQ for a while -- at least long enough to cut a new CLI release. Once we're confident we won't need to roll back: * [ ] Delete branch protection rules for: * staging * testing * [ ] Remove pu/pu from https://app.bors.tech/repositories Fixes https://github.com/pulumi/pulumi/issues/13501 ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-08-12 01:36:48 +00:00
needs: [ci]
if: always()
runs-on: ubuntu-latest
steps:
# If the PR is a community PR, we don't run the usual checks, but instead
# require maintainers to run them manually by commenting with
# /run-acceptance-tests. Since the checks then run outside of the PR,
# ci-ok is not a useful status in these cases. We thus want to skip this
# job for community PRs. We detect community contributions by checking if
# the head repository is different to the base repository.
- name: CI failed
if: ${{ github.event.pull_request.head.repo.full_name == github.repository && needs.ci.result != 'success' }}
run: exit 1
- name: CI succeeded
run: exit 0