pulumi/scripts/go-wrapper.sh

74 lines
2.0 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
#
# To be used with Goreleaser as `gobinary` implementation as a
# replacement for the `go` toolchain.
#
# Function: coverage-enabled builds for Pulumi CLI
#
# This builds binaries via `go test -c` workaround. Disabled for
# Windows builds. Only enabled on the Pulumi CLI binaries.
set -euo pipefail
ci: Track code coverage **Overview** This re-enables tracking of code coverage. For Go, there are two kinds of coverage at play: unit test and integration test coverage. Unit tests follow the usual pattern of running `go test -cover -coverprofile=whatever.cov`. For integration tests, we use the new integration test profiling support [added in Go 1.20](https://go.dev/testing/coverage/). In short, the way it works is: # Build a coverage instrumented binary: go build -cover # Set GOCOVERDIR to a directory and run the integration tests # that will invoke this coverage-instrumented binary. GOCOVERDIR=$(pwd)/coverage go test ./tests # $GOCOVERDIR will now be filled with coverage data # from every invocation of the coverage-instrumented binary. # Combine it into a single coverage file: go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov # The resulting file can be uploaded to codecov as-is. The above replaces the prior, partially working hacks we had in place to get coverage-instrumented binaries with `go test -c` and hijacking the TestMain. **Notable changes** - TestMain hijacking is deleted from the Pulumi CLI. We no longer need this to build coverage-instrumented binaries. - ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH because the Pulumi binary no longer accepts a test.coverprofile flag. This information is now in the GOCOVERDIR environment variable. - We add an `enable-coverage` parameter to the `ci-build-binaries` workflow to mirror some of the other workflows. It will produce coverage-instrumented binaries if this is true. These binaries are then used by `ci-run-test` which will set `GOCOVERDIR` and merge the coverage results from it. - Coverage configuration no longer counts tests, testdata, and Protobuf-generated code against coverage. - go-wrapper.sh: Because we're no longer relying on the `go test -c` hack, this no longer excludes Windows and language providers from coverage tracking. - go-test.py and go-wrapper.sh will include pulumi-language-go and pulumi-language-nodejs in covered packages. *Other changes* - go-test.py: Fixed a bug where `args` parameters added for coverage were ignored. Note that this change DOES NOT track coverage for calls made to Pulumi packages by plugins downloaded from external sources, e.g. provider plugins. Arguably, that's out of scope of coverage trackcing for the Pulumi repository. Resolves #8615, #11419
2023-06-27 16:57:36 +00:00
PULUMI_TEST_COVERAGE_PATH=${PULUMI_TEST_COVERAGE_PATH:-}
PULUMI_BUILD_MODE=${PULUMI_BUILD_MODE:-}
COVER_PACKAGES=( \
"github.com/pulumi/pulumi/pkg/v3/..." \
"github.com/pulumi/pulumi/sdk/v3/..." \
"github.com/pulumi/pulumi/sdk/go/pulumi-language-go/v3/..." \
"github.com/pulumi/pulumi/sdk/nodejs/cmd/pulumi-language-nodejs/v3/..." \
"github.com/pulumi/pulumi/sdk/python/cmd/pulumi-language-python/v3/..." \
ci: Track code coverage **Overview** This re-enables tracking of code coverage. For Go, there are two kinds of coverage at play: unit test and integration test coverage. Unit tests follow the usual pattern of running `go test -cover -coverprofile=whatever.cov`. For integration tests, we use the new integration test profiling support [added in Go 1.20](https://go.dev/testing/coverage/). In short, the way it works is: # Build a coverage instrumented binary: go build -cover # Set GOCOVERDIR to a directory and run the integration tests # that will invoke this coverage-instrumented binary. GOCOVERDIR=$(pwd)/coverage go test ./tests # $GOCOVERDIR will now be filled with coverage data # from every invocation of the coverage-instrumented binary. # Combine it into a single coverage file: go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov # The resulting file can be uploaded to codecov as-is. The above replaces the prior, partially working hacks we had in place to get coverage-instrumented binaries with `go test -c` and hijacking the TestMain. **Notable changes** - TestMain hijacking is deleted from the Pulumi CLI. We no longer need this to build coverage-instrumented binaries. - ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH because the Pulumi binary no longer accepts a test.coverprofile flag. This information is now in the GOCOVERDIR environment variable. - We add an `enable-coverage` parameter to the `ci-build-binaries` workflow to mirror some of the other workflows. It will produce coverage-instrumented binaries if this is true. These binaries are then used by `ci-run-test` which will set `GOCOVERDIR` and merge the coverage results from it. - Coverage configuration no longer counts tests, testdata, and Protobuf-generated code against coverage. - go-wrapper.sh: Because we're no longer relying on the `go test -c` hack, this no longer excludes Windows and language providers from coverage tracking. - go-test.py and go-wrapper.sh will include pulumi-language-go and pulumi-language-nodejs in covered packages. *Other changes* - go-test.py: Fixed a bug where `args` parameters added for coverage were ignored. Note that this change DOES NOT track coverage for calls made to Pulumi packages by plugins downloaded from external sources, e.g. provider plugins. Arguably, that's out of scope of coverage trackcing for the Pulumi repository. Resolves #8615, #11419
2023-06-27 16:57:36 +00:00
)
# Join COVER_PACKAGES with commas.
COVERPKG=$(IFS=,; echo "${COVER_PACKAGES[*]}")
case "$1" in
build)
ci: Track code coverage **Overview** This re-enables tracking of code coverage. For Go, there are two kinds of coverage at play: unit test and integration test coverage. Unit tests follow the usual pattern of running `go test -cover -coverprofile=whatever.cov`. For integration tests, we use the new integration test profiling support [added in Go 1.20](https://go.dev/testing/coverage/). In short, the way it works is: # Build a coverage instrumented binary: go build -cover # Set GOCOVERDIR to a directory and run the integration tests # that will invoke this coverage-instrumented binary. GOCOVERDIR=$(pwd)/coverage go test ./tests # $GOCOVERDIR will now be filled with coverage data # from every invocation of the coverage-instrumented binary. # Combine it into a single coverage file: go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov # The resulting file can be uploaded to codecov as-is. The above replaces the prior, partially working hacks we had in place to get coverage-instrumented binaries with `go test -c` and hijacking the TestMain. **Notable changes** - TestMain hijacking is deleted from the Pulumi CLI. We no longer need this to build coverage-instrumented binaries. - ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH because the Pulumi binary no longer accepts a test.coverprofile flag. This information is now in the GOCOVERDIR environment variable. - We add an `enable-coverage` parameter to the `ci-build-binaries` workflow to mirror some of the other workflows. It will produce coverage-instrumented binaries if this is true. These binaries are then used by `ci-run-test` which will set `GOCOVERDIR` and merge the coverage results from it. - Coverage configuration no longer counts tests, testdata, and Protobuf-generated code against coverage. - go-wrapper.sh: Because we're no longer relying on the `go test -c` hack, this no longer excludes Windows and language providers from coverage tracking. - go-test.py and go-wrapper.sh will include pulumi-language-go and pulumi-language-nodejs in covered packages. *Other changes* - go-test.py: Fixed a bug where `args` parameters added for coverage were ignored. Note that this change DOES NOT track coverage for calls made to Pulumi packages by plugins downloaded from external sources, e.g. provider plugins. Arguably, that's out of scope of coverage trackcing for the Pulumi repository. Resolves #8615, #11419
2023-06-27 16:57:36 +00:00
MODE="$PULUMI_BUILD_MODE"
if [ -z "$MODE" ]; then
# If a build mode was not specified,
# guess based on whether a coverage path was supplied.
MODE=coverage
ci: Track code coverage **Overview** This re-enables tracking of code coverage. For Go, there are two kinds of coverage at play: unit test and integration test coverage. Unit tests follow the usual pattern of running `go test -cover -coverprofile=whatever.cov`. For integration tests, we use the new integration test profiling support [added in Go 1.20](https://go.dev/testing/coverage/). In short, the way it works is: # Build a coverage instrumented binary: go build -cover # Set GOCOVERDIR to a directory and run the integration tests # that will invoke this coverage-instrumented binary. GOCOVERDIR=$(pwd)/coverage go test ./tests # $GOCOVERDIR will now be filled with coverage data # from every invocation of the coverage-instrumented binary. # Combine it into a single coverage file: go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov # The resulting file can be uploaded to codecov as-is. The above replaces the prior, partially working hacks we had in place to get coverage-instrumented binaries with `go test -c` and hijacking the TestMain. **Notable changes** - TestMain hijacking is deleted from the Pulumi CLI. We no longer need this to build coverage-instrumented binaries. - ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH because the Pulumi binary no longer accepts a test.coverprofile flag. This information is now in the GOCOVERDIR environment variable. - We add an `enable-coverage` parameter to the `ci-build-binaries` workflow to mirror some of the other workflows. It will produce coverage-instrumented binaries if this is true. These binaries are then used by `ci-run-test` which will set `GOCOVERDIR` and merge the coverage results from it. - Coverage configuration no longer counts tests, testdata, and Protobuf-generated code against coverage. - go-wrapper.sh: Because we're no longer relying on the `go test -c` hack, this no longer excludes Windows and language providers from coverage tracking. - go-test.py and go-wrapper.sh will include pulumi-language-go and pulumi-language-nodejs in covered packages. *Other changes* - go-test.py: Fixed a bug where `args` parameters added for coverage were ignored. Note that this change DOES NOT track coverage for calls made to Pulumi packages by plugins downloaded from external sources, e.g. provider plugins. Arguably, that's out of scope of coverage trackcing for the Pulumi repository. Resolves #8615, #11419
2023-06-27 16:57:36 +00:00
if [ -z "$PULUMI_TEST_COVERAGE_PATH" ]; then
MODE=normal
fi
fi
Run integration tests with race detection (#15895) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Combining #15120 and #15124 and including fixes to allow tests to pass. Enable race detection in the binary we're using for integration tests. This will allow us to catch more data races before they get into any release. This does mean the binary we're using for integration tests is slightly different from the binary we're releasing, however that's already the case as we're running a binary with coverage enabled for them. Later we rebuild the binary we're actually releasing. This requires us to fix the race between snapshot code, display code, and the step executor. I've done that by adding a lock to the State struct. This does not feel great, but it's a quick way to fix this and get race detection running (and unblocks #15871 which was also hitting the race detector because it started pulling snapshot code into unit tests as well). There's probably a more principled overhaul that doesn't require locking at this level. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 15:58:42 +00:00
RACE=
CGO_ENABLED=0
if [ "$PULUMI_ENABLE_RACE_DETECTION" = "true" ]; then
RACE='-race'
if [ "$(go env GOOS)" != "darwin" ]; then
# On macOS, we don't need CGO but windows and linux still do.
CGO_ENABLED=1
fi
fi
export CGO_ENABLED
case "$MODE" in
normal)
Run integration tests with race detection (#15895) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Combining #15120 and #15124 and including fixes to allow tests to pass. Enable race detection in the binary we're using for integration tests. This will allow us to catch more data races before they get into any release. This does mean the binary we're using for integration tests is slightly different from the binary we're releasing, however that's already the case as we're running a binary with coverage enabled for them. Later we rebuild the binary we're actually releasing. This requires us to fix the race between snapshot code, display code, and the step executor. I've done that by adding a lock to the State struct. This does not feel great, but it's a quick way to fix this and get race detection running (and unblocks #15871 which was also hitting the race detector because it started pulling snapshot code into unit tests as well). There's probably a more principled overhaul that doesn't require locking at this level. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 15:58:42 +00:00
shift
go build ${RACE} "$@"
;;
coverage)
shift
Run integration tests with race detection (#15895) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Combining #15120 and #15124 and including fixes to allow tests to pass. Enable race detection in the binary we're using for integration tests. This will allow us to catch more data races before they get into any release. This does mean the binary we're using for integration tests is slightly different from the binary we're releasing, however that's already the case as we're running a binary with coverage enabled for them. Later we rebuild the binary we're actually releasing. This requires us to fix the race between snapshot code, display code, and the step executor. I've done that by adding a lock to the State struct. This does not feel great, but it's a quick way to fix this and get race detection running (and unblocks #15871 which was also hitting the race detector because it started pulling snapshot code into unit tests as well). There's probably a more principled overhaul that doesn't require locking at this level. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 15:58:42 +00:00
go build ${RACE} -cover -coverpkg "$COVERPKG" "$@"
ci: Track code coverage **Overview** This re-enables tracking of code coverage. For Go, there are two kinds of coverage at play: unit test and integration test coverage. Unit tests follow the usual pattern of running `go test -cover -coverprofile=whatever.cov`. For integration tests, we use the new integration test profiling support [added in Go 1.20](https://go.dev/testing/coverage/). In short, the way it works is: # Build a coverage instrumented binary: go build -cover # Set GOCOVERDIR to a directory and run the integration tests # that will invoke this coverage-instrumented binary. GOCOVERDIR=$(pwd)/coverage go test ./tests # $GOCOVERDIR will now be filled with coverage data # from every invocation of the coverage-instrumented binary. # Combine it into a single coverage file: go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov # The resulting file can be uploaded to codecov as-is. The above replaces the prior, partially working hacks we had in place to get coverage-instrumented binaries with `go test -c` and hijacking the TestMain. **Notable changes** - TestMain hijacking is deleted from the Pulumi CLI. We no longer need this to build coverage-instrumented binaries. - ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH because the Pulumi binary no longer accepts a test.coverprofile flag. This information is now in the GOCOVERDIR environment variable. - We add an `enable-coverage` parameter to the `ci-build-binaries` workflow to mirror some of the other workflows. It will produce coverage-instrumented binaries if this is true. These binaries are then used by `ci-run-test` which will set `GOCOVERDIR` and merge the coverage results from it. - Coverage configuration no longer counts tests, testdata, and Protobuf-generated code against coverage. - go-wrapper.sh: Because we're no longer relying on the `go test -c` hack, this no longer excludes Windows and language providers from coverage tracking. - go-test.py and go-wrapper.sh will include pulumi-language-go and pulumi-language-nodejs in covered packages. *Other changes* - go-test.py: Fixed a bug where `args` parameters added for coverage were ignored. Note that this change DOES NOT track coverage for calls made to Pulumi packages by plugins downloaded from external sources, e.g. provider plugins. Arguably, that's out of scope of coverage trackcing for the Pulumi repository. Resolves #8615, #11419
2023-06-27 16:57:36 +00:00
;;
*)
echo "unknown build mode: $MODE"
exit 1
;;
esac
;;
install)
echo "install command is not supported, please use build"
exit 1
;;
*)
go "$@"
;;
esac