Commit Graph

295 Commits

Author SHA1 Message Date
Julien a3c0ea5455
Add NodeJS test for parameterized providers ()
This takes https://github.com/pulumi/pulumi/pull/16392 and updates local
SDK generation for nodejs and adds parametrisation support for invokes

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-08-09 08:34:29 +00:00
Anton Tayanovskyy 603ce54bc1
refactor: make ProgramTest inline better ()
With this change users can inline ProgramTest call and get back the
sequence with ProgramTestManualLifeCycle to edit further, making it more
obvious how to customize this.
2024-07-26 16:56:53 +00:00
Florian Stadler 06e288d7b5
Clean up PULUMI_HOME after program test ()
Clean up the temporary `PULUMI_HOME` directory we create during a
program test.
This is necessary to reclaim the disk space of the plugins that were
downloaded
during the test.

In pulumi-aws we started seeing test failures because the CI runners
started running out of disk space due to plugins in `PULUMI_HOME` not
being cleaned up.
2024-07-15 13:03:27 +00:00
Julien P d79644f06e
Detect pnpm workspaces ()
Detect pnpm workspaces and run `pulumi install` from the workspace root.
Note that this change does not address issues with function
serialisation in pnpm workspaces.

Fixes https://github.com/pulumi/pulumi/issues/15512
2024-06-18 13:37:06 +00:00
Ian Wahbe 1a538e7466
Allow missing stack config and bookkeeping files ()
Pulumi isn't required to write out these files, so valid tests fail when
it doesn't. Previously this behavior wasn't a problem because the return
values from fsutil.CopyFile were not checked. This regression was
changed in https://github.com/pulumi/pulumi/pull/14695.

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [X] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-05 11:57:00 +00:00
Thomas Gummerer f831cb942b
set PULUMI_HOME in ProgramTests ()
Currently program test runs all programs with PULUMI_HOME unset. This
means that if programs run in parallel they share the same directory.
E.g. for login this means running in parallel only works if we always
log into the same backend, and there is no racyness between tests.

Set PULUMI_HOME to a temp directory, so each program test has its own
setup.

Fixes https://github.com/pulumi/pulumi/issues/15240

This complements https://github.com/pulumi/pulumi/pull/15559
2024-03-04 09:06:56 +00:00
Julien P 5ff35273d6
Fix merge failures ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15528

See https://github.com/pulumi/pulumi/pull/15540 &
https://github.com/pulumi/pulumi/pull/15531

Re-creating this as a PR with `ci/test` label so we can get it merged.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-29 21:06:24 +00:00
Julien P 87b01bf35a
Handle workspaces in computeCodePaths ()
# Description

When we compute the code paths for function serialisation, we are not
handling yarn/npm workspace setups where the node_modules might be in a
different directory.

Fixes 
Fixes https://github.com/pulumi/pulumi/issues/2661
Fixes https://github.com/pulumi/pulumi/issues/7168
Fixes https://github.com/pulumi/pulumi/issues/3013

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-22 11:41:37 +00:00
Thomas Gummerer 14e78f5598
Enable program test to install dev releases of SDKs ()
# Description

Add a new option `InstallDevReleases` to `ProgramTestOptions`, which
will tell ProgramTest to upgrade each pulumi dependency to the latest
dev release. This includes the pulumi/pulumi SDKs and the provider SDKs.
The idea is to use this in the `pulumi/examples` and `pulumi/templates`
repositories to get pre release testing there.

Fixes https://github.com/pulumi/pulumi/issues/15389

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 13:07:29 +00:00
Mikhail Shilkov 55edc30d68
Add RequireEmptyPreviewAfterRefresh option to ProgramTest ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

While working on [this issue in Azure
Native](https://github.com/pulumi/pulumi-azure-native/issues/2798), I
realized that I don't see how to test the scenario that I care about. I
need the following workflow:

1. Run `pulumi up` for the initial deployment.
2. Run `pulumi refresh` to pull changes from API.
3. Run `pulumi preview --expect-no-changes` to guarantee that there is
no loop of `refresh/up` stepping over each other.

The existing `ExpectRefreshChanges` option is different. If it's set to
false, the test would require no changes on step (2) above, which is a
stronger requirement. Unfortunately, it's not always true in Azure
Native, e.g. when a subresource changes properties of its parent
resource. I want to enable "ExpectRefreshChanges: false" where possible,
but I also want to require step (3) to succeed for all tests in Azure
Native (and potentially everywhere, eventually). I've already found
three issues while drafting that.

Therefore, this PR proposes a new option for ProgramTest:
`RequireEmptyPreviewAfterRefresh`. It defaults to false, so no behavior
changes unless users opt in. If set to true, it runs an extra preview
after refreshing, i.e. the step (3).

Let me know if I missed a better way of achieving the same.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-31 09:17:37 +00:00
Fraser Waters afb287d2fb
Rename filestate to DIY ()
This goes through the codebase to try and be consistent about names for
the diy/filestate/local/selfmanaged backend. Every reference to this
backend should now use the terms "DIY". There are a couple of places
that still say "local DIY backend" this is referring to a DIY backend
using the local filesystem (i.e. `pulumi login --local`).
2024-01-30 15:53:10 +00:00
Justin Van Patten 92abd1a9ef
ProgramTest: Python: Use correct venv path for projects with `main` option ()
In preparation for supporting Python 3.12...

I'm planning to remove `pulumi` from `requirements.txt` in a bunch of
the integration tests (at least temporarily) because the current
published version of `pulumi` can't be installed on Python 3.12 due to
its dependency on `grpcio` which is broken on Python 3.12, and the tests
specify a dependency on the locally built Pulumi Python SDK via
`ProgramTestOptions.Dependencies`.

However, after removing `pulumi` from `requirements.txt` for
`TestPythonStackTruncate/main_dir_specified`, the test started to fail
due to a missing `pulumi` package. It shouldn't fail because the test
additionally requires the locally built SDK via
`ProgramTestOptions.Dependencies`.

The reason for the failure is because this test had a `main` specified
in the project to a sub directory (`bar/`). Because of this, the venv
created by `ProgramTest` is inside the sub directory. It would install
the `requirements.txt` in that venv, and then install any
`ProgramTestOptions.Dependencies` deps. Then it would record the
`virtualenv` option in the project as just `venv`. But this option is
relative to the project file and a `venv` directory next to the project
doesn't exist -- it exists in the sub directory.

So when the first `pulumi` operation was run, `pulumi` would see that
the `venv` directory next to the project doesn't exist, and create it,
installing the dependencies from `requirements.txt` in it and carrying
on.

With the change to remove `pulumi` from `requirements.txt` for this
test, this would no longer work. When `pulumi` created the venv, it'd
install an empty `requirements.txt` in the venv it created. And since it
didn't have a `pulumi` package, it fails with a missing module.

This change fixes `ProgramTest` to record the correct path to the venv
in the project file. In most cases, this continues to be just `venv`.
But if `main` is specified in the project such that the "cwd" path is
different from the project file root, then that path is used for the
venv. This way, the CLI will use the correct venv that includes
dependencies from `ProgramTestOptions.Dependencies`.
2024-01-23 23:21:02 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Eron Wright e448b3a6d5
[ProgramTest] Fix for PostPrepareProject option ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description
This PR fixes a small problem with ProgramTest, that the `With` function
doesn't propagate the `PostPrepareProject` option as it should. e.g.

```go
	options := baseOptions.With(integration.ProgramTestOptions{
		PostPrepareProject: func(p *engine.Projinfo) error {
			// ...
		},
	})
```

A follow-up to: https://github.com/pulumi/pulumi/pull/12302

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-04 22:49:38 +00:00
Fraser Waters bbaf871ad1
Check qualified type for root stackness ()
With the introduction of component programs there will be mulitple
"pulumi:pulumi:stack" resources in a program. We should therefore check
against the qualified type of resources to see if they are the root
stack, not just their direct type.
2023-12-04 10:36:51 +00:00
Fraser Waters 2c74dddc91
Switch to use env.Env in filestate ()
Internal refactor to use `env.Env` directly in filestate rather than
mocking `os.Getenv`.
2023-10-18 10:52:54 +00:00
Pat Gavlin 6ac2078d35
[environments] Add integration tests ()
- Add support for environments to the integration test framework
- Add integration tests for simple + merged environments
2023-10-12 16:39:26 +00:00
Eron Wright f13cb785f1
[Test] Eager cancelation for ProgramTester ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This PR changes the behavior of `ProgramTester` in
`pkg/testing/integration` package, to detect test failures more eagerly
and then to skip remaining steps. Specifically it checks whether the
test been marked as failed by a validation function (as defined by
[`t.Failed()`](https://pkg.go.dev/testing#T.Failed)).

The rationale is that, for most integration tests, each step assumes
that the previous step was successful. It is simply noisy to continue
with the subsequent steps. A workaround is to call `t.FailNow()` in the
validation function (or use `require`).

An option is provided `ExpectTestFailure` to continue with steps after
test failure (as is the current behavior).

There exists an option `ExpectFailures` to continue with steps after a
pulumi deployment failure. I took such failures to be orthogonal to test
failure.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

## Example
Here's a snippet of output in the event that a validation function marks
the test as failed (e.g. using an assertion).
```
    ...
    /Users/eronwright/Pulumi/pulumi/tests/examples/program.go:1742: Performing extra runtime validation.
    /Users/eronwright/Pulumi/pulumi/tests/examples/examples_test.go:44:
                Error Trace:    ...
                Error:          Expected nil, but got ...
                Test:           TestAccMinimal
    /Users/eronwright/Pulumi/pulumi/tests/examples/program.go:1744: Extra runtime validation complete.
    /Users/eronwright/Pulumi/pulumi/tests/examples/program.go:1164: Canceling further steps due to test failure
    /Users/eronwright/Pulumi/pulumi/tests/examples/program.go:1349: Destroying stack
    ...
    /Users/eronwright/Pulumi/pulumi/tests/examples/program.go:1365: Test failed, retaining stack '...'
--- FAIL: TestAccMinimal (5.61s)
```
2023-10-11 08:07:44 +00:00
Fraser Waters 156a89e611
Add SelfManaged env vars ()
Tiny fix up to env vars. This wasn't showing in `pulumi env`.
2023-09-25 14:07:46 +00:00
Abhinav Gupta 6af5f0b39d
ci: Track code coverage
**Overview**

This re-enables tracking of code coverage.
For Go, there are two kinds of coverage at play:
unit test and integration test coverage.

Unit tests follow the usual pattern of running
`go test -cover -coverprofile=whatever.cov`.

For integration tests, we use the new integration test profiling support
[added in Go 1.20](https://go.dev/testing/coverage/).
In short, the way it works is:

    # Build a coverage instrumented binary:
    go build -cover

    # Set GOCOVERDIR to a directory and run the integration tests
    # that will invoke this coverage-instrumented binary.
    GOCOVERDIR=$(pwd)/coverage
    go test ./tests

    # $GOCOVERDIR will now be filled with coverage data
    # from every invocation of the coverage-instrumented binary.
    # Combine it into a single coverage file:
    go tool covdata textfmt -i=$(GOCOVERDIR) -o=out.cov

    # The resulting file can be uploaded to codecov as-is.

The above replaces the prior, partially working hacks we had in place
to get coverage-instrumented binaries with `go test -c`
and hijacking the TestMain.

**Notable changes**

- TestMain hijacking is deleted from the Pulumi CLI.
  We no longer need this to build coverage-instrumented binaries.
- ProgramTest no longer tracks or passes PULUMI_TEST_COVERAGE_PATH
  because the Pulumi binary no longer accepts a test.coverprofile flag.
  This information is now in the GOCOVERDIR environment variable.
- We add an `enable-coverage` parameter to the `ci-build-binaries`
  workflow to mirror some of the other workflows.
  It will produce coverage-instrumented binaries if this is true.
  These binaries are then used by `ci-run-test` which will set
  `GOCOVERDIR` and merge the coverage results from it.
- Coverage configuration no longer counts tests, testdata,
  and Protobuf-generated code against coverage.
- go-wrapper.sh:
  Because we're no longer relying on the `go test -c` hack,
  this no longer excludes Windows and language providers
  from coverage tracking.
- go-test.py and go-wrapper.sh will include pulumi-language-go and
  pulumi-language-nodejs in covered packages.

*Other changes*

- go-test.py:
  Fixed a bug where `args` parameters added for coverage were ignored.

Note that this change DOES NOT track coverage for calls made to Pulumi
packages by plugins downloaded from external sources,
e.g. provider plugins. Arguably, that's out of scope of coverage
trackcing for the Pulumi repository.

Resolves , 
2023-06-28 13:30:13 -07:00
Abhinav Gupta 795a3d07be
fix(cli/state rename): Update parent references on rename
If a resource is renamed with `pulumi state rename`,
in addition to updating references in resources that refer to it
as a dependency,
also update references to the resource as a parent of another resource.

Testing:
Includes an integration test that reproduces the panic,
and a unit test that verifies the new behavior at a more isolated level.
To implement the integration test,
a new RunPulumiCommand method was added to ProgramTester
because we don't have other means of running `pulumi state rename`
with ProgramTest.

Resolves 
2023-06-16 13:46:35 -07:00
Robbie McKinstry 5c24b32429
Fix failure to load Pulumi.yaml when RelativeWorkDir provided.
This commit fixes a bug where integration tests specifying
RelativeWorkDir would not load because the path to Pulumi.yaml was not
applying the RelativeWorkDir parameter when searching for the file.
2023-04-26 16:14:46 -04:00
bors[bot] 8a8ad0d1c7
Merge
12699: test(go): Integration test for MLC options r=abhinav a=abhinav

Adds a minimal integration test that verifies
options passed to an MLC in Go are propagated properly.

Currently, two options are checked: DependsOn and Protect.
Other options will be added to this test as part of .

To aid in the `Protect` test,
this also adds an option to ProgramTest to set the `--exclude-protected` flag
when running `pulumi destroy` at the end of the run.

This will allow integration tests that create protected resources
without going back and unprotecting them for cleanup.


12713: refactor(go/host): Extract global state into a struct r=abhinav a=abhinav

Refactors the Go language host plugin
to extract global state (e.g. stdout, command line args)
into a mainCmd struct.
The command line parsing logic is placed into a separate,
independently tested function.

This change does not modify any behavior.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-04-21 17:04:27 +00:00
Abhinav Gupta 0c3584611b
testing/ProgramTest: Don't always fail write package.json
The function `writePackageJSON` used by ProgramTest
always returns a non-nil error--even when encode succeded
because it unconditionally wraps the result with an error.
2023-04-18 13:27:03 -07:00
Abhinav Gupta 6b41a51e3f
testing/ProgramTest: Support exclude-protected at cleanup
Adds an option to ProgramTest to set the `--exclude-protected` flag
when running `pulumi destroy` at the end of the run.

This will allow integration tests that create protected resources
without going back and unprotecting them.
2023-04-18 13:20:43 -07:00
Abhinav Gupta e395deef6b
all: Assert => Assertf
Migrates all remaining usages of
`contract.Assert*` and `contract.Require*` to the f variants,
which require adding meaningful error messages.

There were a couple cases where a `testing.T` or `testing.B`
was already available.
For those, this uses t.FailNow or require.NoError.

Refs 
2023-03-03 14:37:43 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta cc32691bc1
chore(all): strings.Replace(..., -1) => strings.Replace(...)
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
2023-03-01 13:22:33 -08:00
Fraser Waters 48ec2c6336 Fix require reading JSON files 2023-02-28 21:24:04 +00:00
Aaron Friel 8eb9bc7c0e feat: enable simpler Go programtest replacements
This change to ProgramTest allows Go module replacements to omit the major
version number, simplifying the process to release a provider with a new major
version.

When the major version is incremented in a PR, the examples fail to compile if
they reference the incorrect SDK version.

This enables the examples to be written to target a versionless `/sdk`, and have
a Dependencies line such as:

```go
	integration.ProgramTestOptions{
		Dependencies: []string{
			"github.com/pulumi/pulumi-keycloak/sdk=../sdk",
		},
	}
```

This versionless import mapping means that examples run regardless of the major
version of the provider, and updates to major versions in PRs will require only
one PR.

A provider PR proving out the implementation was created against this commit
here:
- https://github.com/pulumi/pulumi-keycloak/pull/165
2023-01-27 00:19:43 -08:00
Fraser Waters cc077b247e Use plugin link and shimless for testprovider 2023-01-19 12:41:19 +00:00
Abhinav Gupta 65194ecdf7
pkg: Fix unnecessary appends
Removes unnecessary appends in pkg.
There were two instances:

- append into a slice that's never used again
- append with nothing to append (`append([]{foo}) == []{foo}`)

Issue found by staticcheck:

```
testing/integration/program.go:2190:15: SA4021: x = append(y) is equivalent to x = y (staticcheck)
testing/integration/program.go:2192:15: SA4021: x = append(y) is equivalent to x = y (staticcheck)
codegen/pcl/utilities.go:158:11: SA4010: this result of append is never used, except maybe in other appends (staticcheck)
```

Refs 
2023-01-13 12:27:54 -08:00
Abhinav Gupta 8280296699
gosimple: printf, time.Since, TrimPrefix, etc.
Fix a bunch of other gosimple issues:

- `Printf` used for static string
- `Skip(Printf(..))` => `Skipf(..)`
- `time.Now().Sub(t)` => `time.Since(t)`
- `HasPrefix; TrimPrefix` => `TrimPrefix`
- Unnecessary type casts
- `IndexRune(..) != 1` => `ContainsRune`
- `if cond { return true } else { return false }` => `return cond`
2023-01-12 09:55:34 -08:00
Abhinav Gupta e845ddf4c4
gosimple: Simplify loops
This replaces for loops and slice appends reported by gosimple
with simpler variants.

Specifically,

    for _, x := range src {
        dst = append(dst, x)
    }
    // can be replaced with
    dst = append(dst, src...)

And,

    for i, x := range src {
        dst[i] = x
    }
    // can be replaced with
    copy(dst, src)

And,

    for true { ... }
    // can be replaced with
    for { ... }

And, given a string `s`,

    for _, r := range []rune(s) { .. }
    // can be replaced with
    for _, r := range s { .. }

Lastly, this fixes in ineffective break statement
also reported by the linter.
Inside a switch block,
`break` affects the current `case` only.
The outer loop needs a label.
2023-01-12 09:55:34 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Aaron Friel 6642fb5f1a Improve ProgramTest errors and path handling for Go programs 2022-12-13 06:53:09 -08:00
bors[bot] f6d669c9e7
Merge
11524: test: use T.TempDir to create temporary test directory r=Frassle a=Juneezee

# Description

A testing cleanup. 

This pull request replaces `ioutil.TempDir` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. 

Reference: https://pkg.go.dev/testing#T.TempDir

```go
func TestFoo(t *testing.T) {
	// before
	tmpDir, err := ioutil.TempDir("", "")
	assert.NoError(t, err)
	defer os.RemoveAll(tmpDir)

	// now
	tmpDir := t.TempDir()
}
```

## Checklist

- [ ] ~~I have added tests that prove my fix is effective or that my feature works~~: Tests refactoring only, no new features added.
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] ~~I have updated the [CHANGELOG-PENDING](https://github.com/pulumi/pulumi/blob/master/CHANGELOG_PENDING.md) file with my change~~: This PR is a non user-facing change.
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] ~~Yes, there are changes in this PR that warrants bumping the Pulumi Service API version~~: N/A
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


11586: ci: Fix extraneous alpha draft releases r=AaronFriel a=AaronFriel

The [PR workflow to update Pulumi YAML to 1.0.4](https://github.com/pulumi/pulumi/actions/runs/3642973677) shows that it created a draft release, and this has been the case for all contributors' PRs.

This fixes the extra alpha releases created as a result of pull requests. The job "prepare-release" shouldn't run unless the ci/test label is set, indicating an intent to evaluate the full CI pipeline.


Co-authored-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-12-08 17:57:51 +00:00
Kyle Pitzen 8e8dd2ce85 fix(testing): Adds cached python venvs for integration tests
The cache key is based on hashing the contents of "requirements.txt" for each
python test.  This should result in unchanged test results among tests
sharing venvs, and maintains environment isolation across requirement sets
2022-12-06 13:57:07 -05:00
Eng Zer Jun 48eff4676a
test: use T.TempDir to create temporary test directory
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-12-03 15:17:08 +08:00
Aaron Friel 3312682ad9 feat(programtest): Enable ProgramTest for Azure-Native split modules 2022-10-16 21:43:43 -07:00
Aaron Friel 41333f1a2a ci: Refactor options setup & constructor 2022-10-11 10:18:14 -07:00
Aaron Friel 8105a4d29a ci: Unblock local ProgramTest usage 2022-10-11 10:16:25 -07:00
Ian Wahbe ede3b5598c Lint 2022-10-04 16:48:22 -07:00
Ian Wahbe 76a6ace4a6 Add expectFailure to another case 2022-10-04 14:33:07 -07:00
Aaron Friel bc6446815a ci: Remove -compat=1.18 flags to validate that tests pass on current and minimum supported Go versions 2022-09-27 10:41:25 -07:00
Fraser Waters b91dc7b303 Fix mutation of workspace.Project 2022-09-22 15:58:57 +01:00
Fraser Waters 9a7ab41347 Merge remote-tracking branch 'origin/master' into useLocalDeps 2022-09-22 15:16:05 +01:00
Aaron Friel 7e555cb6ab ci: Simplify test listing, update go dependencies to 1.18 compat 2022-09-21 09:51:59 -07:00
Aaron Friel a4b1d6b2a7 ci: gofmt 1.18+ clean 2022-09-21 09:48:39 -07:00