Commit Graph

34 Commits

Author SHA1 Message Date
Fraser Waters 2840708bb2
Use a new PULUMI_HOME for every test environment ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Should help with issues like
https://github.com/pulumi/pulumi/issues/15240, and other cases of test
case contamination.

Run every new `testing.Environment` with a new PULUMI_HOME.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-01 21:32:49 +00:00
Julien P 5ff35273d6
Fix merge failures ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15528

See https://github.com/pulumi/pulumi/pull/15540 &
https://github.com/pulumi/pulumi/pull/15531

Re-creating this as a PR with `ci/test` label so we can get it merged.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-29 21:06:24 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Thomas Gummerer 13371f9865
Start policy packs in parallel ()
Currently we start up and thus compile policy packs one by one. When
multiple policy packs need to be loaded, this increases the start up
time substantially. In previous tests plugins took ~1-3s to start up, so
having multiple of these the time adds up quickly.

Loading them in parallel will help reduce the startup time here.

Fixes 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-20 14:08:32 +00:00
Zaid Ajaj 5cff77abf3 address comments: use os.UserHomeDir() and disable parallel tests 2023-06-26 13:53:58 +02:00
Zaid Ajaj c2133cafd5 Fixes resolving plugins with package get-schema and get-mapping when they are not yet installed in plugin cache 2023-06-26 11:21:38 +02:00
Fraser Waters baa04160ec Fix `stack rm` removing config files for the wrong project
Fixes https://github.com/pulumi/pulumi/issues/12246
2023-06-21 10:52:25 +01:00
Abhinav Gupta 42c1a2bf01
testing/Environment: Tolerate faillures in deleting the test directory
CI land attempts have started failing occasionally on Windows
with the following error:

```
=== FAIL: . TestLanguageImportSmoke/go (4.00s)
    environment.go:96: Created new test environment:  C:\Users\RUNNER~1\AppData\Local\Temp\test-env711178549
    smoke_test.go:129: Running command pulumi login --cloud-url file://C:/Users/RUNNER~1/AppData/Local/Temp/test-env711178549
    smoke_test.go:130: Running command pulumi new go --yes
    smoke_test.go:131: Running command pulumi import --yes random:index/randomId:RandomId identifier p-9hUg
    history_test.go:28:
        	Error Trace:	C:/a/pulumi/pulumi/sdk/go/common/testing/environment.go:132
        	            				C:/a/pulumi/pulumi/tests/history_test.go:28
        	            				C:/a/pulumi/pulumi/tests/smoke_test.go:132
        	Error:      	Received unexpected error:
        	            	remove C:\Users\RUNNER~1\AppData\Local\Temp\test-env711178549\project: The process cannot access the file because it is being used by another process.
        	Test:       	TestLanguageImportSmoke/go
        	Messages:   	cleaning up test directory "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\test-env711178549"
```

The code path that is encountering this issue
is hit after the integration test has already passed.
So we're failing tests that are otherwise green
because it's taking a little too long for the system
to mark the resource as free.

This makes make DeleteEnvironment more tolerant of these failures,
logging them without marking the test as failed.
2023-05-11 17:02:51 -07:00
Abhinav Gupta c72b29e442
sdk/common: Add testing/diagtest package
Adds a testing/diagtest package similar to
the previously added iotest package ()
which writes log messages to a testing.TB's log stream.

This makes tests less noisy and helps associate error messages
with the exact test cases they came from.

diagtest, unlike iotest, prefixes each message with stdout or stderr.
To reuse iotest.LogWriter for this, we add support to iotest.LogWriter
to specify an optional prefix string.
2023-03-13 12:48:24 -07:00
Abhinav Gupta 7803b0443d
sdk/go: Add common/testing/iotest package
Adds a new iotest subpackage to the common/testing package
and moves the testLogWriter used in a prior integration test
into this directory.

The new API is:

    package iotest

    func LogWriter(testing.TB) io.Writer

This is a generally useful utility in any test that needs an io.Writer,
where we want them to not pollute actual stdout or stderr.
2023-03-07 11:27:47 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Abhinav Gupta 0bff0b8716 sdk/go: Remove 'nolint' directives from package docs
Go treats comments that match the following regex as directives.

    //[a-z0-9]+:[a-z0-9]

Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)

Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.

    // nolint:foo
    //nolint: foo

This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.

Verification:
Here's the output of `go doc` on some entities
before and after this change.

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

nolint: lll, interfacer

nolint: lll, interfacer

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```

After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine. nolint: revive
```

After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine.
```

Note that golangci-lint offers a 'nolintlint'  linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.

Resolves 

Related: https://github.com/golangci/golangci-lint/issues/892

[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
  tee "$FILES" |
  xargs perl -p -i -e '
    s|// nolint|//nolint|g;
    s|//nolint: |//nolint:|g;
  '
rg '.go$' < "$FILES" | xargs gofmt -w -s
2023-01-06 09:06:47 -08:00
杨成锴 7b09ad142a chore: Update doc comments, coding style, fix lint
* about error string: error string should not be capitalized or end with punctuation mark
* apply suggestions from code review
* lint: fix format error and simplify the code

Co-authored-by: Fraser Waters <frassle@gmail.com>
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-10-13 13:50:49 -07:00
Aaron Friel a4b1d6b2a7 ci: gofmt 1.18+ clean 2022-09-21 09:48:39 -07:00
Aaron Friel 7bef1b717e ci: Prevent yarn mutex timeout by single-threading yarn commands 2022-09-14 10:07:21 -07:00
Fraser Waters 109ade9b3d
Terminate test if RunCommand fails ()
* Terminate test if RunCommand fails

* Use Fatalf
2022-08-17 18:01:15 -04:00
svangordon-fruit 51406fc771
refactor: append to env var list instead of overwriting. () 2022-07-28 08:39:31 +01:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Justin Van Patten 6c7a330051
Fix passphrase secrets provider prompting ()
In , caching was introduced inside the passphrase secrets provider to avoid prompting the user for a password multiple times in the same `pulumi` command. However, the cache is not used in all cases, which leads to double prompting and/or erroring if the passphrase is not set in the `PULUMI_CONFIG_PASSPHRASE` envvar. In some cases, the user is prompted for their passphrase and then the command fails because the passphrase wasn't set via the envvar.

This change addresses these issues. The net effect is that when using the passphrase secrets provider, you'll be prompted for your passphrase up-front once (if interactive and the envvar isn't set), and subsequent uses will use the cached value and won't prompt or error if the envvar isn't set.

Implementation note: This moves the prompting code inside the `passphrase` package. I was originally trying to avoid this, to avoid the package depending on `cmdutil.ReadConsoleNoEcho` and `cmdutil.Interactive`, but this lead to unnatural contortions elsewhere. It ended up being much cleaner to have the prompting happen inside the passphrase implementation. Functions that may prompt (when interactive and the envvar isn't set) include "Prompt" in their name.
2022-02-16 13:39:08 -08:00
Anton Tayanovskyy 76ba788985
Toward replacing MSBuild with make+bash on Windows ()
* Add coverage folder

* Adjust GHA to run Test target on Windows

* Extend error message on packages not found

* Try with path normalizer

* Edit Makefiles

* Skip SDK codegen tests on Windows

* Skip program gen tests on Windows

* Skip tests that started running on Windows and failing

* Fix non-compiling test

* Skip failing Windows HCL2 tests

* Fix lint

* Merged constants

* Skip one more test failing on Windows

* Disable cov-enabled builds on Windows

* Fix  TestDeterminePulumiPackages on Windows

* Fix TestInstallCleansOldFiles test on Windows

* Fix TestCreatingProjectWithPulumiBackendURL on Windows

* Weaken TestAbbreviateFilePath to pass on Windows

* Fix TestDepRootCalc on Windows

* Fix LocalUrl() to be sensible on Windows

* Fix Go lint issue

* Windows fix for TestComments

* Cross-ref skip tracking issue

* Cross-ref issue to fix asset_test skips

* More cross-ref issues

* Use choco not chocolatey

* Use yarn run to ensure the right tools are selected

* Revert python3->python change in common.mk

* In CI context, use python not python3

* More randomness in temp folder names to aoid Windows collisions

* Go lint
2022-01-07 22:27:14 -05:00
Ian Wahbe aa2368d3c0
Give user defined variables precedence in GetCommandResults ()
* Give user defined variables precedence

* Update CHANGELOG_PENDING.md

* Add unit test to encode desired behavior
2022-01-05 11:03:29 -08:00
Justin Van Patten f989e1cc9d
Update pkg -> sdk dependency () 2021-11-03 11:25:20 -07:00
Ian Wahbe 27fc39c00a Fix spelling of environment 2021-08-31 00:54:07 -07:00
Ian Wahbe 59f88030b7 Allow failure to cleanup 2021-08-31 00:25:50 -07:00
Ian Wahbe 8150ef0880 Improve CI error messages
Failures unique to windows require a more informative error message to
diagnose.
2021-08-30 12:55:30 -07:00
Pat Gavlin 2cc89defbc
Read passphrase from the terminal when rotating. ()
Rotating a passphrase requires that the old passphrase is available via
one of the `PULUMI_CONFIG_PASSPHRASE` or `PULUMI_CONFIG_PASSPHRASE_FILE`
environment variables. This confuses `readPassphrase` when reading a new
passphrase, since that function checks the aforementioned environment
variables prior to reading from the console. The overall effect is that
it is impossible to rotate the passphrase for a stack using the
passphrase provider. These changes fix this by always reading from the
console when rotating a passphrase.
2021-06-22 11:13:57 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Kraig Amador 71ec66aa45
Support locking on filestate logins ()
When using the filestate backend (local files and cloud buckets) there is no protection to prevent two processes from managing the same stack simultaneously.

This PR creates a locks directory in the management directory that stores lock files for a stack. Each backend implementation gets its own UUID that is joined with the stack name. The feature is currently available behind the `PULUMI_SELF_MANAGED_STATE_LOCKING=1` environment variable flag.
2021-03-16 14:00:47 +11:00
Luke Hoban 4ecd8f9f56
Add PULUMI_BACKEND_URL env var ()
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend.  This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.

This is especially useful for automation scenarios like Automation API generally (and effectively solves https://github.com/pulumi/pulumi/issues/5591), or https://github.com/pulumi/pulumi-kubernetes-operator/issues/83 specifically.

This also makes things like efe7a599e6/dist/actions/entrypoint.sh (L10) less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
2020-11-22 15:27:59 -08:00
Pat Gavlin 3e6e28af36
Fix lint errors. ()
These errors appear when using golangci-lint 1.27.0.
2020-06-18 14:17:11 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle 6933745342 move pkg/tools -> sdk/go/common/tools 2020-03-18 15:56:58 -07:00
evanboyle a4ec3ec81b move pkg/testing -> sdk/go/common/testing, leave behind pkg/testing/integration 2020-03-18 15:55:41 -07:00