The `Resource` class in the Node.js SDK has the following internal
property:
```typescript
/** @internal */
readonly __prov?: ProviderResource;
```
When a resource is created, the provider specified for the resource is
stored in this property. If it is set, it is passed along in the `Call`
request when a method is called on the resource.
Prior to #13282, the property was only set for custom resources in
`Resource`'s constructor:
```typescript
this.__prov = custom ? opts.provider : undefined;
```
With #13282, it was changed to also store the value for remote
components:
```diff
- this.__prov = custom ? opts.provider : undefined;
+ this.__prov = custom || remote ? opts.provider : undefined;
```
This regressed the behavior when calling a method on a remote component
that had an explicit provider that wasn't the component provider, but
some other provider (e.g. AWS provider) specified as:
```typescript
const component = new MyRemoteComponent("comp", {
}, { provider: awsProvider });
```
The `awsProvider` was being stored in `Resource.__prov`, and when making
the method call on the resource, it would try to invoke `Call` on the
AWS provider, rather than calling the remote component provider's
`Call`, which resulted in an error.
Note that specifying the AWS provider using the more verbose `providers:
[awsProvider]` works around the issue.
The fix is to only set `__prov` if the provider's package is the same as
the resource's package. Otherwise, don't set it, because the user is
specifying a provider with the `provider: awsProvider` syntax as
shorthand for `providers: [awsProvider]`.
Fixes#13777
This commit changes how the Node SDK resolves package.json to recursively look up the directory tree for the file when it's not present in the cwd. The previous behavior was to only look in the current directory, and fallback to an empty object when the file could not be found.
Fixes https://github.com/pulumi/pulumi/issues/12207
Added some tests for this, which should eventually become a matrix test
to ensure we check every language returns the expected error for this.
The `main` field in package.json is standardized by Node to refer
to a file relative to package.json. The previous commits in this
PR used `main` relative to the Pulumi main directory, which was
incorrect. This commit upgrades two tests, and introduces two more,
to verify the correctness of this behavior and the PR in general.
It appears that a large number of tests were using references
to main fields which don't currently exist. The runtime didn't
produce an error because package.json was being ignored.
This commit adds a test to assert Pulumi programs written as ESM
modules can be executed when their package.json file is located
in the parent directory.
This commit adds a failing test, which demonstrates
that the Pulumi NodeJS runtime cannot locate a package.json
file kept in a parent directory. It does not recursively walk
up the directory tree until it finds a package.json file.
Information in the original issue is a bit limited,
but we know that the error occured when the user
passed a `provider=x` to a multi-language component,
which then instantiate a resource handled by that provider
with itself as the parent.
The expected behavior was for the resource to use the explicit provider,
but instead, it used the default global provider.
The reproduction does roughly the same for Go, Python, and Node,
except it instantiates three resources:
1. using the default provider
2. using an explicit provider specified with `provider=x`
3. using an explicit provider specified with `providers=[x]`
3. using an explicit provider specified with `providers={pkg: x}`
Case (2) fails as expected from #12593 for Python.
```
--- PASS: TestConstructProviderPropagationGo (14.66s)
--- PASS: TestConstructProviderPropagationNode (15.03s)
--- FAIL: TestConstructProviderPropagationPython (17.63s)
```
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
After internal discussion, we determined "smoke" is a misleading
adjective for this category of tests. What we called "smoke tests"
are short integration tests for basic cross-platform functionality.
As a result, these are better named "acceptance" tests, since smoke
tests are intended to be a low water mark at the unit level to sniff
out bigger issues with the build as a whole.
This change fixes the `DeletedWith` resource option in the Go, Node.js,
and Python SDKs and adds tests.
This feature was a community contribution and while there were engine
tests included with the original PR, there weren't any tests confirming
the functionality worked correctly from each SDK.
Here's a summary of the fixes:
* Go: The `DeletedWith` resource option was never usable as it accepted
a URN instead of a Resource. We discussed this internally a while back
and decided to go ahead and fix this. (Note: While changing the
signature is technically a breaking change, the feature is currently
unusable, so the change would not break anyone, so there's no need to
wait for a major version bump.)
* Node.js: The `deletedWith` resource option did not work at all from
the Node.js SDK because it was incorrectly passing the resource object
itself in the RegisterResource request, rather than the resource's
URN.
* Python: The `deleted_with` resource option did not work at all from
the Python SDK because it was incorrectly passing the resource object
itself in the RegisterResource request, rather than the resource's
URN.
A `FailsOnDelete` resource has been added to the testprovider, which
will fail when its `Delete` gRPC is called. The tests use this to ensure
`Delete` is not called for resources of this type with the `DeletedWith`
option specified.
11250: ci: Move more tests to linux-only integration tests r=AaronFriel a=AaronFriel
Reduces the set of smoke tests run on Windows & macOS, balances the reduced parallelism #11249 by moving tests which don't seem to obviously entail cross-platform behavior changes. The tests moved to only run on Linux environments are listed below and except for the first, all were split from `integration_test.go`.
* `TestRefreshGo` checks that the Pulumi.yaml `options.refresh` option is obeyed (moved to integration_go_test from integration_go_smoke_test)
* `TestStackTagValidation` which tests Pulumi.yaml validation
* `TestStackInitValidation` which tests an error on running `pulumi stack init`
* `TestConfigPaths` which tests `pulumi config set` under a wide variety of inputs
* `TestDestroyStackRef` which tests running `pulumi destroy` on a stack with a large stack reference
* `TestJSONOutput` which tests output JSON output diagnostics
* `TestExcludeProtected` which tests behavior around marking resources protected
* `TestProviderDownloadURL` which tests the `PluginDownloadURL` resource option, but does not actually perform plugin acquisition.
These tests either didn't seem to be obviously platform specific or there was a narrower test could verify that parsing and handling the same files worked correctly. As examples of those, the remaining tests that will run on smoke tests (all platforms) are:
* `TestConfigSave` which tests `pulumi config set` and stack config parsing
* `TestRotatePassphrase` which tests changing the local secrets provider passphrase via stdin - which hypothetically could vary across platforms
* `TestJSONOutputWithStreamingPreview` tests streaming output and event capturing
* `TestPassphrasePrompting` tests stdin/stdout behavior in prompting for passphrases.
Lastly, the `TestProviderDownloadURL` test was modified to ensure we wouldn't make requests to uncontrolled third party URLs. Though the tests don't acquire plugins, we don't control "get.com", so the `.test` TLD is used to ensure test behavior cannot depend on that domain.
11296: Update YAML to 1.0.2 r=AaronFriel a=AaronFriel
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
10687: add envvar to omit snapshot writes on each state mutation r=dixler a=dixler
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Fixes#10668
Provides an environment variable to override the internal behavior of the SnapshotManager to speed up large deployments by only writing the final state of the snapshot rather than the current behavior which saves every mutation to the snapshot to the state backend.
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have updated the [CHANGELOG-PENDING](https://github.com/pulumi/pulumi/blob/master/CHANGELOG_PENDING.md) file with my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
10729: ci: Fix package parallelism assignment r=AaronFriel a=AaronFriel
10740: Add missing `ProgramTestOptions` overrides in `With` r=justinvp a=justinvp
These options were previously added without also adding the override handling in `With`.
Co-authored-by: Kyle Dixler <kyle@pulumi.com>
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
10720: ci: Enable programtests against local backend, improve isolation r=AaronFriel a=AaronFriel
These changes make integration tests run much more quickly on a local dev loop, for easily parallelized program tests like in the `integration_go_test.go` file the improvement was on the order of minutes across all tests.
The changes rely on setting `PULUMI_BACKEND_URL` to override the backend during particular tests. The backend is set to a temporary directory which is cleaned up on exit.
A helper function `NewBackendUrl(t *testing.T)` is added to enable
When `PULUMI_TEST_USE_SERVICE=true`, the `RequireService` option is set to true.
When `RequireService == true`, the test is skipped if an access token is not present, improving local dev experience by skipping tests which would error very loudly.
When `RequireService == false and CloudURL == ""`, then we use the helper function to create a temporary directory and point the filestate backend to it.
The CloudURL check allows tests which, even in the presence of `PULUMI_TEST_USE_SERVICE=true`, to still run against a local backend. E.g.:
```go
localTestOptions := testOptions.With(integration.ProgramTestOptions{
CloudURL: integration.NewBackendURL(t),
})
```
10734: prepare for next release (v3.41.0) r=AaronFriel a=pulumi-bot
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
Co-authored-by: github-actions <github-actions@github.com>
During Invoke, secrets are stripped and returned back in cleartext. If not specified in the MarshalProperties struct the default value for KeepSecrets(a boolean) is false. This PR keeps them encrypted when returning them.