<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new experimental feature to the NodeJS SDK to register
remote transform functions. These are currently all prefixed 'X' to show
they're experimental.
These transform functions will run even for resources created inside
MLCs.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
We need to synchronously check for transform support in
`registerStackTransformation` and resource constructors when adding
remote transform support to node (c.f.
https://github.com/pulumi/pulumi/pull/14303). This change changes all
the feature checks to be done at startup and then accessed via just a
field lookup. Adding the "transform" feature to this set is clearly
simple.
Sadly there's no single entry point to make these changes in. So we need
to update entry point of construct/call, the entry point of programs,
and test setup. Miss any one of these and current tests start failing.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Updates the node SDK to have types for the generated protobufs and grpc
services. No more `any`.
I don't think these types show up on any user facing parts of the SDK.
The two places they could show up are the mock monitor (but I've left
that as any for now) and the provider interfaces (which I've also left
alone for now).
But this fixes up most of our sdk internal code to have types, which
flagged a couple of oddities like assuming `Input<string>` was `string`
in the alias code, and a whole load of missing undefined checks.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#11719
NodeJS now accepts unknown resource IDs for read resource.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Add support to the core SDKs for reporting resource source positions.
In each SDK, this is implemented by crawling the stack when a resource
is registered in order to determine the position of the user code that
registered the resource.
This is somewhat brittle in that it expects a call stack of the form:
- Resource class constructor
- abstract Resource subclass constructor
- concrete Resource subclass constructor
- user code
This stack reflects the expected class hierarchy of "cloud resource /
component resource < customresource/componentresource < resource".
For example, consider the AWS S3 Bucket resource. When user code
instantiates a Bucket, the stack will look like
this in NodeJS:
new Resource (/path/to/resource.ts:123:45)
new CustomResource (/path/to/resource.ts:678:90)
new Bucket (/path/to/bucket.ts:987:65)
<user code> (/path/to/index.ts:4:3)
In order to determine the source position, we locate the fourth frame
(the `<user code>` frame).
13070: fix uncatchable `Resource monitor is terminating` r=abhinav a=dixler
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/10744Fixes#12970
Allows users to catch `Resource monitor is terminating` exceptions from automation API.
When using Node automation API, and `registerResource()` fails due to the resource monitor shutting down, an uncaught rejection is thrown by a `debuggablePromise` and causes the process to exit without allowing users to handle the exception.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Kyle Dixler <kyle@pulumi.com>
A failed update can lead to an unavailable resource monitor creating an
uncaught rejected promise which makes Automation API exit prematurely
since the exception was uncatchable. Prints the error message to stderr
only once to avoid spamming the user.
Fixes#10744Fixes#12970
This change updates the engine to detect if a `RegisterResource` request
is coming from an older Node.js SDK that is using incorrect alias specs
and, if so, transforms the aliases to be correct. This allows us to
maintain compatibility for users who have upgraded their CLI but are
still using an older version of the Node.js SDK with incorrect alias
specs.
We detect if the request is from a Node.js SDK by looking at the gRPC
request's metadata headers, specifically looking at the "pulumi-runtime"
and "user-agent" headers.
First, if the request has a "pulumi-runtime" header with a value of
"nodejs", we know it's coming from the Node.js plugin. The Node.js
language plugin proxies gRPC calls from the Node.js SDK to the resource
monitor and the proxy now sets the "pulumi-runtime" header to "nodejs"
for `RegisterResource` calls.
Second, if the request has a "user-agent" header that starts with
"grpc-node-js/", we know it's coming from the Node.js SDK. This is the
case for inline programs in the automation API, which connects directly
to the resource monitor, rather than going through the language plugin's
proxy.
We can't just look at "user-agent", because in the proxy case it will
have a Go-specific "user-agent".
Updated Node.js SDKs set a new `aliasSpecs` field on the
`RegisterResource` request, which indicates that the alias specs are
correct, and no transforms are needed.
The Node.js SDK is not sending the right `Parent` and `NoParent` alias
spec.
117955ce14/sdk/nodejs/runtime/resource.ts (L332-L361)
- It does not set `NoParent` when it should, but instead sets `Parent`
to empty.
- It sets `NoParent` to true and leaves `Parent` empty when both the
alias and resource have no `Parent` specified.
- If the resource has a parent and `Parent` isn't set on the alias, it
sends the parent's URN for the `Parent`.
This commit fixes these issues so that the Node.js SDK sends correctly
specified alias specs.
However, we continue to set the alias's `Parent` to the resource's
parent if the alias does not specify a `Parent`. We shouldn't have to do
this, but doing it allows us to maintain backwards compatibility in the
case when the Node.js SDK is upgraded, but not the CLI.
This commit removes instances of prefix-!! where the value
would already be coerced into a boolean. It also removes
instances where the "optional chaining" operator (?.)
would suffice.
This commit applies the Rome autoformatter to the Node SDK.
These changes are automatically produced. To reproduce these
changes, run `make format` from inside sdk/nodejs.
When a resource depends on a local component resource, rather than setting the component resource itself as a dependency, each of the component's descendants is added as a dependency. This can lead to hangs when cycles are introduced.
For example, consider the following parent/child hierarchy, where `ComponentA` is the parent of `ComponentB` and `ComponentB` is the parent of `CustomC`:
```
ComponentA
|
ComponentB
|
CustomC
```
If `ComponentB` specifies it has a dependency on `ComponentA`, the following takes place as part determining the full set of transitive dependencies:
1. `ComponentA` is a component resource so it isn't added as a dependency, its children are.
2. `ComponentA` has one child: `ComponentB`
3. `ComponentB` is a component resource so it isn't added as a dependency, its children are.
4. `ComponentB` has one child: `CustomC`, a custom resource.
5. Since `CustomC` is a custom resource, it is added to the set of dependencies.
6. We try to await its URN, but we'll never get it because `RegisterResource` hasn't yet been called for it. And we hang waiting.
To address this, skip looking at a component's children if it is the component from which the dependency is being added.
In the case of the example, at step 3 the dependency expansion will stop: we won't look at `ComponentB`'s children because we're adding the dependency from `ComponentB`.
fix
We added a warning here with the intention of turning it into an error
in July 2022, but we never did.
While we're discussing whether we still want this to become an error,
drop the specific date and just refer to "a future version."
Refs #8799
This change fixes the `DeletedWith` resource option in the Go, Node.js,
and Python SDKs and adds tests.
This feature was a community contribution and while there were engine
tests included with the original PR, there weren't any tests confirming
the functionality worked correctly from each SDK.
Here's a summary of the fixes:
* Go: The `DeletedWith` resource option was never usable as it accepted
a URN instead of a Resource. We discussed this internally a while back
and decided to go ahead and fix this. (Note: While changing the
signature is technically a breaking change, the feature is currently
unusable, so the change would not break anyone, so there's no need to
wait for a major version bump.)
* Node.js: The `deletedWith` resource option did not work at all from
the Node.js SDK because it was incorrectly passing the resource object
itself in the RegisterResource request, rather than the resource's
URN.
* Python: The `deleted_with` resource option did not work at all from
the Python SDK because it was incorrectly passing the resource object
itself in the RegisterResource request, rather than the resource's
URN.
A `FailsOnDelete` resource has been added to the testprovider, which
will fail when its `Delete` gRPC is called. The tests use this to ensure
`Delete` is not called for resources of this type with the `DeletedWith`
option specified.
https://github.com/pulumi/pulumi/pull/11093 changed the Node.js SDK to pass a provider specified in a MLC's `ResourceOptions.provider` to the engine.
Unfortunately, this regresses behavior that existing programs rely on. For example:
```ts
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
const myRegion = new aws.Provider("us-east-1", {
region: "us-east-1",
});
const vpc = new awsx.ec2.Vpc("awsx-nodejs-default-args", {}, { provider: myRegion });
```
In the above program, an explicit _aws_ provider is being passed to the _awsx_ `VPC` component, with the intention that the _aws_ provider will be used as the provider for all of `Vpc`'s children.
With the change in https://github.com/pulumi/pulumi/pull/11093, the engine would try to call `Construct` for the `Vpc` using the specified `aws` provider, which does not work (it fails with `plugins that can construct components must support secrets`).
This change reverts the change from https://github.com/pulumi/pulumi/pull/11093 that included the `provider` in the `RegisterResourceRequest` for MLCs, and adds a regression test to lock-in the previous behavior.
Note: We do want to be able to support specifying a MLC's provider (to allow explicit providers for MLCs), but we'll address that in a separate change.
We used to always keep resource references when marshaling the results of `pulumi:pulumi:getResource`, but this regressed in #9323 to only keep resource if request's `acceptResources` flag was set. Unfortunately, that flag was not being set when calling `pulumi:pulumi:getResource` in the Go, Node.js, and Python SDKs (although, it was being set for .NET).
Two fixes:
1. Update the monitor to go back to always keeping resources when `pulumi:pulumi:getResource` is being invoked. This way, older SDKs that are not setting `acceptResources` will go back to the original behavior.
2. Update the SDKs to always set `acceptResources`, so that these newer versions of the SDKs will work with older engines that are checking `acceptResources`.
(2) will help us with EKS 1.0. We'll be able to update EKS to use a version of `@pulumi/pulumi` with the fix to set the `acceptResources` flag.
In many cases there is no need to delete resources if the container
resource is going to be deleted as well.
A few examples:
* Database object (roles, tables) when database is being deleted
* Cloud IAM bindings when user itself is being deleted
This helps with:
* Speeding the deletion process
* Removing unnecessary calls to providers
* Avoiding failed deletions when the pulumi user running the
plan has access to the container resource but not the contained
ones
To avoid deleting contained resources, set the `DeletedWith` resource
option to the container resource.
TODO:
Should we support DeletedWith with PendingDeletes?
Special case might be when the contained resource is marked as pending
deletion but we now want to delete the container resource, so
ultimately there is no need to delete the contained anymore
This is approach needs to occur in `resmon` since `*resmon` dispatches
`Construct` calls directly.
TS: Send providers map for all components
Ensure that `providers` is passed in all SDKs
The .NET SDK already does the right thing
Add golang integration test
Add local dependency
Fix test
Use new test
Get test working
Fix nits
* Stop using Get/SetRootResource in nodejs
Just use globalThis to make the stack resource a global rather than a module variable. This ensures that even if we end up with multiple Pulumi modules they can all grab the stack object and read URN off it.
We'll keep calling SetRootResource so old modules SxS continue to pickup the stack URN, at some point (4.0 probably) we can clean all this up.
Note that we also need the stack resource to be global for stack transformations, the roundtripping of the URN via the engine didn't preserve those.
* Update CHANGELOG
* Try to resovle circular module imports
* Remove comment
* Readd comment, remove unneeed await
* Fix comments
* Get rid of getRootResource, just pass parent down explictly
* Fix stack parents
* Add stack transform test
* Send smart aliases via gRPC to engine
* Add to SupportsFeature
* Restore old logic when the engine doesn't support smartAliases
* Add to deploytest ResourceOptions
* Add tests
* Add to CHANGELOG
* Fix test
* Rename proto fields
* Regenerate protobufs
* Fix up SDKs after field rename
* Rename deploytest aliases
* Rename internal fields
* Fix typo in c# code
* Fix typescript
* Rename feature to aliasSpecs
* Rename type to Spec
* Plumb in basics of retainOnDelete
* Add test
* Make test pass
* Add to changelog
* Add to API list
* lint
* Add semicolon
* Fix Infof call
* Fix method call
* new delete mode work
* cleanup
* protectTest
* Fix up test
* Fix replace
* Fix up test
* Warn on drop
* lint
* Change to just a bool flag
* Regenerate proto
* Rework to just a bool flag with no error
* Remove old comment
* Fix C# typo
* rm extra space
* Add missing semicolon
* Reformat python
* False typo
* Fix typo in js function name
* Reword docs
* lint
* Read doesn't need retainOnDelete
* [sdk/nodejs] Pickup provider as part of providers
* Update CHANGELOG_PENDING.md
* Add langhost test
* Handle empty providers
* Fix and expand test
* Make linter happy
* Add pluginServerURL to the nodejs SDK
Tests that this work are included in the PR.
* Update CHANGELOG_PENDING.md
* Add codegen change
* Add codegen test results
* Support invoke
* Setup defaults for InvokeOptions
When a resource `dependsOn` a remote component, we were not correctly waiting on it, because we were skipping over waiting on comoponents, and only waiting on their custom resource children. For remote components, we do not know the children, but waiting on the remote component will always wait on all children.
Co-authored-by: Mike Metral <1112768+metral@users.noreply.github.com>
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.
Fixes#6753.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
* Enable resource reference feature by default
Unless the PULUMI_DISABLE_RESOURCE_REFERENCES flag
is explicitly set to a truthy value, the resource reference feature is now
enabled by default.
* Set AcceptResources in the language SDKs
This can be disabled by setting the `PULUMI_DISABLE_RESOURCE_REFERENCES` environment variable to a truthy value.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.
Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.
The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.
These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).
An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.
This is the core of #2430.