Commit Graph

238 Commits

Author SHA1 Message Date
Zaid Ajaj aa9ca8a4ed
[sdk/nodejs] Maintain secretness and input dependencies of output invokes ()
### Description 

Addresses the nodejs part of , it does the following
 - Keeping track of secretness of inputs
- Unwraps the inputs such that they are sent as plain values to the
engine
 - Marks the invoke result as secret if any of the inputs is a secret
- Marks the dependencies of the output invoke response being the list of
all dependencies of the inputs
- Enables conformance tests `l2-invoke-secrets` and
`l2-invoke-dependencies`
 - Adds unit tests for newly added function `unwrapSecretValues`
2024-10-05 00:40:19 +00:00
Julien 717cb75201
Use @types/node matching the ts version for integration tests ()
@types/node regularly drops support for older typescript versions. To
ensure our tests do not break when this happens, we can use release tags
that point to the latest supported version of @types/node for each ts
version.

Fixes https://github.com/pulumi/pulumi/issues/17135
2024-09-06 09:21:47 +00:00
Julien 7efceb8f96
Pin @types/node in closure tests ()
Tests for Nodejs closure serialisation are currently failing due to a
bug in the node types, see
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/70432
2024-09-02 17:50:31 +00:00
Julien a3c0ea5455
Add NodeJS test for parameterized providers ()
This takes https://github.com/pulumi/pulumi/pull/16392 and updates local
SDK generation for nodejs and adds parametrisation support for invokes

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-08-09 08:34:29 +00:00
Justin Van Patten 5d6e340c10
[sdk/nodejs] Save pkg/mod registrations on async local store ()
Rather than saving package/module registrations in a module variable,
save them in the store where other state is stored. This way, if there
are multiple copies of `@pulumi/pulumi`, they'll use the same store,
rather than having separate copies of registrations.

Fixes 

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-08-02 07:37:06 +00:00
Will Jones 100470d2e7
Revert "Implement `dependsOn` for invokes in the NodeJS SDK ()" ()
From SDKs, we call invokes in one of two ways:

* In a "non-output" context (e.g. `getX`), which has a result dependent
on the language (e.g. a `Promise` in NodeJS) that _does not_ track
dependencies.
* In an "output" context (e.g. `getXOutput`), which has an `Output` type
and does track dependencies.

In the non-output case, `dependsOn` really doesn't make sense, since
this style of invoke is inherently ignoring dependency tracking/outputs.
This commit thus reverts 492c57c7dd so
that we can rethink the design before people's programs are subtly
broken in this case.
2024-07-12 10:16:55 +00:00
Vincent Link 9b009d7db2
Fix usage of getters and setters in Lambda callback functions ()
Fixes  

Hi,

this is an attempt to fix  , feedback is greatly appreciated!

I was not able to test if the result works as expected, I just adjusted
the code and manually created a test snapshot from the mocha console
output until I thought it looked right.

Could someone tell me...
1. how to actually build the `@pulumi/pulumi` package and how to test
the fixes quickly?
2. how to create a snapshot for a new closure test?
3. if there is a quick way to write the TypeScript code in this project,
set breakpoints and run tests against it?
- I plan to investigate the pnpm bundling bug of the `CallbackFunction`
but working on this and testing the code is extremely tedious right now
as I do not know your secrets (yet 😀)
- I didn't find these infos in the `CONTRIBUTING.md` and other READMEs
- I am very familiar with TypeScript and the nodejs tooling, but no clue
how everything is stitched together in this repo
- feel free to send me a Slack message in the Pulumi Community or
suggesting another communication channel



Thanks!

fyi @julienp
2024-07-08 09:51:12 +00:00
Will Jones 492c57c7dd
Implement `dependsOn` for invokes in the NodeJS SDK ()
This commit adds support for passing `dependsOn` to invokes (whether
streamed or not) in the NodeJS SDK. This allows programs to ensure that
certain invokes are executed after things they depend on, even if that
dependency is not implicitly captured with an input-output relationship.

Part of 
2024-07-03 14:47:59 +00:00
Will Jones d64448ec3b
Replace Rome with Biome in the NodeJS SDK ()
As @denbezrukov notes in , Rome (https://github.com/rome/tools,
the JavaScript toolchain we have been using to format and lint code in
the NodeJS SDK) has been deprecated. Biome (https://biomejs.dev/) has
sprung up in its place as a community fork and appears to be the best
bet for migration going forward. This commit introduces Biome, ports the
bits of configuration that need changing and updates formatting
accordingly.

Closes 

Co-authored-by: Denis Bezrukov <6227442+denbezrukov@users.noreply.github.com>
2024-06-24 11:14:56 +00:00
Julien P fa2a196c27
Vendor TypeScript and ts-node ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Fixes https://github.com/pulumi/pulumi/issues/15733

Historically these packages were direct dependencies of
`@pulumi/pulumi`. To decouple the node SDK from the precise version of
TypeScript, the packages are now declared as optional peer pependencies
of `@pulumi/pulumi` and customers can pick the versions they want.

The reason we mark the peer dependencies as *optional* is to prevent
package managers from automatically installing them. This avoids the
situation where the package manger would install a more recent version
of TypeScript without the user explictly opting in. Newer versions have
stricter type checks, and can thus stop existing programs from running
successfully.

When the peer dependencies are not present, we load the vendored
versions of the modules.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-10 15:26:37 +00:00
Will Jones 0a6876c527
Support serialising reserved NodeJS identifiers ()
This commit addresses part of , in which we fail to serialise
closures whose code would use reserved identifiers like `exports`. This
is due to a change we made where module imports are hoisted to avoid
importing the same module multiple times. Previously, code adopted a
strategy of passing all dependencies using a `with` statement, viz.:

```typescript
function x() {
  return (function () {
    with({ fooBarBaz: require("foo/bar/baz"), ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}

function y() {
  return (function () {
    with({ fooBarBaz: require("foo/bar/baz"), ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}
```

This was changed to remove the duplicate imports, yielding code like:

```typescript
const fooBarBaz = require("foo/bar/baz")

function x() {
  return (function () {
    with({ ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}

function y() {
  return (function () {
    with({ ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}
```

However, while the previous approach would work with reserved
identifiers such as `exports` (`with({ exports, ... }) { ... }` is
perfectly acceptable), the new one does not (`const exports = ...` is
not acceptable since NodeJS will not allow redeclaration of the
`exports` global).

This commit combines the two approaches. Modules are only imported once,
but if an import would use a reserved identifier, we generate a fresh
non-conflicting identifier and alias this using a `with` statement. For
example:

```typescript
const fooBarBaz = require("foo/bar/baz")

// __pulumi_closure_import_exports is generated to avoid shadowing the reserved "exports"
const __pulumi_closure_import_exports = require("some/other/module")

function x() {
  return (function () {
    with({ exports: __pulumi_closure_import_exports, ... }) {
      // Use of fooBarBaz and exports
    }
  }).apply(...)
}
```

Note that it is not expected that  will be solved in its entirety.
While this commit fixes code that introduces identifiers like `exports`,
the introduction in question in that issue is caused by the use of
`pulumi.output(...)` in the constructor of a dynamic resource provider.
Since dynamic resource providers are implemented under the hood by
serialising their code, we attempt to serialise `pulumi.output` and its
dependency chain. This commit allows us to make more progress in that
regard, but other things go wrong thereafter. Fixing the deeper issue
that underpins  (and likely other challenges with dynamic
providers) is probably a more involved piece of work.
2024-04-10 15:12:43 +00:00
Julien P 29dfdcae72
Serialize function values obtained from Function.bind ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

When encountering a function obtained via `Function.bind` we previously
failed to serialise the function because we could not get the function
text.

This PR uses the v8 debugger API to grab the internal
`[[TargetFunction]]` property to obtain the original function and
re-bind it.

This does currently not handle successive binds like

```(function f() { ... }).bind("a").bind("b")```

Ref https://github.com/pulumi/pulumi/issues/5294

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-10 11:17:53 +00:00
Julien P ba78d97a93
Handle serialization of aliases for well known native functions ()
# Description

Emit expressions for aliased native functions
https://github.com/pulumi/pulumi/issues/5294#issuecomment-2031404191

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-08 15:06:40 +00:00
Julien P 2d8e09d834
Update builtin module list for function serialization ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Update the list of node modules detected as builtin during function
serialisation.

This helps with some of the cases in
https://github.com/pulumi/pulumi/issues/5294 (notably trying to use
`fs/promises`).

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-02 08:28:18 +00:00
Julien P dd1d8607ae
Make function serialization work on typescript 4 and 5 ()
# Description

Builds on top of https://github.com/pulumi/pulumi/pull/15753

Fixes https://github.com/pulumi/pulumi/issues/15735

There are a couple breaking changes in the typescript API that we use in
`sdk/nodejs/runtime/closure/rewriteSuper.ts`. This PR adds a shim that
is used to bridge the differences and versions the snapshots where
needed.

This does not make typescript a peer dependency yet. Instead the tests
force a specific version to be used via [yarn
resolutions](https://classic.yarnpkg.com/lang/en/docs/selective-version-resolutions/).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-27 10:03:57 +00:00
Julien P f6cbf82d59
Move mockpackage tests to closure integration tests ()
# Description

The tests using `mockpackage` really should be integration tests. We
worked around this by including `mockpackage` in our dependencies, but
this has required some hackery in the build step to remove it again
there.

Now that we have the closure tests setup as integration tests, we can
add the tests using mockpackage there.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-26 11:38:01 +00:00
Julien P e3d0decccc
Reorganize closure tests to prepare for multiple typescript versions ()
# Description

In preparation of https://github.com/pulumi/pulumi/issues/15735 we make
the closure tests proper integration tests so that we can run them with
different typescript versions. Move each test to its own folder instead
of one large file.

This PR only changes tests, and does not touch any of the function
serialisation code.

Some snapshots had to be updated for indentation changes.

Hidden after all the test cases is the test script
[sdk/nodejs/tests/runtime/testdata/closure-tests/test.ts](dfcc953c08/sdk/nodejs/tests/runtime/testdata/closure-tests/test.ts)

Verified that tests run in CI
https://github.com/pulumi/pulumi/actions/runs/8389170587/job/22975068167?pr=15753

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-25 13:19:17 +00:00
Julien P 5a19d754d7
Fix codepaths computation when working dir is nested relative to package.json ()
# Description

When using `tsc` to precompile typescript in a monorepo, we need to work
relative to the location of `package.json`, not where the pulumi program
lives (which is usually nested further down).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-08 16:16:47 +00:00
Julien P 6daee6f609
Properly capture nodejs:crypto and global.crypto ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15479

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-07 13:38:48 +00:00
Julien P af8f82a1bc
Upgrade to grpc-js 1.10.1 and remove calls to deprecated server.start ()
# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15499

This should be safe, as we are effectively already running 1.10.1. This
makes this the new minimum so we can remove the `server.start` calls
(the server is started in `server.bindAsync`) that cause deprecation
warnings to be printed in Automation API or when using dynamic providers
with nodejs.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-23 11:32:40 +00:00
Fraser Waters 994f660042
Move nodejs feature checks to startup ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

We need to synchronously check for transform support in
`registerStackTransformation` and resource constructors when adding
remote transform support to node (c.f.
https://github.com/pulumi/pulumi/pull/14303). This change changes all
the feature checks to be done at startup and then accessed via just a
field lookup. Adding the "transform" feature to this set is clearly
simple.

Sadly there's no single entry point to make these changes in. So we need
to update entry point of construct/call, the entry point of programs,
and test setup. Miss any one of these and current tests start failing.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-19 14:35:23 +00:00
Thomas Gummerer a66e366f75
langhost test: retry if the langhost is unavailable ()
This test is racy, as the langhost can sometimes take a little bit too
long to be ready to accept connections successfully. Just retry a few
times in this case, to make sure it's ready to have the test run against
it.

I haven't been able to reliable reproduce the flakyness locally, so
there's no guarantee that this will actually do what it says
successfully. I think it's still worth a try and we can monitor CI to
verify if this fixed the issue, and revert this PR if the same issue
appears again.

Fixes https://github.com/pulumi/pulumi/issues/14841
2023-12-13 14:31:50 +00:00
Fraser Waters 6e986f90af
Pass root and main info to language host methods ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This is two changes rolled together in a way.

Firstly passing some of the data that we pass on language runtime
startup to also pass it to Run/GetRequiredPlugins/etc. This is needed
for matrix testing, as we only get to start the language runtime up once
for that but want to execute multiple programs with it.
I feel it's also a little more consistent as we use the language
runtimes in other contexts (codegen) where there isn't really a root
directory, and aren't any options (and if we did do options the options
for codegen are not going to be the same as for execution). It also
means we can reuse a language host for shimless and substack programs,
as before they heavily relied on their current working directory to
calculate paths, and obviosly could only take one set of options at
startup. Imagine a shimless python package + a python root program, that
would have needed two startups of the python language host to deal with,
this unblocks it so we can make the engine smarter and only use one.

Secondly renaming some of the fields we pass to
Run/GetRequiredPlugins/etc today. `Pwd` and `Program` were not very
descriptive and had pretty non-obvious documentation:
```
string pwd = 3;     // the program's working directory.
string program = 4; // the path to the program to execute.
```
`pwd` will remain, although probably rename it to `working_directory` at
some point, because while today we always start programs up with the
working directory equal to the program directory that definitely is
going to change in the future (at least for MLCs and substack programs).
But the name `pwd` doesn't make it clear that this was intended to be
the working directory _and_ the directory which contains the program.

`program` was in fact nearly always ".", and if it wasn't that it was
just a filename. The engine never sent a path for `program` (although we
did have some unit tests to check how that worked for the nodejs and
python hosts).

These are now replaced by a new structure with (I think) more clearly
named and documented fields (see ProgramInfo in langauge.proto).

The engine still sends the old data for now, we need to update
dotnet/yaml/java before we break the old interface and give Virtus Labs
a chance to update [besom](https://github.com/VirtusLab/besom).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-10 17:30:51 +00:00
Fraser Waters ac71ebc1d3
Add google-protobuf types ()
Adds "@types/google-protobuf" to the nodesdk and changes `requires` to
plain imports.
2023-10-27 09:55:44 +00:00
Kyle Dixler 3d14952860
NodeJS now accepts unknown resource IDs for read resource ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes  

NodeJS now accepts unknown resource IDs for read resource.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-19 16:00:10 +00:00
Fraser Waters 3f01459991 Fix a working directory issue in language-nodejs
Found this while working on matrix testing which runs the language host
in a totally different directory to where the projects are.
2023-08-03 12:17:12 +01:00
Pat Gavlin 248f78bafe [sdk/*] Add support for resource source positions
Add support to the core SDKs for reporting resource source positions.

In each SDK, this is implemented by crawling the stack when a resource
is registered in order to determine the position of the user code that
registered the resource.

This is somewhat brittle in that it expects a call stack of the form:
- Resource class constructor
- abstract Resource subclass constructor
- concrete Resource subclass constructor
- user code

This stack reflects the expected class hierarchy of "cloud resource /
component resource < customresource/componentresource < resource".

For example, consider the AWS S3 Bucket resource. When user code
instantiates a Bucket, the stack will look like
this in NodeJS:

    new Resource (/path/to/resource.ts:123:45)
    new CustomResource (/path/to/resource.ts:678:90)
    new Bucket (/path/to/bucket.ts:987:65)
    <user code> (/path/to/index.ts:4:3)

In order to determine the source position, we locate the fourth frame
(the `<user code>` frame).
2023-07-13 16:46:04 -07:00
Justin Van Patten db1d8c7217 [sdk/nodejs] Update langhost test for packaged component providers
Update the test for the behavior we expect. Currently fails.
2023-07-13 07:16:56 -07:00
Kyle Dixler b535bcff23
Suppress grpc rejections due to the resource monitor terminating
A failed update can lead to an unavailable resource monitor creating an
uncaught rejected promise which makes Automation API exit prematurely
since the exception was uncatchable. Prints the error message to stderr
only once to avoid spamming the user.

Fixes 
Fixes 
2023-06-14 14:43:32 -07:00
Robbie McKinstry d2d73a6e58
Eliminate async promise executor.
Promise executors are not intended to be async. This PR enforces that.
2023-05-11 13:05:43 -04:00
Robbie McKinstry e6c4e39ad2
Fix closure serialization test cases. 2023-05-09 14:50:28 -04:00
Robbie McKinstry 03f6b5a1a8
Fix brittle test case. 2023-04-28 18:53:02 -04:00
Robbie McKinstry e05a3bd81c
Apply autoformat
This commit applies the Rome autoformatter to the Node SDK.
These changes are automatically produced. To reproduce these
changes, run `make format` from inside sdk/nodejs.
2023-04-28 18:27:10 -04:00
Justin Van Patten 9e73c75cd6 [sdk/nodejs] Fix hang due to component children cycles
When a resource depends on a local component resource, rather than setting the component resource itself as a dependency, each of the component's descendants is added as a dependency. This can lead to hangs when cycles are introduced.

For example, consider the following parent/child hierarchy, where `ComponentA` is the parent of `ComponentB` and `ComponentB` is the parent of `CustomC`:

```
ComponentA
    |
ComponentB
    |
 CustomC
```

If `ComponentB` specifies it has a dependency on `ComponentA`, the following takes place as part determining the full set of transitive dependencies:

1. `ComponentA`  is a component resource so it isn't added as a dependency, its children are.
2. `ComponentA` has one child: `ComponentB`
3. `ComponentB`  is a component resource so it isn't added as a dependency, its children are.
4. `ComponentB` has one child: `CustomC`, a custom resource.
5. Since `CustomC` is a custom resource, it is added to the set of dependencies.
6. We try to await its URN, but we'll never get it because `RegisterResource` hasn't yet been called for it. And we hang waiting.

To address this, skip looking at a component's children if it is the component from which the dependency is being added.

In the case of the example, at step 3 the dependency expansion will stop: we won't look at `ComponentB`'s children because we're adding the dependency from `ComponentB`.

fix
2023-04-07 07:34:38 -07:00
Kyle Pitzen 92cb70996c feat(sdk/nodejs): delegates alias computation to the engine 2022-12-15 17:23:31 -05:00
bors[bot] 78bd0ebc08
Merge
11481: Closure emit requires in global scope, to have them run in Lambda init phase r=RobbieMcKinstry a=pjoe

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

Emit `require` for importing modules in global scope.
This allows these to run during the init phase of AWS Lambda, where more CPU is available, resulting in greatly imrpoved cold start performance (often 5x or more).

Fixes 

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Pelle Johnsen <pelle.johnsen@gmail.com>
2022-12-09 18:10:33 +00:00
Justin Van Patten 8259c2b24e [sdk/nodejs] Fix regression when passing a provider to a MLC
https://github.com/pulumi/pulumi/pull/11093 changed the Node.js SDK to pass a provider specified in a MLC's `ResourceOptions.provider` to the engine.

Unfortunately, this regresses behavior that existing programs rely on. For example:

```ts
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

const myRegion = new aws.Provider("us-east-1", {
  region: "us-east-1",
});

const vpc = new awsx.ec2.Vpc("awsx-nodejs-default-args", {}, { provider: myRegion });
```

In the above program, an explicit _aws_ provider is being passed to the _awsx_ `VPC` component, with the intention that the _aws_ provider will be used as the provider for all of `Vpc`'s children.

With the change in https://github.com/pulumi/pulumi/pull/11093, the engine would try to call `Construct` for the `Vpc` using the specified `aws` provider, which does not work (it fails with `plugins that can construct components must support secrets`).

This change reverts the change from https://github.com/pulumi/pulumi/pull/11093 that included the `provider` in the `RegisterResourceRequest` for MLCs, and adds a regression test to lock-in the previous behavior.

Note: We do want to be able to support specifying a MLC's provider (to allow explicit providers for MLCs), but we'll address that in a separate change.
2022-12-02 09:54:07 -08:00
Pelle Johnsen cc4e34586e fix: emit closure requires in global scope 2022-11-29 07:20:09 +00:00
Justin Van Patten 1fca4e9d89 [sdk/nodejs] Cleanup use of `asyncTest` and `assertAsyncThrows`
The `asyncTest` and `assertAsyncThrows` helpers were necessary when originally written, but are no longer needed as Mocha has built-in support for testing async functions that return promises, and Node's `assert.rejects` can be used to assert whether a promise has been rejected.
2022-11-09 15:08:08 -08:00
Kyle Pitzen 5cfad4a909 fix: Allows for parallel pulumi programs to run in the node runtime 2022-10-13 07:15:25 -04:00
bors[bot] 5493b82530
Merge
10622: Enable ESModule interoperability r=RobbieMcKinstry a=RobbieMcKinstry

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

This PR enables ESModule interop for the NodeJS SDK. By enabling interop, we should in theory remove subtle bugs from Pulumi programs that assume our libraries are interopable. In practice, only a few modules weren't already interoperable. Nevertheless, enabling this compiler flag will require future code to be compatible with ESModules, potentially eliminating future bugs.

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/10621

This PR is required for bundling the NodeJS runtime (https://github.com/pulumi/pulumi/issues/10210).

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have updated the [CHANGELOG-PENDING](https://github.com/pulumi/pulumi/blob/master/CHANGELOG_PENDING.md) file with my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
2022-09-21 22:20:53 +00:00
Robbie McKinstry 4c4b693b86
Remove test case that has bloated by over 12k characters 2022-09-20 13:10:48 -04:00
Robbie McKinstry 2b1147970c
WIP 2022-09-20 13:06:13 -04:00
Robbie McKinstry e978311216
Enable ESModule interoperability 2022-09-20 10:34:00 -04:00
Kyle Pitzen ae346aa33c chore(sdk/nodejs): Adds tests for resource alias combinations 2022-09-16 14:18:59 -04:00
Fraser Waters 615ae91477
Add getOrganization to nodejs and python ()
* Add getOrganization to nodejs and python

* Add to CHANGELOG

* lint

* lint

* format

* Fix python test

* Fix nodejs settings

* Add contextproperty

* Backcompat python kwargs
2022-08-31 10:33:29 +01:00
Aaron Friel d54aaf1615
[sdk/nodejs] Use loopback addresses for providers & automation API () 2022-08-25 15:40:07 -07:00
Kyle Pitzen c4c46c3222
fix(ux): Adds a warning message when module resolution is ambiguous ()
previously, when both index.ts and index.js were present
and no main key was provided in Pulumi.yaml, Pulumi would quietly
resolve the entrypoint as index.js.  This messages that decision in a warning
2022-08-16 14:41:15 -04:00
Kyle Pitzen 83f3d6e260
chore(sdk/python): Deprecate PULUMI_TEST_MODE ()
chore(sdk/nodejs): Deprecate PULUMI_TEST_MODE
2022-08-16 09:25:52 -04:00
Fraser Waters 0d0144bca2
Stop using Get/SetRootResource in nodejs ()
* Stop using Get/SetRootResource in nodejs

Just use globalThis to make the stack resource a global rather than a module variable. This ensures that even if we end up with multiple Pulumi modules they can all grab the stack object and read URN off it.

We'll keep calling SetRootResource so old modules SxS continue to pickup the stack URN, at some point (4.0 probably) we can clean all this up.

Note that we also need the stack resource to be global for stack transformations, the roundtripping of the URN via the engine didn't preserve those.

* Update CHANGELOG

* Try to resovle circular module imports

* Remove comment

* Readd comment, remove unneeed await

* Fix comments

* Get rid of getRootResource, just pass parent down explictly

* Fix stack parents

* Add stack transform test
2022-06-30 11:04:49 +01:00