Commit Graph

103 Commits

Author SHA1 Message Date
Will Jones d64448ec3b
Replace Rome with Biome in the NodeJS SDK ()
As @denbezrukov notes in , Rome (https://github.com/rome/tools,
the JavaScript toolchain we have been using to format and lint code in
the NodeJS SDK) has been deprecated. Biome (https://biomejs.dev/) has
sprung up in its place as a community fork and appears to be the best
bet for migration going forward. This commit introduces Biome, ports the
bits of configuration that need changing and updates formatting
accordingly.

Closes 

Co-authored-by: Denis Bezrukov <6227442+denbezrukov@users.noreply.github.com>
2024-06-24 11:14:56 +00:00
Julien P fa2a196c27
Vendor TypeScript and ts-node ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Fixes https://github.com/pulumi/pulumi/issues/15733

Historically these packages were direct dependencies of
`@pulumi/pulumi`. To decouple the node SDK from the precise version of
TypeScript, the packages are now declared as optional peer pependencies
of `@pulumi/pulumi` and customers can pick the versions they want.

The reason we mark the peer dependencies as *optional* is to prevent
package managers from automatically installing them. This avoids the
situation where the package manger would install a more recent version
of TypeScript without the user explictly opting in. Newer versions have
stricter type checks, and can thus stop existing programs from running
successfully.

When the peer dependencies are not present, we load the vendored
versions of the modules.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-10 15:26:37 +00:00
Julien P dd1d8607ae
Make function serialization work on typescript 4 and 5 ()
# Description

Builds on top of https://github.com/pulumi/pulumi/pull/15753

Fixes https://github.com/pulumi/pulumi/issues/15735

There are a couple breaking changes in the typescript API that we use in
`sdk/nodejs/runtime/closure/rewriteSuper.ts`. This PR adds a shim that
is used to bridge the differences and versions the snapshots where
needed.

This does not make typescript a peer dependency yet. Instead the tests
force a specific version to be used via [yarn
resolutions](https://classic.yarnpkg.com/lang/en/docs/selective-version-resolutions/).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-27 10:03:57 +00:00
Julien P f6cbf82d59
Move mockpackage tests to closure integration tests ()
# Description

The tests using `mockpackage` really should be integration tests. We
worked around this by including `mockpackage` in our dependencies, but
this has required some hackery in the build step to remove it again
there.

Now that we have the closure tests setup as integration tests, we can
add the tests using mockpackage there.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-26 11:38:01 +00:00
Julien P e3d0decccc
Reorganize closure tests to prepare for multiple typescript versions ()
# Description

In preparation of https://github.com/pulumi/pulumi/issues/15735 we make
the closure tests proper integration tests so that we can run them with
different typescript versions. Move each test to its own folder instead
of one large file.

This PR only changes tests, and does not touch any of the function
serialisation code.

Some snapshots had to be updated for indentation changes.

Hidden after all the test cases is the test script
[sdk/nodejs/tests/runtime/testdata/closure-tests/test.ts](dfcc953c08/sdk/nodejs/tests/runtime/testdata/closure-tests/test.ts)

Verified that tests run in CI
https://github.com/pulumi/pulumi/actions/runs/8389170587/job/22975068167?pr=15753

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-25 13:19:17 +00:00
Julien P 6f1438146e
Fix side-by-side tests ()
# Description

We have some tests that ensure that types for pulumi/pulumi are
compatible with the latest shipped version. However test failures do not
result in CI failures.


https://github.com/pulumi/pulumi/actions/runs/8153744675/job/22285983384#step:38:353

The current tests include a test using typescript 3.7.3 which does not
pass. I upgraded this to match 3.8.3, which is what we use for
pulumi/pulumi.

I believe the yarn.lock in the tests was committed by accident.

Fixes https://github.com/pulumi/pulumi/issues/15574

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-05 13:50:45 +00:00
Julien P 0d3d279bdc
Detect npm and yarn workspaces setups ()
# Description

Make `pulumi install` detect npm and yarn workspaces setups and
successfully install dependencies.

Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`
2024-02-16 08:25:12 +00:00
Thomas Gummerer 856c120944
increase the timeout for automation API tests ()
These tests are pretty close to hitting the timeout during regular runs
(e.g. one successful run I picked randomly took ~194000ms). The GitHub
action runners don't have super reliable performance, so we can easily
get pushed over this limit.

To make matters worse here, if we hit this timeout just at the right
time, the test doesn't exit cleanly (potentially related to
https://github.com/pulumi/pulumi-dotnet/issues/134), as I've seen a
`pulumi preview` process stick around in that case, preventing the tests
from shutting down). This means we have to wait until the CI job times
out after an hour until the failure is reported.

I'm not sure if we only got close to this timeout recently, or if this
is a longer standing issue, but it reproduces well for me locally.
Ideally of course the tests would be faster, but this is still "only"
~5min which is much faster than other tests, and should hopefully reduce
the amount of times we need to go through the merge queue, saving a lot
of time there.

Fixes https://github.com/pulumi/pulumi/issues/14842

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-13 10:40:27 +00:00
Fraser Waters e5983715bf Fix makefiles 2023-05-26 11:00:45 +01:00
Robbie McKinstry cf767cc47e
Regularlize lint workflow.
When we introduced Rome into the Node SDK, we added a Make rule
for formatting separate from our linting rule. This commit moves
format-checking into the lint rule so that the Node SDK's Make rules
will more closely match the Python SDK's Make rules.
Now, in both SDKs, `make lint` also checks formatting.
2023-05-11 00:07:16 -04:00
Robbie McKinstry c513796701
Add Rome autoformatter to Node SDK.
This commit adds the Rome autoformatter to the Node SDK toolchain.
It adds rules to the Makefile to validate formatting in CI, and to
autoformat. Lastly, it updates CI to ensure each commit correctly formats
the code.
2023-04-28 18:25:36 -04:00
Justin Van Patten 31bf640375 [auto/nodejs] Test remote operations
Also cleans up some error messages to be consistent with the CLI and other languages.
2022-11-09 05:26:07 -08:00
Aaron Friel 110dd76629 ci: Pin yarn lockfile for security & dependency scanning 2022-09-21 15:35:43 -07:00
Aaron Friel 6ef7ef64d6 ci: Enable testing of language version sets 2022-09-21 09:48:38 -07:00
Aaron Friel 746c5f6e3c ci: Enable robust retries on tests 2022-09-14 12:19:09 -07:00
Aaron Friel 272b987791 ci: ensure trunk is always green 2022-09-13 13:38:14 -07:00
Anton Tayanovskyy b06b860c2a
Remove PROJECT_ROOT Make var () 2022-06-24 10:04:40 -04:00
Anton Tayanovskyy 0e64874da7
Avoid realpath Makefile func that seems flaky on Win ()
* Avoid realpath Makefile func that seems flaky on Win

* Correct relpath for top-level Make
2022-06-23 15:32:18 -04:00
Ian Wahbe 9bad651337
Cleanup make ensure ()
* Don't format generated files

* Allow ensure to be stateful

* Cleanup comments and echo flags
2022-05-16 16:47:04 -07:00
Fraser Waters c3e084d028
Makefiles are very whitespace sensitive () 2022-03-26 09:00:21 +00:00
Matthieu Coudron d4b9d61d70
allow to override makefile variables ()
* allow to override makefile variables

When packaging pulumi-python for www.nixos.org, there is no /bin/bash
available so allow for an alternative path to the shell.

Same for versions, PYPI version can be set via the environment variable
PULUMI_VERSION.

* Update sdk/nodejs/Makefile

Co-authored-by: Fraser Waters <frassle@gmail.com>

* Update sdk/go/Makefile

Co-authored-by: Fraser Waters <frassle@gmail.com>

Co-authored-by: Fraser Waters <frassle@gmail.com>
2022-03-25 21:56:11 +00:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Aaron Friel 9c1d7f9c89 ci: improve x-plat testing on Windows 2022-03-06 14:45:26 -08:00
Aaron Friel 08d3f56982 ci: comprehensive caching 2022-03-06 14:45:26 -08:00
Aaron Friel 7b7ec02521 ci: divide and conquer integration tests by sdk and package group 2022-03-04 18:08:23 -08:00
Kyle Dixler 5caca0bde5
[sdk/nodejs] Fix make install ()
* remove mockpackage from devDependencies during build in Makefile

Co-authored-by: Kyle Dixler <kyle@pulumi.com>
2022-03-01 19:48:38 -08:00
Anton Tayanovskyy b65a1b89a6
Revert "Lower test parallelism on Windows ()" ()
This reverts commit 3b20dc37be.
2022-01-21 17:55:38 -05:00
Anton Tayanovskyy 3b20dc37be
Lower test parallelism on Windows () 2022-01-19 17:58:12 -05:00
Aaron Friel 73dcbe2c5b [sdk/nodejs] Ensure deps installed before lint, tsc
On first run of the makefile here, if a user doesn't have `eslint` or
`typescript` globally installed, `yarn run` won't succeed. If they do
have those dependencies globally, it'll use their locally installed
version, which is not the desired behavior.

This ensures `yarn` is run once before `lint` and `build_package`
targets, and since `build_package` is a dependency for all future
targets that use `yarn run` it covers the remainder of the cases.
2022-01-18 19:07:13 -08:00
Anton Tayanovskyy 76ba788985
Toward replacing MSBuild with make+bash on Windows ()
* Add coverage folder

* Adjust GHA to run Test target on Windows

* Extend error message on packages not found

* Try with path normalizer

* Edit Makefiles

* Skip SDK codegen tests on Windows

* Skip program gen tests on Windows

* Skip tests that started running on Windows and failing

* Fix non-compiling test

* Skip failing Windows HCL2 tests

* Fix lint

* Merged constants

* Skip one more test failing on Windows

* Disable cov-enabled builds on Windows

* Fix  TestDeterminePulumiPackages on Windows

* Fix TestInstallCleansOldFiles test on Windows

* Fix TestCreatingProjectWithPulumiBackendURL on Windows

* Weaken TestAbbreviateFilePath to pass on Windows

* Fix TestDepRootCalc on Windows

* Fix LocalUrl() to be sensible on Windows

* Fix Go lint issue

* Windows fix for TestComments

* Cross-ref skip tracking issue

* Cross-ref issue to fix asset_test skips

* More cross-ref issues

* Use choco not chocolatey

* Use yarn run to ensure the right tools are selected

* Revert python3->python change in common.mk

* In CI context, use python not python3

* More randomness in temp folder names to aoid Windows collisions

* Go lint
2022-01-07 22:27:14 -05:00
Pat Gavlin d3e49ecb65
[cli, testing, github] Gather code coverage data in CI. ()
These changes add support for gathering code coverage data during tests.

For tests that do not involve the Pulumi CLI, this is straightforward: all of
the ecosystems we target already support gathering coverage data, and we follow
the rules accordingly. Support for each language is broken out into its own
commit.

For tests that do involve the Pulumi CLI, the picture is a bit more complicated.
Go does not make it trivial to perform a coverage-instrumented build (go build
does not have a -cover flag, for example). In lieu of official support, we abuse
go test -c and TestMain to produce a build of the CLI that supports collecting
and reporting coverage data.
2021-11-30 17:24:01 -08:00
Pat Gavlin 2dcf3806bd
[automation-api] Exclude tests from test_fast. ()
The Automation API tests take long enough that thay don't really fit
into `test_fast`. These changes move those tests to `test_all`.
2021-09-16 17:33:33 -07:00
Horace Lee a92a005d68
Use ESlint instead of TSlint ()
Migrated TSlint configs to ESlint ones using [tslint-to-eslint-config](https://github.com/typescript-eslint/tslint-to-eslint-config) tool, and refined the configs to better match the current coding style.

Changes:
- [member-delimiter-style](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/member-delimiter-style.md#options) that as suggested default for type definition to be  with `semicolon`
- Indentation fixes that is enforced by [eslint-indent](https://eslint.org/docs/rules/indent#options)
- Added dependencies for ESlint with Typescript
- Removed TSlint
2021-08-10 11:31:59 -07:00
Anton Tayanovskyy ba70b3fdba
Attempt to avoid triple-building projects in the solution () 2021-07-28 20:31:11 -04:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment ()
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Paul Stack 1d3c9edb6c
Ensure that make brew works as expected rather than passing empty version ()
Fixes:#6565

As part of , the logic for determing the version of the build was
moved to be a dependency on pulumictl.

Unfortunately, the homebrew installs use the "make dist" command to
build + install Pulumi to the user maching and as that would have a
dependency on pulumictl and it not existing on the user machine, it
would pass an empty version to the ldflag

This then manifested to the user as:

```
▶ pulumi version
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.22.0', run
   $ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
```

We are able to mitigate this behaviour by bringing back the get-version
script and using that script as part of the make brew installation

We can see that the versions are the same between the 2 different
installation techniques

```
make dist <------- uses pulumict
DIST:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=2.24.0-alpha.1616029310+787eb70a" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
DIST:
BUILD:
```

```
make brew <----- uses the legacy script
▶ make brew
BREW:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=v2.24.0-alpha.1616029310+g787eb70a2" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
BREW:
```

A full post mortem will be carried out to ensure we mitigate these
types of errors going forward and that we are able to better test
these types of situations
2021-03-18 02:07:02 +00:00
Paul Stack c48ba37fcf
Migrate the version calculation to use pulumictl () 2021-03-10 19:03:29 +00:00
Evan Boyle 268abea53b
bump nodejs test timeout for automation api () 2020-12-16 12:47:15 -08:00
Justin Van Patten 918615f072
[sdk/nodejs] Implement getResource in the mock monitor ()
Otherwise, unit tests for programs that reference resources that have been registered with `registerResourceModule` fail with unhandled exceptions.
2020-12-10 09:30:34 -08:00
evanboyle 42f301ac0f bump node test timeout 2020-10-08 12:19:01 -07:00
evanboyle cf194bff4b remove parallelism from node automation api operations 2020-10-08 12:19:01 -07:00
evanboyle 848086fa26 remove stack property getters 2020-10-08 12:19:01 -07:00
evanboyle 66a71f2bab cleanup automation api host runtime settings post-run, fix tests 2020-10-08 12:19:01 -07:00
evanboyle 5380223c28 bump test timeout 2020-10-08 12:19:01 -07:00
evanboyle fa388380c4 LocalWorkspace.projectSettings() 2020-10-08 12:19:01 -07:00
evanboyle ebdd3c1053 project settings 2020-10-08 12:19:01 -07:00
Lee Briggs 95bc138b41
add master branch workflow () 2020-09-21 16:20:05 -07:00
Lee Briggs 8c314ec39a
Add brew make targets to sdk makefiles 2020-05-13 20:42:01 -07:00
Pat Gavlin 9b1b9cca24
Makefile: properly encode dependencies ()
This allows for the use of `make -j`, which speeds up a full build
dramatically at the cost of rather incomprehensible logs.
2020-05-04 14:26:52 -07:00