Commit Graph

30 Commits

Author SHA1 Message Date
Julien P 5ff35273d6
Fix merge failures ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15528

See https://github.com/pulumi/pulumi/pull/15540 &
https://github.com/pulumi/pulumi/pull/15531

Re-creating this as a PR with `ci/test` label so we can get it merged.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-29 21:06:24 +00:00
Julien P 87b01bf35a
Handle workspaces in computeCodePaths ()
# Description

When we compute the code paths for function serialisation, we are not
handling yarn/npm workspace setups where the node_modules might be in a
different directory.

Fixes 
Fixes https://github.com/pulumi/pulumi/issues/2661
Fixes https://github.com/pulumi/pulumi/issues/7168
Fixes https://github.com/pulumi/pulumi/issues/3013

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-22 11:41:37 +00:00
Julien P 2a2d247755
Use pnpm as package manager if we find a pnpm-lock.yaml file ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Initial support pnpm. Note that this does not support pnpm workspaces
yet.

This also does not handle passing the package manager through from
`pulumi new`. Once a user manually runs pnpm, creating a pnpm-lock.yaml,
we'll detect that and pnpm.

Fixes https://github.com/pulumi/pulumi/issues/15455

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 13:41:21 +00:00
Julien P 0d3d279bdc
Detect npm and yarn workspaces setups ()
# Description

Make `pulumi install` detect npm and yarn workspaces setups and
successfully install dependencies.

Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`
2024-02-16 08:25:12 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
bors[bot] ec469864ee
Merge
13021: Use os.tmpfile instead of creating our own. r=RobbieMcKinstry a=RobbieMcKinstry

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
This PR uses os.Tempfile instead of creating our own with UUID. The standard library's os.Tempfile function
is race-safe, collision-proof, and uses the OS's tempdir, which allows the OS to clean up old files if Pulumi were to 
fail to do so.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


13054: ci(lint): Helpful message on tidy failure r=abhinav a=abhinav

When the 'go mod tidy' lint check fails,
it prints a simple error message reporting the failure.

Most will resort to running `go mod tidy` manually
to attempt to fix this, but this will not always work.
We need to run `./scripts/tidy.sh` which updates all go.mod files
and includes specific compatibility flags.


Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-05-30 19:48:20 +00:00
Robbie McKinstry 73ff332f5e
Use os.tmpfile instead of creating our own.
The Node SDK creates a tmp file when packing programs, to house
the packed tarball. We previously used uuid to create our own
tmpfile with a random name. This commit uses os.tempfile instead.
The stdlib function os.tempfile is race safe, collision-proof, and
defaults to a tmp directory, which the OS is free to automatically
clean up in case Pulumi fails to.
2023-05-30 13:58:02 -04:00
bors[bot] 6f0ee42b7c
Merge
13033: test(sdk/npm): Don't download from NPM or Yarn regstries r=Frassle a=abhinav

Follow up to 

Tests that download from NPM or Yarn will be brittle, unsafe, and slow.
We don't need to hit these servers directly.

This change alters the package installation tests to
set up a local HTTP server that implements a subset of the
[NPM Package Registry API][1] -- just enough to satisfy these tests.

  [1]: https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md

In the tests, we'll set `$HOME` to a new temporary directory and write
an `~/.npmrc` or `~/.yarnrc` requesting use of the local server as the
registry.

With this, the tests are entirely local. No external requests.

Also contains minor fix-ups to the test that were pointed out in :

- Use 'require' in chdir
- Use subtests for `Test*Install`
- Don't send command output to /dev/null

These fix-ups are in separate commits to keep the meat of this change
independently reviewable.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-05-29 18:29:29 +00:00
Fraser Waters aa59a7e72f Fix TestNPMInstallCmd 2023-05-29 14:16:23 +01:00
Abhinav Gupta c024a4013a
test(sdk/npm): Don't send command output to /dev/null
Sending all output to /dev/null makes debugging more difficult.
Send the command output to the testing.T logger
2023-05-26 14:12:29 -07:00
Abhinav Gupta 051a864dca
test(sdk/npm): Use subtests for install
Use subtests for NPM and Yarn install tests.
This will ensure that shared state is properly reset
between the cases.
2023-05-26 14:12:28 -07:00
Abhinav Gupta 88e4beaf19
test(sdk/npm): Use 'require' in chdir
chdir should not proceed if certain operations fail.
Use `require` instead of `assert` for those cases
so that test execution halts if they fail.
2023-05-26 14:12:28 -07:00
Abhinav Gupta f9deb7ab57
test(sdk/npm): Don't download from NPM or Yarn regstries
Tests that download from NPM or Yarn will be brittle, unsafe, and slow.
We don't need to hit these servers directly.

This change alters the package installation tests to
set up a local HTTP server that implements a subset of the
NPM Package Registry API [1] -- just enough to satisfy these tests.

  [1]: https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md

In the tests, we'll set `$HOME` to a new temporary directory and write
an `~/.npmrc` or `~/.yarnrc` requesting use of the local server as the
registry.

With this, the tests are entirely local. No external requests.
2023-05-26 14:12:23 -07:00
Robbie McKinstry bc7a73436e
Tests: Trim file extension for Windows 2023-05-26 15:59:41 -04:00
Robbie McKinstry 6a9cf0d153
Fix typo in tests 2023-05-26 12:31:01 -04:00
Robbie McKinstry da17bfe24d
Update sdk/nodejs/npm/npm_test.go
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-05-26 11:45:39 -04:00
Robbie McKinstry ff0e13c5e0
Add unit test for npm installation. 2023-05-26 11:43:22 -04:00
Robbie McKinstry 0251902d50
Abstract package manager behavior behind interface.
This PR moves the logic of `Install` and `Pack` behind an interface
for each of the supported package managers. We now dynamically dispatch
to an NPM implementation for NPM installation WLOG yarn.
2023-05-24 12:55:02 -04:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Eng Zer Jun 48eff4676a
test: use T.TempDir to create temporary test directory
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-12-03 15:17:08 +08:00
Fraser Waters d3c7145614 Clean up a few more pkg/errors uses 2022-11-09 16:33:47 +00:00
Aaron Friel 7bef1b717e ci: Prevent yarn mutex timeout by single-threading yarn commands 2022-09-14 10:07:21 -07:00
Eng Zer Jun 19d84ef1f3
test: use `T.Setenv` to set env vars in tests ()
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-24 10:41:44 +01:00
Ian Wahbe f3e430a1da
Respond to SIGINT during plugin install ()
* Respond to SIGINT

With the current state of the PR, crash on SIGINT. This is progress.

* Don't crash responding to SIGINT

* Close on cancel instead of terminate

* CL

* Fix lint

* Add ctx for node

* Be consistent for test context
2022-06-09 14:57:56 -07:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Evan Boyle 6ae6cc45b7
skip installing dev dependencies for nodejs plugin setup () 2021-06-02 11:09:03 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Lee Briggs 4d96994fd1
switch uuid packages ()
Co-authored-by: Pat Gavlin <pat@pulumi.com>
2020-11-04 03:13:04 +00:00
Justin Van Patten 46c7c327dd
Install plugin dependencies ()
When installing a plugin, if it contains a `PulumiPlugin.yaml` file with a `runtime` value of `nodejs` or `python`, install dependencies for the plugin.

For Node.js, `npm install` is run (or `yarn install` if `PULUMI_PREFER_YARN` is set).

For Python, a virtual environment is created and deps installed into it.
2020-09-14 13:54:26 -07:00