Commit Graph

560 Commits

Author SHA1 Message Date
Fraser Waters b8fd9966a7
When changing parents also fix URNs ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/13903
    
When we change parents in a statefile we also need to fix up the URN.
This is needed so aliases can track over partial updates.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-14 19:52:27 +00:00
Justin Van Patten 4c0b08d8ae
[engine] Check for old resources first by URN and then aliases ()
This change fixes a regression in the step generator when looking for
old resources. When generating steps for a register resource event, we
previously looked for old resources first by the resource's URN and then
by aliases.

This regressed with :

```diff
-	for _, urnOrAlias := range append([]resource.URN{urn}, goal.Aliases...) {
+	aliases[urn] = struct{}{}
+	for urnOrAlias := range aliases {
```

Previously, aliases were in a slice and we always looked for the URN
first, then aliases.

With , aliases changed to being stored in a map (a set). The URN
was added to the map before iterating over it, but there's no guarantee
it will be looked at first (iteration order for maps is unspecified),
and with the current behavior when there are aliases in the map, the URN
very likely won't come first.

This can lead to duplicate resources in the state (stack corruption)
when the wrong old resource is chosen.

The fix is to move back to always checking for old resources using the
URN first. We also move back to maintaining aliases in a slice for
consistent ordering.

Fixes 
2023-09-06 12:17:02 +00:00
Abhinav Gupta b51caa6ab4
cmdutil.ReadConsole[NoEcho]: Use bubbletea ()
Switch the cmdutil.ReadConsole and cmdutil.ReadConsoleNoEcho functions
to use the bubbletea library to render the prompt,
using the textinput widget provided by the accompanying bubbles library.
The resulting input widgets support arrow keys, back space,
and some basic readline-style bindings including Ctrl-A, Alt-B, etc.

I went through all uses of ReadConsole or ReadConsoleNoEcho.
Only the one in new.go had a non-compliant prompt that I had to adjust.

Note: One divergence in behavior I opted for was that
password prompts will echo '*' characters as the user is typing
and then no echo once they've accepted or canceled the value.
Previously, the prompt did not echo anything in either case.

<details>

  <summary>
  Introduction if you're unfamiliar with bubbletea
  </summary>

  bubbletea operates by modeling the widget state as
  an immutable data structure that receives messages for events.
  On receiving a message (key press, e.g.) the model's Update method
  returns a new model instance representing its new state.
  Update may also optionally return additional commands for the program,
  e.g. stop running, or print something and move on.
  The model's View method returns what should be drawn in the terminal
  based on the model's current state.
This programming model makes it reasonably straightforward to unit test
  some of the core functionality of independent widgets
  as demonstrated in this PR.

</details>

Resolves 

---

Demos:

<details>
  <summary>Plain text</summary>
  

![prompt-plain](https://github.com/pulumi/pulumi/assets/41730/66258fc8-f772-4d01-bc7c-1f7b116aebaa)
</details>

<details>
  <summary>Secret</summary>


![prompt-secret](https://github.com/pulumi/pulumi/assets/41730/372f862e-9186-4d47-ba7d-0107c47f52f6)
</details>

<details>
  <summary>Secret prompt with padding</summary>


![prompt-secret-2](https://github.com/pulumi/pulumi/assets/41730/e9b7c253-4c9d-4235-9fa6-197aa0522033)
</details>
2023-08-30 17:08:44 +00:00
Justin Van Patten 0650e8bbbb
[sdk/nodejs] Fix provider for resource methods ()
The `Resource` class in the Node.js SDK has the following internal
property:

```typescript
/** @internal */
readonly __prov?: ProviderResource;
```

When a resource is created, the provider specified for the resource is
stored in this property. If it is set, it is passed along in the `Call`
request when a method is called on the resource.

Prior to , the property was only set for custom resources in
`Resource`'s constructor:

```typescript
this.__prov = custom ? opts.provider : undefined;
```

With , it was changed to also store the value for remote
components:

```diff
- this.__prov = custom ? opts.provider : undefined;
+ this.__prov = custom || remote ? opts.provider : undefined;
```

This regressed the behavior when calling a method on a remote component
that had an explicit provider that wasn't the component provider, but
some other provider (e.g. AWS provider) specified as:

```typescript
const component = new MyRemoteComponent("comp", {
}, { provider: awsProvider });
```

The `awsProvider` was being stored in `Resource.__prov`, and when making
the method call on the resource, it would try to invoke `Call` on the
AWS provider, rather than calling the remote component provider's
`Call`, which resulted in an error.

Note that specifying the AWS provider using the more verbose `providers:
[awsProvider]` works around the issue.

The fix is to only set `__prov` if the provider's package is the same as
the resource's package. Otherwise, don't set it, because the user is
specifying a provider with the `provider: awsProvider` syntax as
shorthand for `providers: [awsProvider]`.

Fixes 
2023-08-30 14:49:53 +00:00
Fraser Waters ac5a629733 Warn about undefined stack outputs
Fixes https://github.com/pulumi/pulumi/issues/7376
2023-07-28 14:33:04 +01:00
Abhinav Gupta 91a079851b
deps: Upgrade google.golang.org/{genproto, grpc}
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(https://github.com/googleapis/go-genproto/issues/1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker ().

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
2023-07-27 16:24:33 -07:00
Abhinav Gupta 975eaa6220
test(regress-13301): Avoid accidental tidying
The regression test for  needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in 

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact,  accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.
2023-07-26 13:05:31 -07:00
Justin Van Patten 0081910d26 [tests] Add integration test for explicit providers for packaged components
Add integration tests for passing an explicit provider for a packaged component.

The Go test already passes. The Node and Python tests fail.
2023-07-13 07:16:56 -07:00
Abhinav Gupta 7165d1efb4
Add regression test for
Adds a regression of the bug using code provided by @phillipedwards.
The reproduction is a bit complicated because of the very specific
scenario we need to replicate here.

Additionally, we need an untidy go.mod file for this,
so we need to teach scripts/tidy.sh how to ignore files.
2023-07-11 16:22:15 -07:00
bors[bot] acad217564
Merge
13315: Always mark dynamic provider code as secret r=Frassle a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Short term workaround for https://github.com/pulumi/pulumi/issues/8265

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-07-06 18:25:09 +00:00
Abhinav Gupta f59ab49fc5
deps(go): Upgrade to grpc 1.56.1
Upgrades to gRPC Go 1.56.1 to resolve the influx of dependabot PRs.
Supersedes all dependabot PRs created for this.

Fixes CVE-2023-32731
2023-07-06 09:04:16 -07:00
Fraser Waters 4baf4a1468 Always mark dynamic provider code as secret 2023-07-06 13:50:59 +01:00
dependabot[bot] 24e44809dc
Bump semver and @pulumi/pulumi
Bumps [semver](https://github.com/npm/node-semver) and [@pulumi/pulumi](https://github.com/pulumi/pulumi/tree/HEAD/sdk/nodejs). These dependencies needed to be updated together.

Updates `semver` from 5.7.1 to 7.5.3
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v5.7.1...v7.5.3)

Updates `@pulumi/pulumi` from 3.24.1 to 3.73.0
- [Release notes](https://github.com/pulumi/pulumi/releases)
- [Changelog](https://github.com/pulumi/pulumi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pulumi/pulumi/commits/v3.73.0/sdk/nodejs)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
- dependency-name: "@pulumi/pulumi"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-30 00:59:22 +00:00
bors[bot] 11f3f1dfa7
Merge
13265: Fix isInstance methods on generated provider types in the nodejs sdks r=abhinav a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/12584

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


13316: Test duplicate outputs r=abhinav a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Regression test to cover https://github.com/pulumi/pulumi/issues/9411

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


13323: [sdk/python] Fix comment about `ROOT_STACK_RESOURCE` r=abhinav a=justinvp

Looks like this was copied from the Node.js SDK. It should be `ROOT_STACK_RESOURCE` not `rootStackResource` for Python.

13326: ci: Include Go and NodeJS hosts in unit tests r=abhinav a=abhinav

The CI test matrix generator expects every module that we want to test
to be provided explicitly.
Since the Go and Node hosts became independent Go modules,
we haven't been running their unit tests in CI.

Fortunately, the tests still pass today,
but we need to include these in the build.


13327: ci/build-binaries: Don't enable cgo for Darwin r=abhinav a=abhinav

Previously, it was necessary to enable cgo
and use the system's DNS resolver on macOS.
In Go 1.20, the `net` package was rewritten
to not use cgo at all,
so CGO_ENABLED no longer has any effect there.

In fact, ever since 389860058d,
this setting has been meaningless
because we've been building Darwin binaries
on Ubuntu machines (so cgo can't be enabled anyway).

This commit just deletes this now-unused block.


13334: ci(new workflow): Run tests on master every 12 hours r=abhinav a=abhinav

We recently started capturing coverage data for Pull Requests again
but we omitted tracking for master.

Part of the reason for this is that the our CI setup uses the same
binaries that were used for the end-to-end tests as part of the release.
And we don't want to upload the coverage-instrumented binaries for a
release.

This is fixable, but instead of making our CI set up even more complex,
this introduces a new workflow to test master periodically.

This workflow will run every 12 hours, running all tests on master,
with acceptance tests on both Windows and macOS,
track coverage for all of this, and post that to codecov.

We'll be able to track code coverage trends on master in Codecov.


Co-authored-by: Fraser Waters <fraser@pulumi.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-06-30 00:28:02 +00:00
Justin Van Patten cf1189bf2c [sdk/nodejs] Support loading package.json from parent dir
This commit changes how the Node SDK resolves package.json to recursively look up the directory tree for the file when it's not present in the cwd. The previous behavior was to only look in the current directory, and fallback to an empty object when the file could not be found.
2023-06-29 07:57:43 -07:00
Fraser Waters 04ce92c59f Test duplicate outputs
Regression test to cover https://github.com/pulumi/pulumi/issues/9411
2023-06-29 10:46:37 +01:00
Fraser Waters 4ef02a09a3 Add --secret to config set hints when needed
Fixes https://github.com/pulumi/pulumi/issues/12207

Added some tests for this, which should eventually become a matrix test
to ensure we check every language returns the expected error for this.
2023-06-22 17:25:30 +01:00
Fraser Waters baa04160ec Fix `stack rm` removing config files for the wrong project
Fixes https://github.com/pulumi/pulumi/issues/12246
2023-06-21 10:52:25 +01:00
dependabot[bot] ed5274c222
Bump github.com/cloudflare/circl
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.1.0 to 1.3.3.
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.1.0...v1.3.3)

---
updated-dependencies:
- dependency-name: github.com/cloudflare/circl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-17 06:29:31 +00:00
Abhinav Gupta 795a3d07be
fix(cli/state rename): Update parent references on rename
If a resource is renamed with `pulumi state rename`,
in addition to updating references in resources that refer to it
as a dependency,
also update references to the resource as a parent of another resource.

Testing:
Includes an integration test that reproduces the panic,
and a unit test that verifies the new behavior at a more isolated level.
To implement the integration test,
a new RunPulumiCommand method was added to ProgramTester
because we don't have other means of running `pulumi state rename`
with ProgramTest.

Resolves 
2023-06-16 13:46:35 -07:00
Justin Van Patten 59dc4be453 Add aliases/python/retype_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 8db3e05fab Add aliases/python/rename_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 200feb43c9 Add aliases/python/extract_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 322e63e0a6 Add aliases/python/adopt_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 6f0ed63622 Add aliases/nodejs/retype_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 65b646472d Add aliases/nodejs/rename_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten b6085639b4 Add aliases/nodejs/extract_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 187955b128 Add aliases/nodejs/adopt_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten a6f4815267 Add aliases/go/retype_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 6e2eadfb61 Add aliases/go/rename_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 34f4464957 Add aliases/go/extract_component_child test 2023-06-14 05:19:17 -07:00
Justin Van Patten 1a27c09e71 Add aliases/go/adopt_component_child test 2023-06-14 05:19:17 -07:00
bors[bot] e57d82fd5b
Merge
12856: Support 'pulumi:tags' config to set stack tags r=Frassle a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Support a "pulumi:tags" config property to set the stacks tags on operations (e.g. up, refresh, etc).

Fixes https://github.com/pulumi/pulumi/issues/5004

Note that I've gone with a slightly different approach to . The suggestion in that ticket was to add a new section "tags" to the yaml, but given that we probably want to allow setting tags per-project and per-stack with some way to merge them I figured it better to reuse normal config and benefit from https://github.com/pulumi/pulumi/issues/11547 when that's done.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-06-01 17:10:29 +00:00
bors[bot] ec469864ee
Merge
13021: Use os.tmpfile instead of creating our own. r=RobbieMcKinstry a=RobbieMcKinstry

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
This PR uses os.Tempfile instead of creating our own with UUID. The standard library's os.Tempfile function
is race-safe, collision-proof, and uses the OS's tempdir, which allows the OS to clean up old files if Pulumi were to 
fail to do so.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


13054: ci(lint): Helpful message on tidy failure r=abhinav a=abhinav

When the 'go mod tidy' lint check fails,
it prints a simple error message reporting the failure.

Most will resort to running `go mod tidy` manually
to attempt to fix this, but this will not always work.
We need to run `./scripts/tidy.sh` which updates all go.mod files
and includes specific compatibility flags.


Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-05-30 19:48:20 +00:00
Robbie McKinstry 73ff332f5e
Use os.tmpfile instead of creating our own.
The Node SDK creates a tmp file when packing programs, to house
the packed tarball. We previously used uuid to create our own
tmpfile with a random name. This commit uses os.tempfile instead.
The stdlib function os.tempfile is race safe, collision-proof, and
defaults to a tmp directory, which the OS is free to automatically
clean up in case Pulumi fails to.
2023-05-30 13:58:02 -04:00
Fraser Waters 92d4411327 Fix project renames for filestate 2023-05-30 13:54:28 +01:00
Pat Gavlin a8f41f031b [sdk] Update uniseg
The latest version of this module dramatically improves its allocation
volume and offers a friendlier API for measuring string width.
2023-05-25 22:24:13 -07:00
Fraser Waters 89a7922b08 Support 'pulumi:tags' config to set stack tags
Fixes https://github.com/pulumi/pulumi/issues/5004
2023-05-24 09:02:22 +01:00
bors[bot] 236452a2ff
Merge
12982: test(sdk/go): panicking programs and components should fail r=abhinav a=abhinav

Adds integration tests for the Go SDK verifying that
if we encounter a panic in a Pulumi Go program
or a Pulumi MLC's Configure step, we fail the program.

Intended to aid in root causing .


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-05-23 16:44:06 +00:00
Abhinav Gupta ed5706ddb9
lint(govet): Enable nilness linter
govet includes a nilness linter that detects a few nil issues.
This linter is not enabled by default in govet because the `go` tool
does not want a dependency on the libraries necessary to implement this
().

Enable the nilness linter and fix the following issues found by it
(commentary below each check mine):

    pkg/cmd/pulumi/new.go:283:9: nilness: impossible condition: nil != nil (govet)
        ^- the error was very likely supposed to be the `os.Remove`
    pkg/cmd/pulumi/new.go:633:10: nilness: impossible condition: nil != nil (govet)
        ^- same error checked on the previous line
    pkg/cmd/pulumi/preview.go:190:11: nilness: impossible condition: nil != nil (govet)
        ^- same error checked a few blocks above
    pkg/codegen/pcl/binder_component.go:101:64: nilness: nil dereference in dynamic method call (govet)
        ^- err is guaranteed nil
    pkg/codegen/pcl/binder_component.go:133:10: nilness: impossible condition: nil != nil (govet)
        ^- err is guaranteed nil
    sdk/go/auto/errors_test.go:374:34: nilness: nil dereference in index operation (govet)
        ^- this is intentional; I replaced it with a deliberate panic
    tests/integration/construct_component_methods_resources/testcomponent-go/random.go:30:10: nilness: impossible condition: non-nil == nil (govet)
        ^- args is rejected if it's nil
2023-05-19 15:32:00 -07:00
Abhinav Gupta 991cd16b18
test(sdk/go): panicking programs and components should fail
Adds integration tests for the Go SDK verifying that
if we encounter a panic in a Pulumi Go program
or a Pulumi MLC's Configure step, we fail the program.

Intended to aid in root causing .
2023-05-19 11:11:40 -07:00
Robbie McKinstry c500b3b997
Revert package.json lookup fix. 2023-05-15 14:14:57 -04:00
Abhinav Gupta 52a47d6295
all: cloudflare/circl 1.1.0 => 1.3.3
Upgrade version of cloudflare/circl to pick up important fixes
and supersede a bunch of dependabot PRs.

Addresses CVE-2023-1732
2023-05-11 13:51:01 -07:00
Robbie McKinstry c3e5e483ca
Update four tests to reflected the standardized "main" behavior.
The `main` field in package.json is standardized by Node to refer
to a file relative to package.json. The previous commits in this
PR used `main` relative to the Pulumi main directory, which was
incorrect. This commit upgrades two tests, and introduces two more,
to verify the correctness of this behavior and the PR in general.
2023-05-10 18:04:59 -04:00
Robbie McKinstry dad42a4954
Remove references to missing files.
It appears that a large number of tests were using references
to main fields which don't currently exist. The runtime didn't
produce an error because package.json was being ignored.
2023-05-10 18:04:59 -04:00
Robbie McKinstry e15fd85696
Test ESM module with package.json located in parent folder
This commit adds a test to assert Pulumi programs written as ESM
modules can be executed when their package.json file is located
in the parent directory.
2023-05-10 15:39:19 -04:00
Robbie McKinstry 78b21ab390
Add failing test for package.json in parent dir
This commit adds a failing test, which demonstrates
that the Pulumi NodeJS runtime cannot locate a package.json
file kept in a parent directory. It does not recursively walk
up the directory tree until it finds a package.json file.
2023-05-10 15:39:17 -04:00
Fraser Waters 1a38eadc69 gRPC for GenerateProject/Program/Package
This changes codegen to be invoked via gRPC from pkg, rather than
invoking pkg/codegen directly.

Consider it a proof-of-concept for moving codegen to a gRPC interface
without the worries of forwards-backwards compatability (because we ship
language plugins at a fixed version side-by-side to users).
2023-05-06 13:14:59 +01:00
Fraser Waters 1376a13d13 Fix destroy without project file
Fixes https://github.com/pulumi/pulumi/issues/12714.

The empty project created for destroy operations when no Pulumi.yaml is
found still needs to have it's name filled in so that project name
consistency checks work.

Our tests didn't pick this up because for filestate we were still
searching from the working directory for consistency checks rather than
checking the backends current project. I have _not_ changed that in this
changeset as previosly that triggered total breakage of the filestate
backend when not in project mode (see
https://github.com/pulumi/pulumi/issues/12760).
2023-05-02 10:36:16 +01:00
Abhinav Gupta 33b5ad6527
feat(go/host): Support vendored dependencies
The Go language host cannot resolve dependencies or plugins if a Pulumi
program vendors its dependencies.

BACKGROUND

The GetRequiredPlugins and GetProgramDependencies methods of the Go
language host rely on the following two commands:

    go list -m -mod=mod all
    go list -m -mod=mod ...
    # '...' means current module and its descendants

GetRequiredPlugins additionally searches the source directories for each
returned module for pulumi-plugin.json files at a pre-determined paths.

    $module/pulumi-plugin.json
    $module/go/pulumi-plugin.json
    $module/go/*/pulumi-plugin.json

This works for most Pulumi programs, except those that vendor private
dependencies with 'go mod vendor'.
For those programs, the above commands fail because -mod=mod forces them
to run in module mode, and their private dependencies are not accessible
in module mode (because they are not exposed publicly).

We use the -mod=mod flag to force 'go list' to run in module mode
because otherwise, it will automatically use vendor mode if a vendor
directory is present. However, in vendor mode, the two 'go list'
commands above are not supported.
The following links add more context on why, but in short:
vendor does not have enough information for the general 'go list'.

- https://stackoverflow.com/a/60660593,
- https://github.com/golang/go/issues/35589#issuecomment-554488544

In short,

- list all with -mod=mod fails because the dependency is private
- list without -mod=mod will use vendor mode
- vendor mode doesn't support the listing all

SOLUTION

Drop the -mod=mod flag so that 'go list' can decide whether to run in
module mode or vendor mode.
However, instead of running it with 'all' or '...',
pass in a list of dependencies extracted from the go.mod.

    go list -m import/path1 import/path2 # ...

This operation is completely offline in vendor mode
so it can list information about private dependencies too.

This alone isn't enough though because in vendor mode,
the JSON output does not include the module root directory.
E.g.

    % go list -mod=vendor -json -m github.com/pulumi/pulumi/sdk/v3
    {
            "Path": "github.com/pulumi/pulumi/sdk/v3",
            "Version": "v3.55.0",
            "GoVersion": "1.18"
    }

    # Versus

    % go list -mod=mod -json -m github.com/pulumi/pulumi/sdk/v3
    {
            "Path": "github.com/pulumi/pulumi/sdk/v3",
            "Version": "v3.55.0",
            "Time": "2023-02-14T11:04:22Z",
            "Dir": "[...]/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.55.0",
            "GoMod": "[...]/go/pkg/mod/cache/download/github.com/pulumi/pulumi/sdk/v3/@v/v3.55.0.mod",
            "GoVersion": "1.18"
    }

Therefore, we have to manually calculate the path for each module root.
That's easy enough: vendor/$importPath.

Lastly, since GetProgramDependencies only needs a dependency list,
it now extracts information from the go.mod without calling 'go list'.

TESTING

Adds a variant of the test added in  that verifies the
functionality with vendoring. It removes the sources for the
dependencies to simulate private dependencies. The new test fails
without the accompanying change.

The fix was further manually verified against the reproduction included
in .

    % cd go-output
    % pulumi plugin rm -a -y
    % pulumi preview
    Previewing update (abhinav):
    Downloading plugin: 15.19 MiB / 15.19 MiB [=========================] 100.00% 0s
                                                                                    [resource plugin random-4.8.2] installing
         Type                      Name               Plan
     +   pulumi:pulumi:Stack       go-output-abhinav  create
     +   └─ random:index:RandomId  rrr                create

    Resources:
        + 2 to create

    % pulumi plugin ls
    NAME    KIND      VERSION  SIZE   INSTALLED       LAST USED
    random  resource  4.8.2    33 MB  26 seconds ago  26 seconds ago

    TOTAL plugin cache size: 33 MB

Note that the version of random (4.8.2) is what's specified in the
go.mod, not the latest release (v4.12.1).

    % grep pulumi-random go.mod
            github.com/pulumi/pulumi-random/sdk/v4 v4.8.2

With the plugin downloaded, I ran this again without an internet
connection.

    % pulumi preview
    Previewing update (abhinav):
         Type                      Name               Plan
     +   pulumi:pulumi:Stack       go-output-abhinav  create
     +   └─ random:index:RandomId  rrr                create

    Resources:
        + 2 to create

This means that if the dependencies are vendored, and the plugin is
already available, we won't make additional network requests, which also
addresses .

Resolves 
Resolves 
2023-04-24 09:49:16 -07:00