Commit Graph

1276 Commits

Author SHA1 Message Date
Paul Roberts a2a63fa024 Add `blocking_call` to the pulumi python api
This allows a python program to make a blocking call (e.g.
a `requests.get` or a `subprocess.check_output`) without preventing the
main thread's event loop from pumping events.
2024-03-06 17:59:57 -08:00
Fraser Waters b34f77e74a
Update dotnet language host to 3.60 () 2024-03-06 21:16:47 +00:00
Thomas Gummerer 0f8e922c71
allow retries for encrypt/decrypt API calls ()
By default our HTTP client retries all GET requests. However these post
requests are also idempotent, as we just expect to encrypt/decrypt a
single value, without changing anything on the server side. Retry them,
so network errors won't abort the pulumi program.

There's also some log decryption events that might be retryable, but I'm
not sure about them, so I left them alone for now.

Fixes https://github.com/pulumi/pulumi/issues/15236
2024-03-06 08:35:36 +00:00
Justin Van Patten 73e2471fee
[sdk/python] Add an `invoke_async` function ()
This commit adds a new `pulumi.runtime.invoke_async` function that
allows calling invokes asynchronously. The initial intended use for this
is inside the Kubernetes Python SDK's `yaml.ConfigFile`,
`yaml.ConfigGroup`, `helm`, `kustomize`, etc. components to avoid stalls
in resource registrations which severely limits parallelism.

I'd love to add some kind of benchmark, perhaps along the lines of the
repro in , but we can add that as a fast-follow.

Part of  and 

After this is merged and released, we can ship an updated version of the
Kubernetes Python SDK that makes use of it.
2024-03-06 00:32:39 +00:00
Thomas Gummerer 898a682ef6
Make sure non-targeted resources are not updated ()
When the `--target` option is used, resources that already exist in the
snapshot, but aren't directly targeted should not be updated at all.
Internally in the engine, this is done by turning them into a
`SameStep`, meaning no updates will actually be preformed, and we will
make it look like the resource stayed the same.

However, we currently still write the "new" state of the resource (e.g.
updated dependencies, inputs, etc.) into the snapshot. This is mostly
fine as long as the new dependencies already exist. If a dependency on a
resource is that doesn't already exist is added however this breaks.
Since the resource that's being depended on doesn't exist in the
snapshot and isn't targeted, we won't create it. At the same time we're
adding a dependency on that virtually non-existing resource, which makes
the snapshot invalid.

Since we're in `--target` mode, we should do what we promised the user,
and only update the targeted resources, nothing else. Introduce a new
`NonTargetedSameStep` here, which does exactly that. It's essentially
the same as a `SameStep`, but we always use the *old* state instead of
the new one when writing it out. Since the resource is not targeted,
this leaves it in the same state as before.

Fixes 
Fixes 
2024-03-05 07:49:11 +00:00
Justin Van Patten 73dc0d673d
[cli/new] Strip sensitive data from template URLs ()
This change strips credentials and query strings from template URLs that
are saved in the generated projects from `pulumi new`.

Note: I considered a different approach where we'd only keep templates
from the default templates repo or Pulumi AI. However, the
implementation ended up being more complex to figure out what was a
default template (and not a local file path, for example), and this way
other sources are still included to help with code searches and other
tools to identify the source of a project.

Fixes 
2024-03-04 23:09:42 +00:00
Anton Tayanovskyy ee17f803aa
Schema loader made to respect PULUMI_DEBUG_PROVIDERS ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

With this change pulumi-yaml can pick up local provider and attach to it
from PULUMI_DEBUG_PROVIDERS for the purposes of schema resolution, which
enables using non-existent test-only providers. Before the change it
would fail hard trying to download it.


## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-03-04 21:54:05 +00:00
Eron Wright 8c7a9e5d3f
Support for non-overlay components in codegen for pulumi-kubernetes provider ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR enhances the codegen for dotnet to allow the pulumi-kubernetes
provider to have non-overlay components. Previously all components were
overlays and didn't require codegen.

The specific change is to not apply the `KubernetesResource` base class
when the resource is a component. It is inappropriate to use
`KubernetesResource` in this case because it extends `CustomResource`
not `ComponentResource`.

The sdkgen test suite for `kubernetes20` mode was updated with a new
example resource:
- a non-overlay component resource
([code](https://github.com/pulumi/pulumi/pull/15490/files#diff-31beec99a7baef687bd1024481d8c6e1d13b1c7f4494b417cc044fe71ad0f8cd))

The fix is manifested
[here](https://github.com/pulumi/pulumi/pull/15490/files#diff-79ca901e45591fc7db0c022cf914e99eccc0e35b2d3b1e8ee2ce45ad82faaf29R16).
## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-04 19:37:54 +00:00
Thomas Gummerer f831cb942b
set PULUMI_HOME in ProgramTests ()
Currently program test runs all programs with PULUMI_HOME unset. This
means that if programs run in parallel they share the same directory.
E.g. for login this means running in parallel only works if we always
log into the same backend, and there is no racyness between tests.

Set PULUMI_HOME to a temp directory, so each program test has its own
setup.

Fixes https://github.com/pulumi/pulumi/issues/15240

This complements https://github.com/pulumi/pulumi/pull/15559
2024-03-04 09:06:56 +00:00
Justin Van Patten 4f2373cafd
Python: Fix determining plugins for old packages ()
The Python language host determines which Pulumi plugins are required by
the program by calling out to `python -m pip list -v -format json` to
get the list of installed packages. If a package contains a
`pulumi-plugin.json` file that indicates it is for a resource plugin,
then that package is considered a Pulumi package that has an associated
plugin.

As a fallback for older packages that do not have a `pulumi-plugin.json`
file, if the package name is prefixed with `pulumi-`, it is also
considered to have an associated Pulumi plugin.

However, this fallback no longer works when using recent versions of
setuptools because of a behavior change in v69.0.3 and later of
setuptools. Previously, underscores were replaced by dashes. So `python
-m pip list --format json` would return the name of the package as:

```
{"name": "pulumi-cloudinit", "version": "1.3.0"}
```

But now with setuptools v69.0.3 and later, it returns the name as
pulumi_cloudinit (with an underscore rather than dash):

```
{"name": "pulumi_cloudinit", "version": "1.3.0"}
```

The underscore in the name is now preserved and the name is no longer
prefixed with pulumi-, so the langhost no longer considers it to have an
associated Pulumi plugin.

This commit fixes the fallback to consider packages prefixed by either
`pulumi_` or `pulumi-` to be Pulumi packages.

Fixes 
2024-03-04 07:23:34 +00:00
Eron Wright 1c4393b30f
[dotnet] codegen fix for resources without constant input properties ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This PR fixes a bug in the dotnet codegen where the wrong "args" class
name would be used, in the case that the resource has no constant
inputs. This is an edge case because most resources do have at least one
such input.

For example, a new resource definition in p/k produced this output:
```csharp
namespace Pulumi.Kubernetes.Yaml.V2
{
        public ConfigGroup(string name, Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs? args = null, CustomResourceOptions? options = null)
            : base("kubernetes:yaml/v2:ConfigGroup", name, args ?? new ConfigGroupArgs(), MakeResourceOptions(options, ""), remote: true)
        {
        }
}
```
Which doesn't compile because `ConfigGroupArgs` is in a separate
namespace.

Should be:
```csharp
        public ConfigGroup(string name, Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs? args = null, CustomResourceOptions? options = null)
            : base("kubernetes:yaml/v2:ConfigGroup", name, args ?? new Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs(), MakeResourceOptions(options, ""), remote: true)
        {
        }
```

[Here's
](https://github.com/pulumi/pulumi/pull/15488/files#diff-18b12fabab20d68398aced2890b1ca3073cc32081bb62a022b77a5090c209e3bR45)where
the fix manifests itself in the new test case.

## Testing

A new SDK test case was added to cover the whole `kubernetes20`
compatibility mode, based on a simplified schema from the
pulumi-kubernetes provider.

The schema contains a representative set of resources:
1. `kubernetes:core/v1:ConfigMap` - a non-overlay resource representing
a Kubernetes kind.
2. `kubernetes:core/v1:ConfigMapList` - a Kubernetes list kind
3. `kubernetes:helm.sh/v3:Release` - a non-overlay, non-Kubernetes
resource
4. `kubernetes:yaml:ConfigGroup` - an overlay component resource

An important detail is whether a resource has any input properties that
have a constant value, such as we see with `kind` and `apiVersion`. The
`Release` resource intentionally has no such constant inputs.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-02 06:01:31 +00:00
Justin Van Patten 91aa685304 chore: post-release go.mod and changelog updates for v3.108.0 2024-02-29 21:48:42 -08:00
Julien P 5ff35273d6
Fix merge failures ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15528

See https://github.com/pulumi/pulumi/pull/15540 &
https://github.com/pulumi/pulumi/pull/15531

Re-creating this as a PR with `ci/test` label so we can get it merged.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-29 21:06:24 +00:00
Thomas Gummerer 3b4715ae1b
push post-release changelog and go.mod updates straight to master ()
Post release, we need to update the changelog (clear out all the pending
entries), and update the go.mod in pkg, and also tag the latest version
of that. Currently we create a new PR for that. However since we are
using squash merges, and need to do the tagging while creating the PR,
we're tagging a commit that doesn't end up on the main branch. This
means the Go tooling doesn't work properly with pkg. See also
https://github.com/pulumi/pulumi/issues/15458.

There is nothing really we are doing with that PR other than merging it
(in fact queue-merge is set to true, so it should be queued and merged
automatically, but that doesn't seem to work).

Because of that, we can just tag the release, and push it straight to
the main branch instead. This removes one human step from the release
process and makes the Go tooling work correctly.

The only potential downsides of this change are:
- a small race condition, if a commit gets merged just before we do the
rebase we might end up failing that and the release workflow fails. This
is very unlikely, since the release process should be relatively quick.
- we need to adjust the branch protection rules for pulumi-bot so it's
able to push to the main branch directly. This should not be a problem,
since it's an automated tool, so all of its potential activity has been
reviewed by humans beforehand.

Curious especially if @justinvp has any thoughts on this, since he's
mostly taking care of the release process.

Fixes https://github.com/pulumi/pulumi/issues/15458
2024-02-27 15:41:35 +00:00
Fraser Waters ae161d6758
Go SDK transform support ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->


This adds a new experimental feature to the Go SDK to register remote
transform functions. These are currently all prefixed 'X' to show
they're experimental (they can't be in their own package because of
circular dependencies).

These transform functions will run even for resources created inside
MLCs.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-27 13:00:45 +00:00
Ian Wahbe fe74e0d4f4
Exit no-op `pulumi state edit` ()
The current behavior of `pulumi state edit` when the edited file is
unchanged is to prompt the user to `accept`, `edit`, `reset` or
`cancel`. Since the file is unchanged, `accept` and `cancel` do the same
thing. `edit` (again) is the same as `reset` which is the same as
running the command from scratch.

The feature would be much easier to use if a no-op edit simply exited.
If the user wants to edit again, they can just run `!!` to repeat the
command.

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [X] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-27 12:44:07 +00:00
Julien P be0fe6b1a6
Replace deprecated read-package-tree with @npmcli/arborist ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

read-package-tree is deprecated. Additionally it has a dependency that
is flagged by security scanners.

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/9129

Ref https://github.com/pulumi/pulumi/issues/12688

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-26 18:40:28 +00:00
Ringo De Smet d513dea5ee
Use `SYSTEM_PULLREQUEST_SOURCECOMMITID` for AZ pipeline on GH PR ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Use `SYSTEM_PULLREQUEST_SOURCECOMMITID` when AZ pipelines checks against
a Github PR. Fix made based on investigation from @blampe in 

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-26 14:55:54 +00:00
Paul C. Roberts fdd2d8eb39
Add vim shortcuts and Home and End Keys to terminal ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Updates the terminal support for the tree view in Pulumi CLI

This adds the vim shortcuts :
`j` &ndash; down
`k` &ndash; up
`Ctrl+F` &ndash; page down 
`Ctrl+B` &ndash; page up
`g` &ndash; home
`G` &ndash; end

Also adds Home and End key support

Fixes # (issue)

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

Co-authored-by: Paul Roberts <proberts@pulumi.com>
2024-02-24 08:31:36 +00:00
Guinevere Saenger 26bdac7438
Add ability to process docs without dedicated Examples logic ()
This pull request depends on corresponding changes in the
pulumi-terraform bridge.

The bridge will mark up each code section it finds (not just ones found
under an "Example Usage" header) with an HTML comment - currently this
will be `<!--Begin Code Chooser -->` and `<!--End Code Chooser -->` but
the name is subject to change.

In this PR, we are adding logic to detect if an incoming Description
block has an `{{% examples }}` shortcode markup: if yes, we will process
it as before. If no, we will switch to new functionality that relies on
documentation with code sections having no short codes, no "Examples
Section" logic, and will generate every code section it finds in an
incoming schema that is in the Description/Content section of a registry
page with a chooser and language choosables.

This work is part of https://github.com/pulumi/pulumi-aws/issues/2624.


Edit: unit tests are added

This PR is a prerequisite for
https://github.com/pulumi/pulumi-terraform-bridge/pull/1689. It aims to
support both legacy and new behavior.



<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-23 18:06:44 +00:00
Julien P af8f82a1bc
Upgrade to grpc-js 1.10.1 and remove calls to deprecated server.start ()
# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15499

This should be safe, as we are effectively already running 1.10.1. This
makes this the new minimum so we can remove the `server.start` calls
(the server is started in `server.bindAsync`) that cause deprecation
warnings to be printed in Automation API or when using dynamic providers
with nodejs.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-23 11:32:40 +00:00
Fraser Waters 485debf7da
Revert "Warn about non-project mode ()" ()
This reverts commit 038ea95d40.

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2024-02-23 07:52:44 +00:00
Fraser Waters e42cfbb349
Ensure project plugins are absolute paths ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15467.

This tightens the restriction on paths passed to `NewProgramInfo`.
Previously it allowed relative paths like "./providers/my_provider".
That is now an error. This is correct behaviour. The fields of this
structure are passed via protobuf and the descriptions for them in the
proto spec are that they should always be absolute paths.

Where we build plugin paths we ensure that if they're relative we
resolve them to what they are relative to. That is generally _not_ the
current working directory so `filepath.Abs` doesn't do the right thing
here.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-22 11:43:18 +00:00
Julien P 87b01bf35a
Handle workspaces in computeCodePaths ()
# Description

When we compute the code paths for function serialisation, we are not
handling yarn/npm workspace setups where the node_modules might be in a
different directory.

Fixes 
Fixes https://github.com/pulumi/pulumi/issues/2661
Fixes https://github.com/pulumi/pulumi/issues/7168
Fixes https://github.com/pulumi/pulumi/issues/3013

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-22 11:41:37 +00:00
Pulumi Bot e654626abf
Changelog and go.mod updates for v3.107.0 ()
Co-authored-by: github-actions <github-actions@github.com>
2024-02-22 00:35:07 +00:00
Fraser Waters 7422c44ca4
Engine support for remote transforms ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This adds support to the engine for "remote transformations".

A transform is "remote" because it is being invoked via the engine on
receiving a resource registration, rather than being ran locally in
process before sending a resource registration. These transforms can
also span multiple process boundaries, e.g. a transform function in a
user program, then a transform function in a component library, both
running for a resource registered by another component library.

The underlying new feature here is the idea of a `Callback`. The
expectation is we're going to use callbacks for multiple features so
these are _not_ defined in terms of transformations. A callback is an
untyped byte array (usually will be a protobuf message), plus an address
to define which server should be invoked to do the callback, and a token
to identify it.

A language sdk can start up and serve a `Callbacks` service, keep a
mapping of tokens to in-process functions (currently just using UUID's
for this), and then pass that service address and token to the engine to
be invoked later on.

The engine uses these callbacks to track transformations callbacks per
resource, and on a new resource registrations invokes each relevant
callback with the resource properties and options, having new properties
and options returned that are then passed to the next relevant transform
callback until all have been called and the engine has the final
resource state and options to use.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 16:30:46 +00:00
Komal d30112ac55
Add an --open flag to config ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14770

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 16:22:32 +00:00
Julien P 2a2d247755
Use pnpm as package manager if we find a pnpm-lock.yaml file ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Initial support pnpm. Note that this does not support pnpm workspaces
yet.

This also does not handle passing the package manager through from
`pulumi new`. Once a user manually runs pnpm, creating a pnpm-lock.yaml,
we'll detect that and pnpm.

Fixes https://github.com/pulumi/pulumi/issues/15455

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 13:41:21 +00:00
Thomas Gummerer 14e78f5598
Enable program test to install dev releases of SDKs ()
# Description

Add a new option `InstallDevReleases` to `ProgramTestOptions`, which
will tell ProgramTest to upgrade each pulumi dependency to the latest
dev release. This includes the pulumi/pulumi SDKs and the provider SDKs.
The idea is to use this in the `pulumi/examples` and `pulumi/templates`
repositories to get pre release testing there.

Fixes https://github.com/pulumi/pulumi/issues/15389

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 13:07:29 +00:00
Fraser Waters ea74b8aa26
Ensure ArgsDependencies are filled in ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Adds argsDependencies support to the deploytest resource monitor and add
some tests that the engine fills that map in with output values from
args if the SDK hasn't filled it in.

We also need to do this the other way (if the SDK fills in
argsDependencies but doesn't send output<T> values with those deps to
args).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-21 09:15:38 +00:00
Zaid Ajaj 3b9f5c09b7
[program-gen] Fix stack overflow when binding invoke that resolves to promise ()
# Description


Fixes  because using `ContainsPromises` doesn't account for
recursive object references where as `ContainsEventuals` does.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-20 15:49:08 +00:00
Zaid Ajaj c5ae74a74e
[program-gen] Emit Output-returning JSON serialization methods without rewriting applies ()
### Description

A while ago we started implementing [specialized JSON serialization
methods](https://github.com/pulumi/pulumi/issues/12519) for Pulumi
programs which can accept nested outputs without having to rewrite and
combine applies.
 - `Output.SerializeJson` in .NET
 - `pulumi.jsonStringify` in nodejs
 - `pulumi.Output.json_dumps` in Python

This PR extends program-gen for TypeScript, C# and Python to start
emitting these JSON serialization functions (when necessary). The PR
special-cases the `toJSON` PCL function when rewriting applies so that
nested outputs aren't rewritted.

Example PCL program and generated results:

> Also check out the downstream codegen tests to see improved generated
examples

```
resource vpc "aws:ec2:Vpc" {
	cidrBlock = "10.100.0.0/16"
	instanceTenancy = "default"
}

resource policy "aws:iam/policy:Policy" {
	description = "test"
	policy = toJSON({
		"Version" = "2012-10-17"
		"Interpolated" = "arn:${vpc.arn}:value"
		"Value" = vpc.id
	})
}
```


### Generated TypeScript Before
```typescript
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("vpc", {
    cidrBlock: "10.100.0.0/16",
    instanceTenancy: "default",
});
const policy = new aws.iam.Policy("policy", {
    description: "test",
    policy: pulumi.all([vpc.arn, vpc.id]).apply(([arn, id]) => JSON.stringify({
        Version: "2012-10-17",
        Interpolated: `arn:${arn}:value`,
        Value: id,
    })),
});
```

### Generated TypeScript After
```typescript
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("vpc", {
    cidrBlock: "10.100.0.0/16",
    instanceTenancy: "default",
});
const policy = new aws.iam.Policy("policy", {
    description: "test",
    policy: pulumi.jsonStringify({
        Version: "2012-10-17",
        Interpolated: pulumi.interpolate`arn:${vpc.arn}:value`,
        Value: vpc.id,
    }),
});
```
### Generated Python Before
```python
import pulumi
import json
import pulumi_aws as aws

vpc = aws.ec2.Vpc("vpc",
    cidr_block="10.100.0.0/16",
    instance_tenancy="default")
policy = aws.iam.Policy("policy",
    description="test",
    policy=pulumi.Output.all(vpc.arn, vpc.id).apply(lambda arn, id: json.dumps({
        "Version": "2012-10-17",
        "Interpolated": f"arn:{arn}:value",
        "Value": id,
    })))
```

### Generated Python After
```python
import pulumi
import json
import pulumi_aws as aws

vpc = aws.ec2.Vpc("vpc",
    cidr_block="10.100.0.0/16",
    instance_tenancy="default")
policy = aws.iam.Policy("policy",
    description="test",
    policy=pulumi.Output.json_dumps({
        "Version": "2012-10-17",
        "Interpolated": vpc.arn.apply(lambda arn: f"arn:{arn}:value"),
        "Value": vpc.id,
    }))
```

### Generated C# Before
```csharp
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() => 
{
    var vpc = new Aws.Ec2.Vpc("vpc", new()
    {
        CidrBlock = "10.100.0.0/16",
        InstanceTenancy = "default",
    });

    var policy = new Aws.Iam.Policy("policy", new()
    {
        Description = "test",
        PolicyDocument = Output.Tuple(vpc.Arn, vpc.Id).Apply(values =>
        {
            var arn = values.Item1;
            var id = values.Item2;
            return JsonSerializer.Serialize(new Dictionary<string, object?>
            {
                ["Version"] = "2012-10-17",
                ["Interpolated"] = $"arn:{arn}:value",
                ["Value"] = id,
            });
        }),
    });

});
```

### Generated C# After
```csharp
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() => 
{
    var vpc = new Aws.Ec2.Vpc("vpc", new()
    {
        CidrBlock = "10.100.0.0/16",
        InstanceTenancy = "default",
    });

    var policy = new Aws.Iam.Policy("policy", new()
    {
        Description = "test",
        PolicyDocument = Output.JsonSerialize(Output.Create(new Dictionary<string, object?>
        {
            ["Version"] = "2012-10-17",
            ["Interpolated"] = vpc.Arn.Apply(arn => $"arn:{arn}:value"),
            ["Value"] = vpc.Id,
        })),
    });

});
```

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-20 15:48:46 +00:00
Anton Tayanovskyy 5f8ca933d4
Fix docs generator parent module computation ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Fixes 
Fixes 

Fixes a bug in docs generator parent module computation that used to
lead to duplicate files panics and undesirable documentation URL layout
for affected providers such as
[pulumi/fortios](https://github.com/pulumiverse/pulumi-fortios/issues).

Joint work with @tmeckel .

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Meckel <tmeckel@users.noreply.github.com>
2024-02-20 15:44:37 +00:00
Komal b2d3f5f7ee
[go] Automation API support for `pulumi refresh --preview-only` ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Adds Go automation API support for `pulumi refresh --preview-only`. 

Depends on  

Fixes: 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-20 02:59:35 +00:00
Julien P 0d3d279bdc
Detect npm and yarn workspaces setups ()
# Description

Make `pulumi install` detect npm and yarn workspaces setups and
successfully install dependencies.

Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`
2024-02-16 08:25:12 +00:00
Pulumi Bot def21b977e
Changelog and go.mod updates for v3.106.0 ()
Co-authored-by: github-actions <github-actions@github.com>
2024-02-16 04:14:52 +00:00
Thomas Gummerer cf9562c6ae
Publish dev versions of the python and node SDKs ()
Publish dev versions of the python and node SDKs, so we can consume them
pre-release and make our releases more stable.

Also clean up the publish_npm.sh script a little bit, removing now
unused cruft.
2024-02-15 16:25:57 +00:00
Fraser Waters 60f1abc2b8
Fix TestDestroySetsEncryptionsalt test and resulting bug ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

While working on some other secret manager changes I found that
`TestDestroySetsEncryptedkey` wasn't actually fully testing what we
thought it was.

Firstly it was a bad name, we're checking for `encryptionsalt` being set
not `encryptedkey`. But more importantly it wasn't checking that the
salt stayed the same.

Turned out `destroy` was loading the stack config, seeing no
`encryptionsalt` and so new'ing up a brand new passphrase secret manager
and state and then saving that to the stack config.

This is now fixed that the test asserts that the salt is exactly what's
expected and I've fixed up the engine code to do this correctly.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-15 09:33:27 +00:00
Fraser Waters 556cd9b6cd
Allow component providers to return outputs to the engine ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This adds a new flag to provider Construct and Call requests to indicate
that the engine will accept output values in the response.

If a component provider sees that flag set they can return Output
PropertyValues and optionally skip filling in the dependency map in the
response.

The engine will use the combined information of the dependency map and
any dependencies found on any output values in the property to build the
final dependency map that is returned to programs.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-14 08:15:24 +00:00
Fraser Waters 038ea95d40
Warn about non-project mode ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

In preparation for deprecating non-project mode at some point. This is
phase 1 of encouraging users off of legacy mode.

Any time a user opens a backend in legacy mode (either old states, or
new states with DIY_BACKEND_LEGACY_LAYOUT set) we'll warn that
non-project mode is due to deprecate this year.

They can disable this warning by setting
PULUMI_DIY_BACKEND_ACKNOWLEDGE_LEGACY_WARNING to true.

The next phase (in a few months) will turn this into an error which
they'll be able to opt out of by setting
PULUMI_DIY_BACKEND_ACKNOWLEDGE_LEGACY_ERROR to true.

After that it will be a hard error and all non-project related code will
be removed.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-13 08:25:24 +00:00
Zaid Ajaj ed79536f65
[program-gen] Emit missing trivia for resources and local variables ()
# Description

For C#, Python and TypeScript: emit missing comments for local variable
declaration
For Go: emit missing comments for local variables and resources


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-11 17:02:12 +00:00
Fraser Waters f2b726d1c7
Program dependencies aren't necessarily semantic versions ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This weakens the contract of GetProgramDependencies that the versions
returned are just "strings". The format of those strings is defined by
the language plugin. This is especially useful for python which uses
PEP440 not semantic versions for it's pip dependencies.

There wasn't anywhere in the system that took a strong dependency on
these being semvers.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-09 22:18:42 +00:00
Fraser Waters ff6404a55f
Update engine test to use GRPC and fix issues ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

So it turned out that the server code for providers was set up to never
return output values. I don't think any actual providers used this code
but it meant the tests running in GRPC mode started to fail.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-08 16:06:40 +00:00
Fraser Waters 65b8f5c841
Split CallRequest into ResourceCallRequest ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Similar to what we did to the `InvokeRequest` a while ago. We're
currently using the same protobuf structure for `Provider.Call` and
`ResourceMonitor.Call` despite different field sets being filled in for
each of them.

This splits the structure into `CallRequest` for providers and
`ResourceCallRequest` for the resource monitor. A number of fields in
each are removed and marked reserved with a comment explaining why.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-08 13:16:23 +00:00
Pulumi Bot 7d03ac7174
Changelog and go.mod updates for v3.105.0 ()
Co-authored-by: github-actions <github-actions@github.com>
2024-02-08 08:59:28 +00:00
Fraser Waters 3cece090d1
Fix context in state edit command ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15345.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-07 09:01:56 +00:00
Fraser Waters 1138fdb63c
Upgrade output values in source_eval ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Stage 1 of updating how we deal with OutputValues in source_eval for
transforms.

This makes two changes that should result in basically a no-op for
standard resources, but update all Computed/Secret values to
OutputValues for Call and Construct.

This is a no-op for standard resources because previously we set
"KeepOutputs" to false when we unmarshalled them. That replaced all
OutputValues with Computed/Secret. Now we do "UpdgradeOutputs" at the
start of source_eval but then call "DowngradeOutputs" before passing the
properties on to the rest of the system. So this is a complete no-op.

But for Call/Construct we used to use "KeepOutputs" but now we do
"UpgradeOutputs". So values that previously got sent as Computed/Secret
will now get sent as OutputValue. This _should_ be fine, all users of
Construct/Call already had to handle OutputValue on their interface, so
this just means a few more cases of seeing those values.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-06 16:46:31 +00:00
Justin Van Patten e13780c0bd
[sdk/python] Upgrade to grpcio 1.60.1 ()
grpcio 1.60.1 includes a fix for the regression that results in a
traceback being written to stderr for unimplemented gRPC methods.
https://github.com/grpc/grpc/releases/tag/v1.60.1

This change upgrades to 1.60.1 and reverts the workarounds we put in
place in https://github.com/pulumi/pulumi/pull/15190.

Note that we have an integration test for Python dynamic providers that
ensures nothing unexpected is written to stderr:
16a1fb31c2/tests/integration/integration_python_acceptance_test.go (L111-L117)

Fixes 
2024-02-05 19:49:08 +00:00
Thomas Gummerer 6f41d715d9
State: fix panic when deleting non-unique Provider ()
# Description

In
30f59eb30a,
we made sure that we
don't delete multiple resources, when a user is prompted to choose
between multiple resources with the same URN.

If there are non-unique URNs, we currently assume that we can just go
ahead and delete it, as there's still another resource that will
fulfill the requirements.

However that is not true if the dependent is a provider.  Providers
are identified by a {URN, ID} tuple.  Therefore even though after the
deletion of a provider with a specific URN there's still another one
with the same URN left, this is not good enough to fulfill the
dependency requirements.

If we don't have a unique URN, check again to make sure there's no
provider dependency on the condemned resource.  If there is one, we
either error out, or delete the dependencies depending on the options
passed in.

Note that I haven't managed to reproduce this by just issuing pulumi
commands, but I did by editing the state in a way to reproduce the same
issue.

One unanswered question for me is how the deleted cluster in the
original issue ended up lingering in the state. While this PR fixes the
panic, and will make it easier for the user to actually delete the
cluster from the state (it's going to suggest `--target-dependents`
which will do the right thing), it doesn't address that bit.

Fixes https://github.com/pulumi/pulumi/issues/15166

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-02-05 16:21:38 +00:00
Thomas Gummerer b76b072434
add no_tmp_dir query parameter to file:// URLs ()
In gocloud.dev 0.34.0 the behaviour for file:// URLs changed, making
pulumi fail when the state is supposed to be written to a mounted
directory. See also https://github.com/pulumi/pulumi/issues/15352.

Restore the old behaviour by adding a `no_tmp_dir=true` flag to the URL,
unless the user provided it themselves. We also allow users to pass
`no_tmp_dir=false` in the parameters, which will opt in to the new
behaviour.

This actually implements option 2 from the linked issue since it was
easy enough to do, though I don't expect may users to make use of it,
since I don't think a lot of people care about the contents of whatever
directory the state is stored in anyway, so additional temp files there
don't really matter.

Fixes https://github.com/pulumi/pulumi/issues/15352

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2024-02-05 13:40:31 +00:00