Commit Graph

1003 Commits

Author SHA1 Message Date
Luke Hoban f1e4b4ff94
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes ()
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:

* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.

In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.

The implementation covers the following:

* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.

Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.

For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.

### Notes

- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.

### Fixes

* Fixes 
* Fixes 
* Fixes  
* Not quite , but likely replaces the need for that

Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
Mikhail Shilkov 96780a6d84
Refine the error message when a full stack name is not supplied ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Refine the error message when a full stack name is not supplied outside
a project directory

Context:

1. Open a brand-new clean directory that is not a child directory to any
Pulumi program.
2. Run `pulumi stack init dev`
3. CLI doesn't have a project name at this point, because it has not
Pulumi.yaml. So, the user needs to supply the full name.

Fixes https://github.com/pulumi/pulumi/issues/15967

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-03 16:38:25 +00:00
Julien P 578e0937a9
[Python] Move existing dependency installation and python command invocation to a Toolchain interface ()
# Description

This PR refactors the existing Python dependency installation and
command running code to use the `Toolchain` interface. This will make it
possible to swap out the default Pip based toolchain for a Poetry based
toolchain.

Fixes https://github.com/pulumi/pulumi/issues/16285
Ref https://github.com/pulumi/pulumi/issues/15937

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`
2024-06-03 13:52:27 +00:00
Thomas Gummerer 7ab7f00ced
Improve error handling when downloading policy packs ()
The HTTP client in Go only returns an error if there is a protocol level
failure, or an error caused by client policy (which we don't use).
However the policy pack can fail to be downloaded, and return a non 200
HTTP status. This should still be considered an error.

Currently we would only detect this when trying to unpack the policy
pack. Catching and returning this error earlier allows us to give better
information about the failure to the user.

I got curious about https://github.com/pulumi/pulumi/issues/16275, and
did a short dive into the code. It looks like this should give us a
better error message there.
2024-06-03 09:34:55 +00:00
Will Jones 98d7246ea8
Don't rewrite step operations following failure ()
When displaying the progress of a Pulumi operation to the user, we want
the operation being displayed to reflect what is actually happening at
that moment in time. Most of the time, this means "just display the
operation in question" -- if a `create` is being executed, show
"creating", if a `delete` just completed, show "deleted", and so on.
However, there are cases where we can do better than just displaying the
"raw" operation. Specifically, our "replacement-like" operations
comprise a _series_ of steps that must execute for the operation as a
whole to make sense. For create-before-replace, we have:

* `create replacement` resource
* `replace` the old resource
* `delete original` resource

Other sequences, such as delete-before-replace, are similar (in the case
of delete-before-replace, the `delete original` step comes first).

While it might make sense to display the underlying steps as the
operation progresses, when the series of steps has _completed_, it's
(arguably) much clearer to simply render the string `replaced` so that
the user knows what has gone on. Similarly, during a preview, it (again
arguably) makes more sense for us to state that the intention is to
`replace`, rather than any one of `create replacement`/`replace`/`delete
original` and so on.

Alas, there is a case where this is potentially misleading and thus
undesirable behaviour. If an _error_ occurs during execution, the
operation will terminate at the next opportunity. In doing so, it will
enter a "done" state. At this point, we _do not_ want to rewrite the
step that was actually happening before the error interrupted it (e.g.
`create replacement`) with the "end" state (e.g. `replaced`), since the
error may mean we never reached that desired state. We want the display
to be as true to the raw series of steps as possible. This PR implements
this change, so that programs which terminate due to errors do not
rewrite their steps.

This PR addresses some of the confusion in , in which we
incorrectly reported that a delete-before-replace resource had been
`replaced` when in fact we had only completed the deletion before being
interrupted by an error elsewhere.
2024-05-31 10:48:07 +00:00
Pat Gavlin 7273bc02c9
[display] Enable WASM compilation ()
These changes contain some minor refactorings to conditionally disable
the use of packages that are cannot be built for `GOOS=js GOARCH=wasm`.
With these edits, `pkg/display` can be built targeting WASM.

These changes act as a safeguard to ensure that we are not adding
additional code that will _prevent_ building `pkg/display` for WASM
targets. They are not sufficient to produce a version of the display
renderer that is appropriate for actual use in a WASM environment:

- The current renderer API is not well-suited for use outside the
context of the CLI
- The current event stream format has no versioning data
- Actually building this code into a WASM module results in an
unpleasantly large file (70M uncompressed, 13M gzipped)

These changes also add a size gate for the built WASM module. The gate
is set to the 110% of the size of the WASM module as of this commit. Our
goal is to lower the size of the WASM module over time; as we do so we
will tighten this gate.

Part of .
2024-05-24 20:27:56 +00:00
Will Jones 41b967d451
Rewrite `DeletedWith` properties when renaming stacks ()
When renaming a stack, we need to rewrite URN references to use the new
stack name instead of the old one. Presently such references can occur
as:

* `Parent` references
* `Provider` references
* `Dependencies`
* `PropertyDependencies`
* `DeletedWith` references

The current renaming implementation fails to rewrite `DeletedWith`
references. Previously this went unnoticed since such references were
also omitted when checking snapshot integrity. Now that integrity checks
correctly inspect `DeletedWith` properties, the omission during rename
has been surfaced. This commit fixes renaming to handle `DeletedWith`
correctly. A follow-up should also ensure that renaming checks integrity
itself prior to operation completion.

Fixes 
2024-05-16 15:09:32 +00:00
Thomas Gummerer 873626854c
remove extraneous `ContinueOnError` field ()
This field was something I tried to use during development, but then
decided to only use it in the engine options. Remove it to avoid further
confusion.
2024-05-13 09:24:34 +00:00
Thomas Gummerer fc10da33d9
Add display to the engine tests ()
We want to add more test coverage to the display code. The best way to
do that is to add it to the engine tests, that already cover most of the
pulumi functionality.

It's probably not really possible to review all of the output, but at
least it gives us a baseline, which we can work with.

There's a couple of tests that are flaky for reasons I don't quite
understand yet. I marked them as to skip and we can look at them later.
I'd rather get in the baseline tests sooner, rather than spending a
bunch of time looking at that. The output differences also seem very
minor, so not super concerning.

The biggest remaining issue is that this doesn't interact well with the
Chdir we're doing in the engine. We could either pass the CWD through,
or just try to get rid of that Chdir. So this should only be merged
after https://github.com/pulumi/pulumi/pull/15607.

I've tried to split this into a few commits, separating out adding the
testdata, so it's hopefully a little easier to review, even though the
PR is still quite large.

One other thing to note is that we're comparing that the output has all
the same lines, and not that it is exactly the same. Because of how the
engine is implemented, there's a bunch of race conditions otherwise,
that would make us have to skip a bunch of tests, just because e.g.
resource A is sometimes deleted before resource B and sometimes it's the
other way around.

The biggest downside of that is that running with `PULUMI_ACCEPT` will
produce a diff even when there are no changes. Hopefully we won't have
to run that way too often though, so it might not be a huge issue?

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-05-13 07:18:25 +00:00
Fraser Waters dcf0a1024b
Revert "Revert "Run integration tests and dev builds with race detection" ()" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This reverts commit 75340dd942.

Fixes https://github.com/pulumi/pulumi/issues/16018.

This re-enables the locking and race detection. The locking is more
finely scoped to not be held over provider methods like Read/Update.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-09 16:15:41 +00:00
Justin Van Patten 82c546d873
Fix mangled diffs of strings containing url-encoded chars ()
Fix mangled diffs of strings containing url-encoded chars

Currently, modifying a stack output that contains url-encoded characters
can lead to it showing the value as `"!(NOVERB)"` or `"!(MISSING)"`.

For example, changing a stack output `path` from `"%2F"` to `"%2f"`
shows the diff as:

```
Outputs:
  ~ path: "%!(NOVERB)F" => "%!(NOVERB)f"
```

The expected diff output should be:

```
Outputs:
  ~ path: "%2F" => "%2f"
```

This is happening because the code that emits parts of the diff is
expecting a format string. Go is trying to parse the `%` in the value as
a format string and is having problems parsing it.

The fix is to emit the parts of the diff verbatim, so it isn't treated
as a format string.

Fixes 

Note: I've done an audit of the other calls to `write` in
`object_diff.go` and all the other call sites are passing actual format
strings or constants. This was the only case where we were passing user
input as a format string.
2024-05-08 08:23:40 +00:00
Paul C. Roberts 95f06deccd
Revert "The `--expect-no-changes` flag checks for output diffs" ()
Reverts pulumi/pulumi#15903

This is implicated in Issue  so reverting to allow time to provide
a proper solution

Fixes 
2024-05-06 17:34:24 +00:00
Thomas Gummerer 14680cd588
fix deadlock in stopwatch mutex ()
The `m` mutex deadlocks in the display code. Split it up into an event
mutex that works as it did before
https://github.com/pulumi/pulumi/pull/16101, and add a separate mutex
for the stopwatch, to avoid any deadlocks here.


Fixes https://github.com/pulumi/pulumi/issues/16127
2024-05-06 16:28:18 +00:00
Thomas Gummerer 7b0355bc35
reenable parallelism in httpstate snapshot_test ()
We had to remove the parallelism here because these tests were flaky
because the engine Chdir'd to the plugin directory. (See also
https://github.com/pulumi/pulumi/issues/15461).

However we have since removed this Chdir, so these tests should no
longer be flaky even when run in parallel.

Fixes https://github.com/pulumi/pulumi/issues/15461
2024-05-03 11:49:14 +00:00
Will Jones f029bb923b
Lock access when writing event timings in display ()
This should fix , in which we are getting concurrent reads and
writes to the `opStopwatch` maps.
2024-05-03 11:17:06 +00:00
Fraser Waters dc12644fe4
Never show property diffs for OpSame ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15944.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-05-02 12:28:43 +00:00
Paul C. Roberts b72399bd28
The `--expect-no-changes` flag checks for output diffs ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a pseudo op `OpOutputChange` to the set of changes that are
recorded in `display.ResourceChanges` to count the number of output
changes, this is then included in the check used to evaluate the
`--expect-no-changes` flag

When resource outputs are registered, they are checked against their
previous value using existing functionality, the total count of changes
is then added

The internal capability is validated with an engine test, the cli is
validated using an integration test

This will break user workflows that depend on the previous behavior

Fixes 

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Paul Roberts <proberts@pulumi.com>
2024-05-01 18:30:49 +00:00
Will Jones a2b227695c
Report snapshot write errors as actual errors ()
When we finish an operation and close the snapshot manager, the snapshot
manager may perform a final write if there are unflushed changes (e.g.
refresh changes whose writes were elided at the time). Things may go
wrong at this point -- if something is wrong with the snapshot, the
integrity check following this write could fail; if a write error
occurs, pieces of the snapshot may be missing. In these cases, we really
don't want to swallow the error or report it as "just" a diagnostic,
because it's a real problem that will almost certainly rear its head on
the next Pulumi invocation when we attempt (and fail) to read the
snapshot (or worse, operate with an incomplete one). This commit changes
the existing flow to report all snapshot close errors as bonafide
failures, even if the actual operation has succeeded and been applied.
2024-04-30 17:23:18 +00:00
Thomas Gummerer f3f0b19b33
be consistent in the way we round seconds ()
In some cases we only display full seconds, particularly when rendering
the summary event, and when converting the engine event for use in the
API. However we are currently not consistent with that, rounding up in
the display code and rounding down when converting the engine event.

It probably doesn't matter what exactly we are doing here, but for
writing display snapshot tests it does. Let's split the difference here,
and always round up, so we never give the impression that the operation
took no time.

It doesn't look like Pulumi Cloud shows this at all, so it probably
doesn't matter for users, and we could just round up for everyone as
well. I don't have strong opinions about that.
2024-04-29 08:46:36 +00:00
Thomas Gummerer 8219c922a2
display: only hide replacement steps in diff ()
When displaying diff events, we currently try to hide non-logical
replacement steps unless we specifically enable showing them. However we
currently do that for all non-logical operations, regardless whether
they are replacement steps or not.

In particular a RefreshStep is non-logical, but it's also not a
replacement step. We still want to show them during the diff because
their output can be important. Especially if the user just requested a
diff it doesn't make sense to hide the diff from them at the same time.

The intention here is to only hide replacement steps, so do that.

The full diff with the display tests is here:
https://gist.github.com/tgummerer/fcd012f13669a9cdc39530cde7770260 It's
unedited, so it includes some flakyness which isn't interesting.

I looked it over, and I think it looks like what we want, but I'm
curious to hear what others think. E.g.
https://gist.github.com/tgummerer/fcd012f13669a9cdc39530cde7770260#file-testdata-diff-L558
looks more correct now, as it shows the two delete operation that
actually happened, that it didn't show before, and it still shows the
same operation (Calling this one out in particular, since it took me a
bit to understand that we still have the same operation in the diff)

Fixes https://github.com/pulumi/pulumi/issues/7665
2024-04-26 14:54:21 +00:00
Thomas Gummerer a650905c8d
disable parallelism for all snapshot tests ()
These tests can all exhibit the same issue as described in
https://github.com/pulumi/pulumi/issues/15461. Temporarily disable
parallelism for them.
2024-04-19 23:34:43 +00:00
Fraser Waters 75340dd942
Revert "Run integration tests and dev builds with race detection" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This reverts the sdk and pkg changes from commit
655b76d030.

It also disables race detection from all builds and tests.

Fixes https://github.com/pulumi/pulumi/issues/15991.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-19 16:57:16 +00:00
Mikhail Shilkov fb20b6d78b
Change the link to Stack References docs ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Replace the link that we print to point to stack references docs. The
old link is for `pulumi.io` that isn't used anywhere else,
and also I think it goes to a wrong page and non-existing anchor.

[The
page](https://github.com/pulumi/pulumi-hugo/blob/master/themes/default/content/docs/using-pulumi/stack-outputs-and-references/index.md?plain=1)
doesn't seem to have a short link defined.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-19 14:23:13 +00:00
Pat Gavlin 5d73fc5b05
[snapshot] Elide writes for RRO with no changes ()
These changes elide checkpoint writes for RegisterResourceOutputs steps
if the associated resource's outputs have not changed. This can
eliminate unnecessary snapshot writes for stacks with large numbers of
component resources whose outputs do not change.
2024-04-18 22:09:08 +00:00
Pat Gavlin 64bee7d287
[backend] Fix dependency sorting in mustWrite ()
Just use slices.Sort instead of sort.Slice.

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-04-18 08:14:51 +00:00
Komal 9a370c727a
Use new API for deployments ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Updates Remote Automation API to use the new stable URLs for the
Deployments API. Adds support for `inheritSettings` to allow inheriting
deployment settings pre-existing on the stack.

I've tested this manually by editing the automation-api-examples for
remote deployments, but not sure of a great way to add automated tests
since automation api doesn't yet have support for setting deployment
settings.

EDIT: I considered just setting up a static stack for this but abandoned
it because of concerns around parallel runs. We can add automated tests
for this once we support creating deployment settings with automation
api (coming soon).

Fixes  
Fixes 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-16 23:23:56 +00:00
Thomas Gummerer cd9d1603ba
Fix race condition when listing stacks ()
When listing stacks, we currently try to get all the stacks from the
backend, and then get the checkpoint to for each stack to show the
summary.

However there's a race between listing the stacks and getting the
checkpoints, so the checkpoint may be missing by the time we get to that
part of the code.

Unfortunately the errors we're getting form getCheckpoint are not
structured, so we can't tell if we got a 404 from the cloud here, so the
next best option we have is re-checking whether the stack still exists.

This is another attempt at fixing
https://github.com/pulumi/pulumi/issues/15658.
2024-04-15 11:25:20 +00:00
Fraser Waters 8588aa365c
Lift context parameter to SerializeDeployment/Resource/Operations/Properties ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

SerializePropertyValue needed a `context.Context` object to pass to the
`config.Encrypter`. It was using `context.TODO()`, this change instead
accepts a context on the parameters and lifts that up to
SerializeProperties, SerializeResource, SerializeOperation, and
SerializeDeployment.

There were a few call sites for those methods that already had a context
on hand, and they now pass that context. The other calls sites now use
`context.TODO()`, we should continue to iterate in this area to ensure
everywhere that needs a context has one passed in.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-15 07:45:46 +00:00
Fraser Waters b610557f5a
Fix warning about Snapshot write failed for previews ()
This stops the CLI warning about `error: Snapshot write failed: failed
to save snapshot: [403] Updating the checkpoint is not permitted for
preview operations.` on every preview operation.
2024-04-12 18:40:55 +00:00
Fraser Waters d28fa18eec
Test SnapshotManager and Journal in engine tests ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This tests the production snapshot manager against the journalling
snapshot system in engine tests. This should ensure that both systems
produce similar (we ignore exact resource ordering, as long as both are
valid) results.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-11 22:54:08 +00:00
Fraser Waters 1b65e6c885
Add some missing checks for DisableIntegrityChecking ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This would have helped with
https://github.com/pulumi/pulumi/issues/15831 as the `destroy` would
have been able to run instead of err'ing on the first save because of
the already broken snapshot.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-11 16:38:19 +00:00
Fraser Waters 959fc6c8b4
Run integration tests with race detection ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Combining  and  and including fixes to allow tests to pass.

Enable race detection in the binary we're using for integration tests.
This will allow us to catch more data races before they get into any
release. This does mean the binary we're using for integration tests is
slightly different from the binary we're releasing, however that's
already the case as we're running a binary with coverage enabled for
them. Later we rebuild the binary we're actually releasing.

This requires us to fix the race between snapshot code, display code,
and the step executor. I've done that by adding a lock to the State
struct. This does not feel great, but it's a quick way to fix this and
get race detection running (and unblocks  which was also hitting
the race detector because it started pulling snapshot code into unit
tests as well). There's probably a more principled overhaul that doesn't
require locking at this level.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-11 15:58:42 +00:00
hanghuge 6b91857018
chore: fix function name in comment ()
fix function name in comment

Signed-off-by: hanghuge <cmoman@outlook.com>
2024-04-07 12:19:17 +00:00
VenelinMartinov c81c236354
Fix nil pointer dereference in pulumi new --ai ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Found by @Frassle

We were dereferencing the response pointer before checking the returned
error, resulting in a nil pointer dereference.

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15860

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-04 15:26:09 +00:00
Will Jones eadf1ec185
Support always qualifying stack names () ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This commit adds the `--fully-qualify-stack-names` (or `-Q` for short)
global command-line argument, which when supplied will always print
stack names in the fully-qualified form of `organization/project/stack`.
Fixes .

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [X] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-04 10:11:46 +00:00
Fraser Waters c27a2837ba
Always use the snapshot secret manager ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

There were a number of places where we passed a `Snapshot` and a
`secret.Manager` as arguments to a method, where if the `Manger` was nil
we'd fall back to the `Snapshot.SecretManager` (which could also be
nil).

Turns out in all but one place this was always passed as nil or just as
directly the snapshot's `SecretManager` field.
The one place it differed was in
`pkg/cmd/pulumi/stack_change_secrets_provider.go` where we're changing
the secret manager, but it's fine to just set the snapshot's
`SecretManager` field to the new manager.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-25 10:30:14 +00:00
Thomas Gummerer 8099d314a7
destroy: implement --continue-on-error ()
Implement a `--continue-on-error` flag for `pulumi destroy`. This makes
sure we continue processing destroys even if errors occur. We will only
continue for resources which do not depend on the failed resource, to
make sure we always have a valid snapshot available, and to not leave
any orphaned resources behind.

Resources that fail to destroy will still continue to be managed by
pulumi, and the process ends in an error to indicate to the user that
there were problems and the destroy didn't succeed cleanly.

The output will continue to be the same as for failed destroys, except
we now may succeed in destroying more resources than before.

/cc https://github.com/pulumi/pulumi/issues/3304
2024-03-22 09:22:40 +00:00
Fraser Waters 14371fa36c
Asset and Archive can have missing contents ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15729.

This fixes Assets and Archives to allow four states, as opposed to the
three that https://github.com/pulumi/pulumi/pull/14007 had enforced.

An asset can either be Text, Path, Uri, or none. That is `IsText`,
`IsPath`, and `IsURI` can all return false. Similarly for archives
except with Assets instead of Text.
This happens when a provider returns an Assert (or Archive) with a hash
value set, but no other contents.

The Asset and Archive objects have been updated to handle this case, and
a number of places in the CLI that asserted that one of
IsText/IsPath/IsURI were true have been fixed up to handle the case of
all three being false.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-21 12:32:26 +00:00
Paul C. Roberts d864cce061
Decouple persist and display events ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Retry  with fix for the issue that required the revert in  

This removes a scenario where events could not be persisted to the cloud
because they were waiting on the same event being displayed

Instead of rendering the tree every time a row is updated, instead, this
renders when the display actually happens in the the `frame` call. The
renderer instead simply marks itself as dirty in the `rowUpdated`,
`tick`, `systemMessage` and `done` methods and relies on the frame being
redrawn on a 60Hz timer (the `done` method calls `frame` explicitly).
This makes the rowUpdated call exceedingly cheap (it simply marks the
treeRenderer as dirty) which allows the ProgressDisplay instance to
service the display events faster, which prevents it from blocking the
persist events.

This requires a minor refactor to ensure that the display object is
available in the frame method

Because the treeRenderer is calling back into the ProgressDisplay object
in a goroutine, the ProgressDisplay object needs to be thread safe, so a
read-write mutex is added to protect the `eventUrnToResourceRow` map.
The unused `urnToID` map was removed in passing.

## Impact

There are scenarios where the total time taken for an operation was
dominated by servicing the events.

This reduces the time for a complex (~2000 resources) `pulumi preview`
from 1m45s to 45s

For a `pulumi up` with `-v=11` on a the same stack, where all the
register resource spans were completing in 1h6m and the
postEngineEventBatch events were taking 3h45m, this PR removes the time
impact of reporting the events (greatly inflated by the high verbosity
setting) and the operation takes the anticipated 1h6m


<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes  

This was happening because the renderer was being marked dirty once per
second in a tick event, which caused frame to redraw. There is a check
in the render method that `display.headerRow` is not nil that was
previously used to prevent rendering when no events had been added. This
check is now part of the `markDirty` logic

Some of the tests needed to be updated to make this work and have also
been refactored

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Paul Roberts <proberts@pulumi.com>
2024-03-18 16:53:13 +00:00
Fraser Waters c465c02f29
Revert "Decouple persist and display events ()" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This reverts commit ff7a7b4975.

Fixes https://github.com/pulumi/pulumi/issues/15668.
2024-03-15 20:37:31 +00:00
yeahyear cd1008562d
chore: remove repetitive words ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

Signed-off-by: yetyear <flite@outlook.com>
2024-03-13 05:14:04 +00:00
Thomas Gummerer f3a0917cfc
mark snapshot tests as non-parallel ()
These tests are currently flaky when run in parallel. Since the proper
fix is a bit more involved, and we really want to turn of rerunning
tests on failures, make them not run in parallel for the time being.
We'll keep https://github.com/pulumi/pulumi/issues/15461 open to remind
us to fix this properly and get them running in parallel again.
2024-03-12 15:37:47 +00:00
Thomas Gummerer 0f8e922c71
allow retries for encrypt/decrypt API calls ()
By default our HTTP client retries all GET requests. However these post
requests are also idempotent, as we just expect to encrypt/decrypt a
single value, without changing anything on the server side. Retry them,
so network errors won't abort the pulumi program.

There's also some log decryption events that might be retryable, but I'm
not sure about them, so I left them alone for now.

Fixes https://github.com/pulumi/pulumi/issues/15236
2024-03-06 08:35:36 +00:00
Thomas Gummerer 4083bbc238
display/progress: sort the correct array ()
We're doing a sort here, so the output is consistent. However we sort
the policyKeys array, which is no longer used at this point, instead of
the policyNames array, which is being displayed just after this point.
Fix this typo.

I noticed this while reading the code for
https://github.com/pulumi/pulumi/pull/15582.
2024-03-05 07:47:46 +00:00
Paul C. Roberts ff7a7b4975
Decouple persist and display events ()
# Description

This removes a scenario where events could not be persisted to the cloud
because they were waiting on the same event being displayed

~This uses the same buffer size for both the display and persist
channels~ [Removed to make PR a single change]

The primary change, however, is to stop rendering the tree every time a
row is updated, instead, theis renders when the display actually happens
in the the `frame` call. The renderer instead simply marks itself as
dirty in the `rowUpdated`, `tick`, `systemMessage` and `done` methods
and relies on the frame being redrawn on a 60Hz timer (the `done` method
calls `frame` explicitly). This makes the rowUpdated call exceedingly
cheap (it simply marks the treeRenderer as dirty) which allows the
ProgressDisplay instance to service the display events faster, which
prevents it from blocking the persist events.

This requires a minor refactor to ensure that the display object is
available in the frame method

Because the treeRenderer is calling back into the ProgressDisplay object
in a goroutine, the ProgressDisplay object needs to be thread safe, so a
read-write mutex is added to protect the `eventUrnToResourceRow` map.
The unused `urnToID` map was removed in passing.

## Impact

There are scenarios where the total time taken for an operation was
dominated by servicing the events.

This reduces the time for a complex (~2000 resources) `pulumi preview`
from 1m45s to 45s

For a `pulumi up` with `-v=11` on a the same stack, where all the
register resource spans were completing in 1h6m and the
postEngineEventBatch events were taking 3h45m, this PR removes the time
impact of reporting the events (greatly inflated by the high verbosity
setting) and the operation takes the anticipated 1h6m

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Paul Roberts <proberts@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-03-04 20:22:26 +00:00
Thomas Gummerer 99dcdced9d
show all paths when we have multipe local policy packs with the same name ()
It is possible for multiple local policy packs to have the same name.
When there are policy violation, we currently only show one of the paths
for those policy packs (as we consider them the same policy pack because
it has the same name).

Display both of the paths in this case to avoid any confusion. Also sort
the names, so the test we have for this is deterministic.

I think in ideal world it would be better to just error out if there are
multiple policy packs with the same name, but given this behaviour
worked for a long time I wouldn't be surprised if that ends up breaking
some use case, so I'd be a bit worried about doing that now.

A possible alternative would be to not consider policy packs with the
same name as a single policy pack, but if we name it differently I could
imagine people getting confused with that. Happy to hear alternative
opinions on what the right behaviour here is since I'm not quite sure
myself.

Fixes 
2024-03-04 14:02:25 +00:00
Anton Tayanovskyy 9544fb00e3
Label the batchDecrypt endpoint ()
Before this change looking at traces per instructions below had a span
marked "unknown". The span actually belongs to the batchDecrypt
endpoint.

https://www.pulumi.com/docs/support/troubleshooting/#performance

With the changes the span is labelled as you would expect.

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-01 21:21:43 +00:00
Julien P 5ff35273d6
Fix merge failures ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15528

See https://github.com/pulumi/pulumi/pull/15540 &
https://github.com/pulumi/pulumi/pull/15531

Re-creating this as a PR with `ci/test` label so we can get it merged.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-02-29 21:06:24 +00:00
Paul C. Roberts fdd2d8eb39
Add vim shortcuts and Home and End Keys to terminal ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Updates the terminal support for the tree view in Pulumi CLI

This adds the vim shortcuts :
`j` &ndash; down
`k` &ndash; up
`Ctrl+F` &ndash; page down 
`Ctrl+B` &ndash; page up
`g` &ndash; home
`G` &ndash; end

Also adds Home and End key support

Fixes # (issue)

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

Co-authored-by: Paul Roberts <proberts@pulumi.com>
2024-02-24 08:31:36 +00:00
Fraser Waters 485debf7da
Revert "Warn about non-project mode ()" ()
This reverts commit 038ea95d40.

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2024-02-23 07:52:44 +00:00