There are a number of use cases for `Delete` steps in a Pulumi
operation. Aside from direct deletions, where a resource has been
removed from a program, they are also a key component of _replacements_,
whereby a resource has changed in a manner where it cannot simply be
updated in the provider but instead must be completely removed and
reconstructed. In such cases, Pulumi offers two options:
* "Delete after replace", in which a new resource is first created
before the old one is deleted.
* "Delete before replace", in which the old resource is first deleted
before a replacement is created.
Delete-after-replace is the default, since where possible it is
typically the preferred option by users, enabling e.g. zero-downtime
deployments. However, there are cases where delete-after-replace is not
possible (e.g. because the new and old resources would clash in some
manner), and so delete-before-replace is an option that can be opted
into.
In cases where the deletion must happen first, we must be careful how we
handle the Pulumi state. Either or both of the delete and create calls
could fail, and we always want a state file that tells us how to resume
a failed call to yield the desired outcome. In the case of
delete-before-replace operations, a key component of resumable state is
the `PendingReplacement` field on a resource. `PendingReplacement`
indicates that a resource has been deleted in the provider, but that
this deletion is part of a replacement (and thus that a create call will
subsequently occur). In this way, the deleted resource can remain in the
state file throughout the operation, meaning that e.g. resources that
depend on the deleted resource won't have their dependencies violated
(causing a snapshot integrity error).
Alas, until this point, `PendingReplacement` was set unconditionally on
the creation of a delete-before-replace step, meaning that if the
provider delete failed, we'd elide the delete on a retry and end up with
a bad state failing snapshot integrity checks. This commit fixes this by
moving the setting of `PendingReplacement` inside `DeleteStep.Apply`, so
that it occurs only if the provider delete call succeeds. It also adds a
lifecycle test to test this case and hopefully guard against
regressions.
Fixes#16597
Part of #16667
As well as indicating that a resource's state has changes, a diff can
also indicate that those changes require the _replacement_ of the
resource, meaning that it must be recreated and not just updated. In
this scenario, there are two possible ways to replace the resource -- by
first creating another new resource before deleting the old one
("create-before-replace"), or by first deleting the old resource before
creating its replacement ("delete-before-replace").
Create-before-replace is the default since generally, if possible to
implement, it should result in fewer instances of "downtime", where a
desired resource does not exist in the system.
Should delete-before-replace be chosen, Pulumi implements this under the
hood as three steps: delete for replacement, replace, and create
replacement. To track things consistently, as well as enable resumption
of an interrupted operation, Pulumi writes a flag, `PendingReplacement`
to the state of a deleted resource that will later be cleaned up by a
completed replacement.
Should an interrupted operation be resumed, Pulumi does not currently
take `PendingReplacement` into account, and always enqueues a(nother)
delete operation. This is typically fine (albeit wasteful) since deletes
are (should) be idempotent, but unnecessary. This commit adds
@jesse-triplewhale's fix for this behaviour whereby the
`PendingReplacement` flag is simply removed before the remainder of the
required steps (replace, create replacement) are actioned as normal. It
also extends this work with some lifecycle tests for this scenario and a
few others that may arise as a result of an interrupted replacement.
Fixes#16288Closes#16303
Co-authored-by: Jesse Grodman <jesse@triplewhale.com>
When displaying the progress of a Pulumi operation to the user, we want
the operation being displayed to reflect what is actually happening at
that moment in time. Most of the time, this means "just display the
operation in question" -- if a `create` is being executed, show
"creating", if a `delete` just completed, show "deleted", and so on.
However, there are cases where we can do better than just displaying the
"raw" operation. Specifically, our "replacement-like" operations
comprise a _series_ of steps that must execute for the operation as a
whole to make sense. For create-before-replace, we have:
* `create replacement` resource
* `replace` the old resource
* `delete original` resource
Other sequences, such as delete-before-replace, are similar (in the case
of delete-before-replace, the `delete original` step comes first).
While it might make sense to display the underlying steps as the
operation progresses, when the series of steps has _completed_, it's
(arguably) much clearer to simply render the string `replaced` so that
the user knows what has gone on. Similarly, during a preview, it (again
arguably) makes more sense for us to state that the intention is to
`replace`, rather than any one of `create replacement`/`replace`/`delete
original` and so on.
Alas, there is a case where this is potentially misleading and thus
undesirable behaviour. If an _error_ occurs during execution, the
operation will terminate at the next opportunity. In doing so, it will
enter a "done" state. At this point, we _do not_ want to rewrite the
step that was actually happening before the error interrupted it (e.g.
`create replacement`) with the "end" state (e.g. `replaced`), since the
error may mean we never reached that desired state. We want the display
to be as true to the raw series of steps as possible. This PR implements
this change, so that programs which terminate due to errors do not
rewrite their steps.
This PR addresses some of the confusion in #16270, in which we
incorrectly reported that a delete-before-replace resource had been
`replaced` when in fact we had only completed the deletion before being
interrupted by an error elsewhere.
We want to add more test coverage to the display code. The best way to
do that is to add it to the engine tests, that already cover most of the
pulumi functionality.
It's probably not really possible to review all of the output, but at
least it gives us a baseline, which we can work with.
There's a couple of tests that are flaky for reasons I don't quite
understand yet. I marked them as to skip and we can look at them later.
I'd rather get in the baseline tests sooner, rather than spending a
bunch of time looking at that. The output differences also seem very
minor, so not super concerning.
The biggest remaining issue is that this doesn't interact well with the
Chdir we're doing in the engine. We could either pass the CWD through,
or just try to get rid of that Chdir. So this should only be merged
after https://github.com/pulumi/pulumi/pull/15607.
I've tried to split this into a few commits, separating out adding the
testdata, so it's hopefully a little easier to review, even though the
PR is still quite large.
One other thing to note is that we're comparing that the output has all
the same lines, and not that it is exactly the same. Because of how the
engine is implemented, there's a bunch of race conditions otherwise,
that would make us have to skip a bunch of tests, just because e.g.
resource A is sometimes deleted before resource B and sometimes it's the
other way around.
The biggest downside of that is that running with `PULUMI_ACCEPT` will
produce a diff even when there are no changes. Hopefully we won't have
to run that way too often though, so it might not be a huge issue?
---------
Co-authored-by: Fraser Waters <fraser@pulumi.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a pseudo op `OpOutputChange` to the set of changes that are
recorded in `display.ResourceChanges` to count the number of output
changes, this is then included in the check used to evaluate the
`--expect-no-changes` flag
When resource outputs are registered, they are checked against their
previous value using existing functionality, the total count of changes
is then added
The internal capability is validated with an engine test, the cli is
validated using an integration test
This will break user workflows that depend on the previous behavior
Fixes#15564
## Checklist
- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Paul Roberts <proberts@pulumi.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
Retry #15529 with fix for the issue that required the revert in #15705
This removes a scenario where events could not be persisted to the cloud
because they were waiting on the same event being displayed
Instead of rendering the tree every time a row is updated, instead, this
renders when the display actually happens in the the `frame` call. The
renderer instead simply marks itself as dirty in the `rowUpdated`,
`tick`, `systemMessage` and `done` methods and relies on the frame being
redrawn on a 60Hz timer (the `done` method calls `frame` explicitly).
This makes the rowUpdated call exceedingly cheap (it simply marks the
treeRenderer as dirty) which allows the ProgressDisplay instance to
service the display events faster, which prevents it from blocking the
persist events.
This requires a minor refactor to ensure that the display object is
available in the frame method
Because the treeRenderer is calling back into the ProgressDisplay object
in a goroutine, the ProgressDisplay object needs to be thread safe, so a
read-write mutex is added to protect the `eventUrnToResourceRow` map.
The unused `urnToID` map was removed in passing.
## Impact
There are scenarios where the total time taken for an operation was
dominated by servicing the events.
This reduces the time for a complex (~2000 resources) `pulumi preview`
from 1m45s to 45s
For a `pulumi up` with `-v=11` on a the same stack, where all the
register resource spans were completing in 1h6m and the
postEngineEventBatch events were taking 3h45m, this PR removes the time
impact of reporting the events (greatly inflated by the high verbosity
setting) and the operation takes the anticipated 1h6m
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#15668
This was happening because the renderer was being marked dirty once per
second in a tick event, which caused frame to redraw. There is a check
in the render method that `display.headerRow` is not nil that was
previously used to prevent rendering when no events had been added. This
check is now part of the `markDirty` logic
Some of the tests needed to be updated to make this work and have also
been refactored
## Checklist
- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Paul Roberts <proberts@pulumi.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Signed-off-by: yetyear <flite@outlook.com>
We're doing a sort here, so the output is consistent. However we sort
the policyKeys array, which is no longer used at this point, instead of
the policyNames array, which is being displayed just after this point.
Fix this typo.
I noticed this while reading the code for
https://github.com/pulumi/pulumi/pull/15582.
# Description
This removes a scenario where events could not be persisted to the cloud
because they were waiting on the same event being displayed
~This uses the same buffer size for both the display and persist
channels~ [Removed to make PR a single change]
The primary change, however, is to stop rendering the tree every time a
row is updated, instead, theis renders when the display actually happens
in the the `frame` call. The renderer instead simply marks itself as
dirty in the `rowUpdated`, `tick`, `systemMessage` and `done` methods
and relies on the frame being redrawn on a 60Hz timer (the `done` method
calls `frame` explicitly). This makes the rowUpdated call exceedingly
cheap (it simply marks the treeRenderer as dirty) which allows the
ProgressDisplay instance to service the display events faster, which
prevents it from blocking the persist events.
This requires a minor refactor to ensure that the display object is
available in the frame method
Because the treeRenderer is calling back into the ProgressDisplay object
in a goroutine, the ProgressDisplay object needs to be thread safe, so a
read-write mutex is added to protect the `eventUrnToResourceRow` map.
The unused `urnToID` map was removed in passing.
## Impact
There are scenarios where the total time taken for an operation was
dominated by servicing the events.
This reduces the time for a complex (~2000 resources) `pulumi preview`
from 1m45s to 45s
For a `pulumi up` with `-v=11` on a the same stack, where all the
register resource spans were completing in 1h6m and the
postEngineEventBatch events were taking 3h45m, this PR removes the time
impact of reporting the events (greatly inflated by the high verbosity
setting) and the operation takes the anticipated 1h6m
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Paul Roberts <proberts@pulumi.com>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
It is possible for multiple local policy packs to have the same name.
When there are policy violation, we currently only show one of the paths
for those policy packs (as we consider them the same policy pack because
it has the same name).
Display both of the paths in this case to avoid any confusion. Also sort
the names, so the test we have for this is deterministic.
I think in ideal world it would be better to just error out if there are
multiple policy packs with the same name, but given this behaviour
worked for a long time I wouldn't be surprised if that ends up breaking
some use case, so I'd be a bit worried about doing that now.
A possible alternative would be to not consider policy packs with the
same name as a single policy pack, but if we name it differently I could
imagine people getting confused with that. Happy to hear alternative
opinions on what the right behaviour here is since I'm not quite sure
myself.
Fixes#15566
Turn on the golangci-lint exhaustive linter. This is the first step
towards catching more missing cases during development rather than
in tests, or in production.
This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.
I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
Fixes#14601
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945
This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.
Most of these edits were made by running `perfsprint --fix`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Small refactor I noticed while writing a test with engine events. We
always had to call `NewEvent` with the tag and payload value for an
event and these _had_ to match up else the engine panics. But we can
just pass the payload and type switch to work out the tag. Means one
less parameter to pass to `NewEvent` and pretty much no chance of it
going wrong. To ensure there's really no chance I've added a generic
union type so you can only pass payload types to this method now.
Cancel had to be handled separately because it doesn't have a payload
type, it's just nil.
# Description
🚧
This is a work in progress attempting to add some output for when policy
packs are being loaded/compiled (which I realized was the actual reason
for the slowdown, not that they were being run already). There's still
some pretty obvious code quality issues as I've been throwing this
together, and the URN's are not constructed properly yet either, so some
of the output is less helpful than it could be. I'm mainly opening this
to get some early feedback on whether this is the appropriate place to
add the output, or if there are better alternatives.
Also including a gif here of what this looks like. In particular the
Name here needs to be improved, and the "Plan" might want to say
"loading/compiling" or something similar instead?
![policy-pack](https://github.com/pulumi/pulumi/assets/191004/85cca146-d740-4e4c-96dc-cc26dca6bb90)
Fixes#14453
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.
This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.
There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.
Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
Similar to how https://github.com/pulumi/pulumi/pull/13953 moves some
code from sdk/go/common to /pkg. This display code is only used in /pkg,
another simple reduction of what's in sdk/go/common.
Pulumi previews now print the summary event if there are diagnostic
messages. The display will not print the summary if there are error
diagnostics.
Fixes#10880
This change causes only mandatory policy violations to skip the summary
display causing the resources changed and duration details to not be
printed.
Policy Violations are categorized as `mandatory`, `disabled`, and
`advisory`. We were treating all 3 as policy violations to skip
displaying the update summary. Now we only use `mandatory`.
simplifyTypeName now type.DisplayName() is being used in many areas. It
is a pure function, and has a single argument token.Type. This change
improves its discoverability as it needs to be made visible between the
backend and display modules.
Retain on delete currently applies to replaces and deletes. It is
unclear when a resource is retained on delete. To clarify this, this
commit:
- Improves update status messages with "[retain]"
to clarify when a resource is retained on delete and dropped.
- Adds info warning on preview as to which resources will be dropped in the cloud environment.
Switch this to "View in Browser" per discussion
which is closer to the old "View Live" message.
sdk/go/auto doesn't need an update because the regex there
is still using "View in Browser".
12380: [cli] Add an "open in browser" shortcut r=pgavlin a=pgavlin
Add support for using ctrl+o to open the current update in the browser
for backends that support permalinks.
Co-authored-by: Pat Gavlin <pat@pulumi.com>
Add support for using Ctrl+O to open the current update in the browser
for backends that support permalinks.
The keybinding is advertised in the interactive display as part of the
message that displays the permalink:
```
Previewing update (dev)
View in Browser (Ctrl+O): https://<some-url>
Type Name Plan
+ pulumi:pulumi:Stack vpc-dev create
+ ├─ aws:ec2:Vpc vpc create
+ ├─ aws:ec2:SecurityGroup secgroup create
+ ├─ aws:ec2:SecurityGroupRule rule-2 create
+ ├─ aws:ec2:SecurityGroupRule rule-0 create
+ ├─ aws:ec2:SecurityGroupRule rule-1 create
+ └─ aws:ec2:SecurityGroupRule rule-3 create
```
In order to maintain backwards compatibility with older versions of the
Automation API, the message is not changed for non-interactive
scenarios.
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
Go treats comments that match the following regex as directives.
//[a-z0-9]+:[a-z0-9]
Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)
Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.
// nolint:foo
//nolint: foo
This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.
Verification:
Here's the output of `go doc` on some entities
before and after this change.
Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
nolint: lll, interfacer
nolint: lll, interfacer
const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```
After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```
Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
func URN_(o string) ResourceOption
URN_ is an optional URN of a previously-registered resource of this type to
read from the engine. nolint: revive
```
After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
func URN_(o string) ResourceOption
URN_ is an optional URN of a previously-registered resource of this type to
read from the engine.
```
Note that golangci-lint offers a 'nolintlint' linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.
Resolves#11785
Related: https://github.com/golangci/golangci-lint/issues/892
[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
tee "$FILES" |
xargs perl -p -i -e '
s|// nolint|//nolint|g;
s|//nolint: |//nolint:|g;
'
rg '.go$' < "$FILES" | xargs gofmt -w -s
The number of Unicode code points in a string is not the same as the
number of user-visible characters (graphemes). When measuring colorized
strings, we want the latter rather than the former. Notably, these
changes fix some issues where the interactive display cut off before the
right edge of the terminal.
Replace direct interaction with the terminal with an abstraction. This
abstraction is tightly constrained to the capabilities needed for the
CLI's display. Using this abstraction allows for straightforward testing
of the interactive renderers.
The display pipleline looks like this:
╭──────╮
│Engine│
╰──────╯
⬇ engine events
╭────────────────╮
│Progress Display│
╰────────────────╯
⬇ display events: ticks, resource updates, system messages
╭─────────────────╮
│Progress Renderer│
╰─────────────────╯
⬇ text
╭────────╮
│Terminal│
╰────────╯
The existing implementation of the interactive Progress Renderer is broken
into two parts, the display renderer and the message renderer. The display
renderer converts display events into progress messages, each of which
generally represents a single line of text at a particular position in
the output. The message renderer converts progress messages into screen
updates by identifying whether or not the contents of a particular
message have changed and if so, re-rendering its output line. In
somewhat greater detail:
╭────────────────╮
│Display Renderer│
╰────────────────╯
⬇ convert resource rows into a tree table
⬇ convert the tree table and system messages into lines
⬇ convert each line into a progress message with an index
╭────────────────╮
│Message Renderer│
╰────────────────╯
⬇ if the line identified in a progress message has changed,
⬇ go to that line on the terminal, clear it, and update it
╭────────╮
│Terminal│
╰────────╯
This separation of concerns is unnecessary and makes it difficult to
understand where and when the terminal is updated. This approach also
makes it somewhat challenging to change the way in which the display
interacts with the terminal, as both the display renderer and the
message renderer need to e.g. understand terminal dimensions, movement,
etc.
These changes reimplement the interactive Progress Renderer using a
frame-oriented approach. The display is updated at 60 frame per second.
If nothing has happened to invalidate the display's contents (i.e. no
changes to the terminal geometry or the displayable contents have occurred),
then the frame is not redrawn. Otherwise, the contents of the display
are re-rendered and redrawn.
An advantage of this approach is that it made it relatively simple to
fix a long-standing issue with the interactive display: when the number
of rows in the output exceed the height of the terminal, the new
renderer clamps the output and allows the user to scroll the tree table
using the up and down arrow keys.
The progress display is logically composed of two pieces:
- An event listener that transforms raw engine events into renderable
data
- A renderer that renders that data on to the screen
The interface for the latter is relatively simple. It only really needs
to know the following:
- When an idle interval has elapsed
- When a row has changed
- When a system message (e.g. a line printed to stdout) has arrived
- When the update is done
- When the display is closed
- When to print a raw line
This interface is general enough to accommodate multiple renderers.
* Moving previewDigest and exporting it, closes#9851
* Moving previewDigest and exporting it, closes#9851
* Updating changelog-pending
* Go Mod Tidy
* replacing to local
* more go.mod changes
* reseting go mod
* full move
* Fixing golint
* No go.mod changes needed
Reuse the support for decoding JSON/YAML strings into objects to display
such values as objects during create, same, and delete operations (we
already do this for updates).
These changes also unexport a number of methods that should be internal
to the display package.
- Move the code that deals in translating detailed diffs into object
diffs out of package `backend/display` and into package `engine`
- Move the code that deals in displaying diffs out of package `engine`
and into package `backend/display`
This command renders a recorded stream of engine events using either the
progress or diff renderer. This is useful for debugging problems with
these renderers.
* Readd "Make StackReference.Name a tokens.Name (#9088)"
This reverts commit f0aa4df149.
This also removes the AsName asserting casts for stack names. We do want
to add them in at some point to be sure that bad names don't slip in
somehow but they don't need adding with this.
* Update sdk/go/common/util/fsutil/qname.go
Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Ian Wahbe <ian@wahbe.com>