Commit Graph

44 Commits

Author SHA1 Message Date
Will Jones b4ee9680a0
Display `[retain]` in all cases of delete retention ()
When the `RetainOnDelete` resource option is set, the console should
display `[retain]` next to resources which are being deleted from the
Pulumi program to indicate that, while the Pulumi state for those
resources will be deleted, the provider will not be instructed to
actually `Delete` the resource (effectively leaving the resource
unmanaged).

Currently, Pulumi decides whether or not to add the `[retain]` text by
inspecting resource state attached to a display step. However, there are
two places in which a display step is constructed:

* As part of normal/"in-memory" execution, whereby resource state is
inherently available and attached to the display step.
* As part of deserializing an existing JSON event stream (e.g. from a
Pulumi REST API call), whereby resource state is _not_ available and so
absent (`nil`) from the display step. This is also the codepath taken by
display tests.

The absence of the state in the latter codepath means that `[retain]` is
not displayed. This needn't be the case however. The `RetainOnDelete`
option is explicitly persisted in our JSON events and copied on to the
display step, so we can just use that instead. This commit makes this
change. Additionally, the single test we have for `RetainOnDelete` is
pulled out into its own file and display testing is now enabled for that
test.
2024-06-28 23:19:26 +00:00
Luke Hoban f1e4b4ff94
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes ()
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:

* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.

In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.

The implementation covers the following:

* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.

Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.

For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.

### Notes

- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.

### Fixes

* Fixes 
* Fixes 
* Fixes  
* Not quite , but likely replaces the need for that

Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
Fraser Waters dc12644fe4
Never show property diffs for OpSame ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15944.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-05-02 12:28:43 +00:00
hanghuge 6b91857018
chore: fix function name in comment ()
fix function name in comment

Signed-off-by: hanghuge <cmoman@outlook.com>
2024-04-07 12:19:17 +00:00
Fraser Waters 516979770f
Allow anything in resource names ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/13968.
Fixes https://github.com/pulumi/pulumi/issues/8949.

This requires changing the parsing of URN's slightly, it is _very_
likely that
providers will need to update to handle URNs like this correctly.

This changes resource names to be `string` not `QName`. We never
validated this before and it turns out that users have put all manner of
text for resource names so we just updating the system to correctly
reflect that.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-20 08:59:00 +00:00
Joe Duffy 96a9a77167
Policy remediations feature ()
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
2023-10-09 18:31:17 +00:00
Fraser Waters 4ebf61f896
Move sdk/go/common/display to /pkg/display ()
Similar to how https://github.com/pulumi/pulumi/pull/13953 moves some
code from sdk/go/common to /pkg. This display code is only used in /pkg,
another simple reduction of what's in sdk/go/common.
2023-09-18 11:01:28 +00:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
Kyle Dixler d747baef3a
Addressed feedback from Code Review 2023-04-13 15:04:53 -07:00
Kyle Dixler 1507872dbb
Move simplifyTypeName to DisplayName() method on token.Type
simplifyTypeName now type.DisplayName() is being used in many areas. It
is a pure function, and has a single argument token.Type. This change
improves its discoverability as it needs to be made visible between the
backend and display modules.
2023-04-13 14:44:07 -07:00
Kyle Dixler d5f42a56e0
Add more detail to update display to indicate `retainOnDelete` usage.
Retain on delete currently applies to replaces and deletes. It is
unclear when a resource is retained on delete. To clarify this, this
commit:
- Improves update status messages with "[retain]"
to clarify when a resource is retained on delete and dropped.
- Adds info warning on preview as to which resources will be dropped in the cloud environment.
2023-04-05 17:25:23 -07:00
Abhinav Gupta c97d9401f2
getDiffInfo: Fix slice length check
Checking if the slice is non-nil instead of checking length
causes breakage when we pre-allocate diffs based on `len(md.Diffs)`.
2023-01-11 21:53:05 -08:00
Richard Shade a5687d16a9
Moving previewDigest to sdk/go/common/display, and exporting it. ()
* Moving previewDigest and exporting it, closes 

* Moving previewDigest and exporting it, closes 

* Updating changelog-pending

* Go Mod Tidy

* replacing to local

* more go.mod changes

* reseting go mod

* full move

* Fixing golint

* No go.mod changes needed
2022-06-27 09:08:06 -05:00
Pat Gavlin 427d8635d5
[display] Refactor diff translation and display. ()
- Move the code that deals in translating detailed diffs into object
  diffs out of package `backend/display` and into package `engine`
- Move the code that deals in displaying diffs out of package `engine`
  and into package `backend/display`
2022-04-07 12:05:33 -07:00
Fraser Waters 86f3f712aa
Readd "Make StackReference.Name a tokens.Name ()" ()
* Readd "Make StackReference.Name a tokens.Name ()"

This reverts commit f0aa4df149.

This also removes the AsName asserting casts for stack names. We do want
to add them in at some point to be sure that bad names don't slip in
somehow but they don't need adding with this.

* Update sdk/go/common/util/fsutil/qname.go

Co-authored-by: Ian Wahbe <ian@wahbe.com>

Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-03-17 21:37:11 +00:00
Ian Wahbe f0aa4df149
Revert "Make StackReference.Name a tokens.Name ()" ()
This reverts commit a72df9aefa.
2022-03-09 13:48:52 -08:00
Fraser Waters a72df9aefa
Make StackReference.Name a tokens.Name ()
* Make StackReference.Name a tokens.Name

* Don't panic in ParseStackReference
2022-03-07 20:59:44 +00:00
Ian Wahbe 9df8d3a028
Bold in-progress diffs diffrently ()
* Initial take on coloring in-progress diffs

* Update CHANGELOG_PENDING.md

* Add high and low color levels

* Don't print functions

* Change colorprogress to bold

* Fix @komalali's nits.

* Restore background colors comment
2021-09-28 15:16:09 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Luke Hoban b82b850af0
Fix `pulumi refresh` diffs () 2021-03-19 13:11:59 +11:00
Pat Gavlin 418e2291a2
Protect against engine event mutation. ()
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
2020-07-16 23:52:31 -07:00
Paul Stack 0824fc9a8b
Revert "Narrow a few interfaces." () 2020-07-09 15:19:12 +01:00
Pat Gavlin 45d2fa95d6
Narrow a few interfaces. ()
- Remove `Info` from `Source`. This method was not used.
- Remove `Stack` from `EvalSource`. This method was not used.
- Remove `Type` and `URN` from `Step`. These values are available via
  `Res().URN.Type()` and `Res().URN`, respectively. This removes the
  possibility of inconsistencies between the type, URN, and state of the
  resource associated with a `Step`.
- Remove URN from StepEventMetadata.
2020-07-01 15:32:50 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle d3f5bbce48 go fmt 2020-03-18 17:27:02 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle fba783caf9 move pkg/resource -> sdk/go/common/resource, but leave nested resource packages 2020-03-18 13:36:19 -07:00
Jamie Kinkead abd1b98003
improve PAC error output ()
* Improve pac error output

* Print policy packs applied if no violations present

* Remove unnecessary policyInfo struct

* Adjust integration test
2020-02-13 15:16:46 -08:00
Alex Clemmer 429bde332b Print PolicyPacks run as part of update summary
Fixes .
2019-09-30 14:49:21 -07:00
CyrusNajmabadi 1387afec8f
Color 'reads' as cyan so they don't look like 'creates'. () 2019-09-18 09:49:13 -07:00
CyrusNajmabadi c30b8a509e
Always print out the last diagnostics we got. Unless we had an error and we're in the tree-view. () 2019-08-01 10:21:47 -07:00
Alex Clemmer fc80eaaa3d Implement `GetAnalyzerInfo` in analyzer plugin 2019-07-16 00:58:33 -07:00
Pat Gavlin e1a52693dc
Add support for importing existing resources. ()
A resource can be imported by setting the `import` property in the
resource options bag when instantiating a resource. In order to
successfully import a resource, its desired configuration (i.e. its
inputs) must not differ from its actual configuration (i.e. its state)
as calculated by the resource's provider.

There are a few interesting state transitions hiding here when importing
a resource:
1. No prior resource exists in the checkpoint file. In this case, the
   resource is simply imported.
2. An external resource exists in the checkpoint file. In this case, the
   resource is imported and the old external state is discarded.
3. A non-external resource exists in the checkpoint file and its ID is
   different from the ID to import. In this case, the new resource is
   imported and the old resource is deleted.
4. A non-external resource exists in the checkpoint file, but the ID is
   the same as the ID to import. In this case, the import ID is ignored
   and the resource is treated as it would be in all cases except for
   changes that would replace the resource. In that case, the step
   generator issues an error that indicates that the import ID should be
   removed: were we to move forward with the replace, the new state of
   the stack would fall under case (3), which is almost certainly not
   what the user intends.

Fixes .
2019-07-12 11:12:01 -07:00
Pat Gavlin 6e5c4a38d8
Defer all diffs to resource providers. ()
Thse changes make a subtle but critical adjustment to the process the
Pulumi engine uses to determine whether or not a difference exists
between a resource's actual and desired states, and adjusts the way this
difference is calculated and displayed accordingly.

Today, the Pulumi engine get the first chance to decide whether or not
there is a difference between a resource's actual and desired states. It
does this by comparing the current set of inputs for a resource (i.e.
the inputs from the running Pulumi program) with the last set of inputs
used to update the resource. If there is no difference between the old
and new inputs, the engine decides that no change is necessary without
consulting the resource's provider. Only if there are changes does the
engine consult the resource's provider for more information about the
difference. This can be problematic for a number of reasons:

- Not all providers do input-input comparison; some do input-state
  comparison
- Not all providers are able to update the last deployed set of inputs
  when performing a refresh
- Some providers--either intentionally or due to bugs--may see changes
  in resources whose inputs have not changed

All of these situations are confusing at the very least, and the first
is problematic with respect to correctness. Furthermore, the display
code only renders diffs it observes rather than rendering the diffs
observed by the provider, which can obscure the actual changes detected
at runtime.

These changes address both of these issues:
- Rather than comparing the current inputs against the last inputs
  before calling a resource provider's Diff function, the engine calls
  the Diff function in all cases.
- Providers may now return a list of properties that differ between the
  requested and actual state and the way in which they differ. This
  information will then be used by the CLI to render the diff
  appropriately. A provider may also indicate that a particular diff is
  between old and new inputs rather than old state and new inputs.

Fixes .
2019-07-01 12:34:19 -07:00
CyrusNajmabadi ef3cad6bf1
Reads should not cause resources to be displayed in our progress display () 2019-06-18 15:38:32 -07:00
Alex Clemmer 0fc4bc7885 Remove policy ID from policy API 2019-06-13 17:39:30 -07:00
Alex Clemmer 8b7d329c69 Use Analyzer PB in analyzer code 2019-06-13 16:04:13 -07:00
James Nugent edab10e9c8 Use Go Modules for dependency tracking
This commit switches from dep to Go 1.12 modules for tracking Pulumi
dependencies. Rather than _building_ using Go modules, we instead use the `go
mod vendor` command to populate a vendor tree in the same way as `dep ensure`
was previously doing.

In order to prevent checksum mismatches, it was necessary to also update CI to
use Go 1.12 instead of 1.11 - which also necessitated fixing some linting errors
which appeared with the upgraded golangci-lint for 1.12.
2019-04-10 08:37:51 +04:00
Pat Gavlin 4b33a45561
Filter diff keys based on provider info ()
If a provider returns information about the top-level properties that
differ, use those keys to filter the diffs that are rendered to the
user.

Fixes .
2019-03-06 16:41:19 -08:00
CyrusNajmabadi 4e2a6aee0f
Ensure we show the properties that changed when doing a multi-stage replace. () 2018-11-05 12:37:53 -08:00
joeduffy b06a02f03c Use go-humanize instead of lame hand-pluralization 2018-09-24 14:40:14 -07:00
joeduffy 3468393490 Make a smattering of CLI UX improvements
Since I was digging around over the weekend after the change to move
away from light black, and the impact it had on less important
information showing more prominently than it used to, I took a step
back and did a deeper tidying up of things. Another side goal of this
exercise was to be a little more respectful of terminal width; when
we could say things with fewer words, I did so.

* Stylize the preview/update summary differently, so that it stands
  out as a section. Also highlight the total changes with bold -- it
  turns out this has a similar effect to the bright white colorization,
  just without the negative effects on e.g. white terminals.

* Eliminate some verbosity in the phrasing of change summaries.

* Make all heading sections stylized consistently. This includes
  the color (bright magenta) and the vertical spacing (always a newline
  separating headings). We were previously inconsistent on this (e.g.,
  outputs were under "---outputs---"). Now   the headings are:
  Previewing (etc), Diagnostics, Outputs, Resources, Duration, and Permalink.

* Fix an issue where we'd parent things to "global" until the stack
  object later showed up. Now we'll simply mock up a stack resource.

* Don't show messages like "no change" or "unchanged". Prior to the
  light black removal, these faded into the background of the terminal.
  Now they just clutter up the display. Similar to the elision of "*"
  for OpSames in a prior commit, just leave these out. Now anything
  that's written is actually a meaningful status for the user to note.

* Don't show the "3 info messages," etc. summaries in the Info column
  while an update is ongoing. Instead, just show the latest line. This
  is more respectful of width -- I often find that the important
  messages scroll off the right of my screen before this change.

    For discussion:

        - I actually wonder if we should eliminate the summary
          altogether and always just show the latest line. Or even
          blank it out. The summary feels better suited for the
          Diagnostics section, and the Status concisely tells us
          how a resource's update ended up (failed, succeeded, etc).

        - Similarly, I question the idea of showing only the "worst"
          message. I'd vote for always showing the latest, and again
          leaving it to the Status column for concisely telling the
          user about the final state a resource ended up in.

* Stop prepending "info: " to every stdout/stderr message. It adds
  no value, clutters up the display, and worsens horizontal usage.

* Lessen the verbosity of update headline messages, so we now instead
  of e.g. "Previewing update of stack 'x':", we just say
  "Previewing update (x):".

* Eliminate vertical whitespace in the Diagnostics section. Every
  independent console.out previously was separated by an entire newline,
  which made the section look cluttered to my eyes. These are just
  streams of logs, there's no reason for the extra newlines.

* Colorize the resource headers in the Diagnostic section light blue.

Note that this will change various test baselines, which I will
update next. I didn't want those in the same commit.
2018-09-24 08:43:46 -07:00
Sean Gillespie d80db466a6
Display diff when Protect bit changes ()
It's useful to show a diff whenever a user toggles the protect bit on a
resource, so they can be sure that it has actually occured.
2018-09-17 15:03:06 -07:00
joeduffy bf51d7594a Refactor display logic out of pkg/backend/filestate
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.

Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
2018-09-05 07:33:18 -07:00