Commit Graph

153 Commits

Author SHA1 Message Date
Kyle Dixler 485718f533
[ci] `pkg/resource/deploy/(step(_generator|_executor)?|import).go` coverage ()
covers
- step.go
- step_executor.go
- step_generator.go
- import.go

They all depend on an added field to steps that use providers in
`Apply()`

# Includes changes to a non-test file: step.go

steps query the deployment for providers. There is not a straightforward
way of mocking a provider for a step. I've added a field called
`provider plugin.Provider` to steps that use providers to be used
instead of querying the Deployment with getProvider().

This approach aims to minimize the cognitive complexity and potential
for errors in comparison to the branching alternative due to the
behavior of `:=` and assigning the value to an existing value while also
defining a new variable err.

```diff
-               prov, err := getProvider(s, s.provider)
-               if err != nil {
-                       return resource.StatusOK, nil, err
+               prov := s.provider
+               if prov == nil {
+                       var err error
+                       prov, err = getProvider(s)
+                       if err != nil {
+                               return resource.StatusOK, nil, err
+                       }
                }
```
2023-12-22 21:14:04 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Fraser Waters bbaf871ad1
Check qualified type for root stackness ()
With the introduction of component programs there will be mulitple
"pulumi:pulumi:stack" resources in a program. We should therefore check
against the qualified type of resources to see if they are the root
stack, not just their direct type.
2023-12-04 10:36:51 +00:00
Fraser Waters 47c4d4c79b
Pass inputs to read as inputs and state ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Also adds some more checks to the import tests to give confidence that
the import machinery was behaving as expected.

I noticed this inconsistency of Read when looking at
https://github.com/pulumi/pulumi/pull/14678. Comment in the code at the
change site, but just to repeat here. I don't think we can stop sending
inputs as state because providers have probably come to depend on that,
but it's a little more consistent if we pass them as inputs as well.
Means in the stackref code we can _just_ look at the inputs set.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-02 14:39:38 +00:00
Fraser Waters 3107c6f62e
Allow `import` to create empty component resources ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14444.

This adds two new flags to the import file to signal that a resource is
a component, and also if that is a remote (MLC) component. The import
system will create empty resources for those import descriptions, but
importantly they can then be used as parents for other resources in the
import deployment.

We need to know if it's a remote component to know if a provider should
be looked up for it. We could probably get away with _just_ knowing if
it's a component at import time and not doing any validation that the
type token is well formed and the provider exists for MLCs. That makes
import a little simpler for users (no need to set if it's remote or not)
but pushes error cases downstream to when they actually try to `up`
their program. It also means we can't lookup and fill in the default
providers for these resources, resulting in a provider diff at `up`
time.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-13 17:58:35 +00:00
Fraser Waters cfe8165ded
Allow importing a parent and child resource at the same time ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Allow `pulumi import` to import one resource and then use that resource
as a parent for another imported resource.

Currently parents can only refer to resources that already exist in the
deployment meaning if you want to do this today you have to do two
imports.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-10 13:31:11 +00:00
Fraser Waters ca12edfb4e
Send old inputs to Delete ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14115.

This was missed as part of https://github.com/pulumi/pulumi/pull/13139.

Adds a new configure flag (sends_old_inputs_to_delete) which the engine
will now always set to true. If that's set providers can rely on the old
inputs being sent to delete, otherwise they'll get nil.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-13 14:12:26 +00:00
Fraser Waters 72a9aa9670
Don't edit state in completion functions ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
The completion functions run _after_ the step executor has called
OnResourceStepPost where we trigger the snapshot save.

So if the data was saved or not depended on it racing the snapshot
goroutine.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-25 18:22:23 +00:00
Fraser Waters e263e15363
Replace use of result.Result in step generator ()
More cleanup of result.Result, this time a pass through the step
generator.
2023-09-25 12:25:23 +00:00
Fraser Waters 4ebf61f896
Move sdk/go/common/display to /pkg/display ()
Similar to how https://github.com/pulumi/pulumi/pull/13953 moves some
code from sdk/go/common to /pkg. This display code is only used in /pkg,
another simple reduction of what's in sdk/go/common.
2023-09-18 11:01:28 +00:00
Pat Gavlin 948bb36e7e [engine] Add support for source positions
These changes add support for passing source position information in
gRPC metadata and recording the source position that corresponds to a
resource registration in the statefile.

Enabling source position information in the resource model can provide
substantial benefits, including but not limited to:

- Better errors from the Pulumi CLI
- Go-to-defintion for resources in state
- Editor integration for errors, etc. from `pulumi preview`

Source positions are (file, line) or (file, line, column) tuples
represented as URIs. The line and column are stored in the fragment
portion of the URI as "line(,column)?". The scheme of the URI and the
form of its path component depends on the context in which it is
generated or used:

- During an active update, the URI's scheme is `file` and paths are
  absolute filesystem paths. This allows consumers to easily access
  arbitrary files that are available on the host.
- In a statefile, the URI's scheme is `project` and paths are relative
  to the project root. This allows consumers to resolve source positions
  relative to the project file in different contexts irrespective of the
  location of the project itself (e.g. given a project-relative path and
  the URL of the project's root on GitHub, one can build a GitHub URL for
  the source position).

During an update, source position information may be attached to gRPC
calls as "source-position" metadata. This allows arbitrary calls to be
associated with source positions without changes to their protobuf
payloads. Modifying the protobuf payloads is also a viable approach, but
is somewhat more invasive than attaching metadata, and requires changes
to every call signature.

Source positions should reflect the position in user code that initiated
a resource model operation (e.g. the source position passed with
`RegisterResource` for `pet` in the example above should be the source
position in `index.ts`, _not_ the source position in the Pulumi SDK). In
general, the Pulumi SDK should be able to infer the source position of
the resource registration, as the relationship between a resource
registration and its corresponding user code should be static per SDK.

Source positions in state files will be stored as a new `registeredAt`
property on each resource. This property is optional.
2023-07-10 14:35:40 -07:00
bors[bot] db579129f4
Merge
13139: Send old inputs to diff and update r=Frassle a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/5973

Both Azure and Kubernetes set an extra field on created resource outputs (called "__inputs") that doesn't exist in the inputs to the corresponding Pulumi resources. Their Diff logic needs the subset of inputs that are managed by Pulumi to compare these values to live state in subsequent updates, while not showing diffs for fields that we don't manage directly. This change adds that data to the RPC protocol directly so these providers no longer need to manage these extra "__input" fields.

This adds two new properties to the RPC interfaces with providers. Firstly we pass a new bool flag to Configure to tell the provider if it will get sent old inputs as part of Diff. Secondly we now pass the old inputs and the old outputs to Diff and Update as opposed to just the old outputs.
    
The flag passed to Configure isn't strictly needed. Providers should be able to tell that the old inputs aren't being sent to Diff and Updae by virtue of them being `null/nil/None` rather than an empty map. For DiffConfig this is the only way to detect this case because it's called before Configure. But its still useful to send this explictly to Configure to allow providers to return a configure error that they need to be ran against a newer engine version.

There is one fairly major failure case that can happen with this feature and providers who update to make use of this feature should ensure they call this out in their changelog.
That is if a provider has been saving old inputs in outputs as a way to get access to them in diff, and then updates to use these new protocol properties instead and stops saving the inputs in outputs _and then_ the user downgrades back to an old provider version things will probably get _very_ confused.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-06-23 07:14:31 +00:00
Kyle Dixler 0af5877f09
pulumi state unprotect command now uses OS specific quoting
Error messages would suggest the command `pulumi state unprotect 'urn'`
on Windows which was not a valid quoting and now windows displays:
`pulumi state unprotect "urn"`.
2023-06-22 11:42:08 -07:00
Fraser Waters 841c99a9dd Send old inputs to diff and update
Fixes https://github.com/pulumi/pulumi/issues/5973

This adds two new properties to the RPC interfaces with providers.
Firstly we pass a new bool flag to Configure to tell the provider if it
will get sent old inputs as part of Diff.
Secondly we now pass the old inputs and the old outputs to Diff and
Update as opposed to just the old outputs.

The flag passed to Configure isn't strictly needed. Providers should be
able to tell that the old inputs aren't being sent to Diff and Updae by
virtue of them being `null/nil/None` rather than an empty map. For
DiffConfig this is the only way to detect this case because it's called
before Configure. But its still useful to send this explictly to
Configure to allow providers to return a configure error that they need
to be ran against a newer engine version.

There is one fairly major failure case that can happen with this feature
and providers who update to make use of this feature should ensure they
call this out in their changelog.
That is if a provider has been saving old inputs in outputs as a way to
get access to them in diff, and then updates to use these new protocol
properties instead and stops saving the inputs in outputs _and then_ the
user downgrades back to an old provider version things will probably get
_very_ confused.
2023-06-21 21:04:50 +01:00
Fraser Waters 209d279353 Don't load providers at startup
This changes the provider registry to no longer load all the providers
from the old state on startup (in `NewRegistry`) instead the load logic
has been moved to the `Same` method. The step_executor and
step_generator have been fixed up to ensure that for cases where a
resource might not have had it's provider created yet (i.e. for DBR'ing
the old version of a resource, for refreshes or deletes) they ask the
`Deployment` to look up the provider in the old state and `Same` it in the
registry.

All of the above means we only load providers we're going to use (even
taking --targets into account).

One fix mot done in this change is to auto-update providers for deletes.
That is given a program state with two resources both using V1 of a
provider, if you run the program to update one of those resource to use
V2 of the provider but to delete the other resource currently we'll
still load V1 to do that delete. It _might_ be possible (although this
is definitly questionable) to see that another resource changed it's
provider from V1 to V2 and to just assume the same change should have
happened to the deleted resource.
This could be helpful for not loading old provider versions at all, but
can be done in two passes now pretty easily. Just run `up` without any
program changes except for the SDK version bump to update all the
provider references to V2 of the provider, then do another `up` that
deletes the second resource.

Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-05-12 10:04:21 +01:00
Kyle Dixler 433dc8811f
`--target` now only creates Providers if targeted in step generation
Providers must now be specified in `--target` if they are to be created
during an update. This fixes a bug where a provider that was not
specified in `--target` was being created with a parent that did not
exist because the parent was also not specified in `--target` this would
break snapshot validation.
2023-04-28 11:25:17 -07:00
Bryce Lampe 7e0dd3f3c2
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)

When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339

Note: Older versions of pulumi may strip these fields when modifying the state.

For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).

	operationTimestamps: {
		created: ...,
		updated: ...,
		imported: ...,
	}

Fixes https://github.com/pulumi/pulumi/issues/12022
2023-03-27 09:28:26 -07:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta d8089b0c53
pkg/resource/deploy: Prefer Assertf
pkg/resource/deploy has a bunch of Assert usages
without messages.

This migrates all of them to `*f` variants,
preferring `Requiref` for paramters.

Refs 
2023-02-21 15:14:42 -08:00
Ian Wahbe bf239e04d0 Cleanup diag error 2023-01-11 15:20:35 -08:00
Sam Eiderman 4bbe365f15 Add DeletedWith resource option
In many cases there is no need to delete resources if the container
resource is going to be deleted as well.

A few examples:
 * Database object (roles, tables) when database is being deleted
 * Cloud IAM bindings when user itself is being deleted

This helps with:
 * Speeding the deletion process
 * Removing unnecessary calls to providers
 * Avoiding failed deletions when the pulumi user running the
   plan has access to the container resource but not the contained
   ones

To avoid deleting contained resources, set the `DeletedWith` resource
option to the container resource.

TODO:
 Should we support DeletedWith with PendingDeletes?
 Special case might be when the contained resource is marked as pending
 deletion but we now want to delete the container resource, so
 ultimately there is no need to delete the contained anymore
2022-10-31 12:03:18 +02:00
Fraser Waters 749d8f6c9e
Save random seeds to update plans ()
* Save random seeds to update plans

* Fix up seeds through imports

* Fix import

* Handle seed not being set in the plan

* Add missing RandomSeed to providerServer

* Add missing RandomSeed to provider
2022-08-09 17:40:59 +01:00
Fraser Waters 9dfbee3b2d
Remove sequence numbers ()
* Remove sequenceNumber from protobufs

* Regenerate protobufs

* Remove setting and reading of sequence number in Check

* Remove sequence numbers from state

* Replace sequenceNumber with randomSeed in Check

* Fix tests

* Add to CHANGELOG
2022-07-25 12:08:03 +01:00
Richard Shade a5687d16a9
Moving previewDigest to sdk/go/common/display, and exporting it. ()
* Moving previewDigest and exporting it, closes 

* Moving previewDigest and exporting it, closes 

* Updating changelog-pending

* Go Mod Tidy

* replacing to local

* more go.mod changes

* reseting go mod

* full move

* Fixing golint

* No go.mod changes needed
2022-06-27 09:08:06 -05:00
Fraser Waters 859052d6d9
Revert "Strip Aliases from state ()" ()
This reverts commit 17068e9b49.

Turns out NormalizeURNReferences needs this in the state to fix up URNs while the deployment is running. It feels like we should be able to either thread this information through to the snapshot manager another way but it's not obvious how. It's also tricky to test because snapshot code differs massively in unit tests compared to proper runs.
2022-03-24 20:08:18 +01:00
Fraser Waters 17068e9b49
Strip Aliases from state ()
* Strip Aliases from state

* chore: note fix, reason for change in changelog

* remove redundant test

see: ed2923653c/pkg/engine/lifeycletest/step_generator_test.go (L16)

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-03-23 17:55:06 -07:00
Fraser Waters ecca04d16e
Read needs a non-empty ID ()
* Read needs a non-empty ID

* Add to CHANGELOG

* Fix TestRecordingReadFailurePreviousResource

* Fix TestRecordingReadFailureNoPreviousResource

* Fix TestRecordingReadSuccessNoPreviousResource

* Fix TestRecordingReadSuccessPreviousResource

* lint
2022-03-18 17:04:54 +00:00
Fraser Waters c83aafd5e9
Import uses all input properties and warns on check failure ()
* Import uses all input properties and warns on check failure

* Add to changelog

* lint

* Change check failure message order

* Fix typo in changelog

* Move error message
2022-03-08 12:27:15 +00:00
Fraser Waters d4b16f4af0
Fix PastTense for ops ending in 'd' () 2022-03-07 11:43:07 +00:00
Fraser Waters 5d91f8f322
Add RetainOnDelete resource option ()
* Plumb in basics of retainOnDelete

* Add test

* Make test pass

* Add to changelog

* Add to API list

* lint

* Add semicolon

* Fix Infof call

* Fix method call

* new delete mode work

* cleanup

* protectTest

* Fix up test

* Fix replace

* Fix up test

* Warn on drop

* lint

* Change to just a bool flag

* Regenerate proto

* Rework to just a bool flag with no error

* Remove old comment

* Fix C# typo

* rm extra space

* Add missing semicolon

* Reformat python

* False typo

* Fix typo in js function name

* Reword docs

* lint

* Read doesn't need retainOnDelete
2022-02-16 22:11:12 +00:00
Fraser Waters 2b561d140a
Add option to set property names to import with ()
* Add optio to set property names to import with

* Change to --properties flag

* fix typo

* lint

* Add to changelog

* Refine error message
2022-02-02 13:04:59 +00:00
Fraser Waters 40eee5868e
Preview of update plans ()
* Implement resource plans in the engine

* Plumb plans through the CLI.

* Update wording

* plan renderer

* constraints

* Renames

* Update message

* fixes for rebase breaks and diffs

* WIP: outputs in plans

* fix diff

* fixup

* Liniting and test fixing

* Test and fix PropertyPath.String()

* Fix colors

* Fix cmdutil.PrintTable to handle non-simple strings

* More tests

* Readd test_plan.go

* lint

* Test expected deletes

* Test expected delete

* Test missing create

* Fix test for missing creates

* rm Paths()

* property set shrink test

* notes

* More tests

* Pop op before constraint check

* Delete plan cmd, rename arguments to preview and up

* Hide behind envvars

* typo

* Better constraint diffs

* Adds/Deletes/Updates

* Fix aliased

* Check more constraints

* fix test

* revert stack changes

* Resource sames test

* Fix same resource test

* Fix more tests

* linting

* Update pkg/cmd/pulumi/up.go

Co-authored-by: Alex Mullans <a.mullans@pulumi.com>

* Update pkg/cmd/pulumi/preview.go

Co-authored-by: Alex Mullans <a.mullans@pulumi.com>

* Auto refresh if using plans

* Fix TestGetRefreshOption

* Fix TestExplicitDeleteBeforeReplace

* lint

* More copying in tests because I do not trust myself to get mutation correct

* Small preview plan test

* Add TestPlannedUpdateChangedStack

* Revert auto-refresh changes

* Validate outputs don't change

* omitempty

* Add manifest to plan

* Add proper Plan type

* wip config work

* Config and manifest serder

* linting

* Asset NoError

* Actually check error

* Fix clone

* Test diag message

* Start on more tests

* Add String and GoString to Result

I got fed up assert errors in tests that looked like:
```
Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)}
```

It was very hard to work out at a glance what had gone wrong and I kept
having to hook a debugger just to look at what the error was.

With GoString these now print something like:
```
Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}>
}
```

Which is much more ussful.

* Add test error text

* Fix reporting of unseen op errors

* Fix unneeded deletes

* Fix unexpected deletes

* Fix up tests

* Fix merge conflict

* lint

* Fix nil map error

* Fix serialisation typo

* Diff against old inputs

* Diff against checked goal

* Diff against empty for creates

* Fix test

* inputs not outputs

* Seperate PlanDiff type

* Add properties

* Fix input diffs

* Handle creates

* lint

* Add plan message

* Clone plan for update preview

* Save and serialise env vars in plans

* lint

* pretty print json

* input output difference test

* test alias

* fix typo in for loop

* Handle resource plans with nil goal

* go mod tidy

* typo

* Auto use plans from up previews in experimental mode

* Don't preview if we have plan

* Don't run previews with plans now

* fixing tests

* Handle diffs and goals

* Update copystructure

* tests/go.sum

* Revert mod changes

* Add copystructure to tests/go.sum

* includeUnknowns

* go mod tidy

* Make plans for imports

* Remove unused function

* Move code more locally

* Handle nil in serialize

* Handle empty output diffs

* Add test for dropping computed values

* Allow computed properties to become deletes

* if out the generation of plans unless experimental mode is opt'd into

* lint

* typo

* Revert back to plans not skipping previews, this is orthognal to --skip-preview

* Trying to work out non-determinism

* Remove notes.txt

* Hacking with check idea

* Pass checked inputs back to Check from plan file

* Include resource urn in constraint error

* Give much more informative errors when plans fail

* lint

* Update expected diag strings in tests

* Remove unused code

* Duplicate Diff and DeepEquals methods for plans

* Add comment about check ops with failures

* Fix CheckedInputs comment

* OutputDiff doesn't need to be a pointer

* Fix checks against computed

* diffStringSets

* lint

* lint pkg

* Use 4 space indent

* Don't wrap Buffer in Writer

* Mark flags hidden rather than disabled

* Remove envvars from plans

* Assert MarkHidden error

* Add to changelog

* Note plan/save-plan is experimental

Co-authored-by: Pat Gavlin <pat@pulumi.com>
Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
Fraser Waters 73690cd9ba
Allow replace if either old or new protect is false ()
* Allow replace if either old or new protect is false

* Add to changelog
2022-01-29 08:57:36 +00:00
Ian Wahbe 33f9c7a1ea
Allow disabling default providers ()
* Allow disabling default providers

This is done with an opt-in setting in `pulumi config`. For example, to
disable default providers for `aws`, use:

```sh
pulumi config set --path pulumi:disable-default-providers[0] aws
```

To add `kubernetes` to the disabled list, use
```sh
pulumi config set --path pulumi:disable-default-providers[1] kubernetes
```

To disable all default providers, `*` can be used.

---

Under the hood, whenever we handle a default provider request (with
`defaultProviders.handleRequest`), we make sure it isn't on the deny
list. If it is, we replace the requested reference with a special
`DenyDefaultProvider` reference. We check for this reference whenever we
are about to get a provider to do actual work. By intercepting denied
providers when references are created, we ensure that we never use a
denied provider.

* Update CHANGELOG_PENDING.md

* Fix lints

The lint errors appear to be unrelated to the original PR. Fixing them
unblocks the CI.

* Add engine tests

* Fix nits

* Clarify function
2022-01-26 18:08:36 +01:00
Fraser Waters ec3ef44841
Make resource autonames determinstic ()
* Start adding SequenceNumber

* Start adding sequence number to state

* New generate functions

* notes

* Don't increment if unknown

* Deterministic name test

* Check replace

* typo

* lint

* Increment on targetted replace

* Some comments and external fixes

* Add test for resetting sequence number after replace

* Reset sequence numbers after replace

* assert check we never pass -1 to check

* Add to dynamic providers

* lint

* Add to changelog
2022-01-20 11:18:54 +00:00
raphaelauv dbc8ab9ad6
[FIX] unprotect cmd example ()
* [FIX] unprotect cmd example
2021-11-19 12:39:34 -08:00
Ian Wahbe 272c4643b2
Update error handling ()
This is the result of a change applied via `go-rewrap-errors`.
2021-11-12 18:37:17 -08:00
Ian Wahbe 5b7a65939b
Clarify error message language () 2021-10-11 15:21:58 -07:00
Ian Wahbe 9df8d3a028
Bold in-progress diffs diffrently ()
* Initial take on coloring in-progress diffs

* Update CHANGELOG_PENDING.md

* Add high and low color levels

* Don't print functions

* Change colorprogress to bold

* Fix @komalali's nits.

* Restore background colors comment
2021-09-28 15:16:09 -07:00
Evan Boyle f4efb7564b
Respect provider aliases () 2021-07-28 12:12:53 -07:00
Pat Gavlin 7b1d6ec1ac
Reify `Input` and `Optional` types in the schema type system. ()
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
Paul Stack 03e7601301
[cli] Provide user feedback when protected resources can't be deleted () 2021-05-18 00:43:43 +01:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Pat Gavlin 860ea4dc84
Add ignoreChanges preprocessing in `ImportStep`. ()
The step generator applies `ignoreChanges` pre-processing for all
resources by copying old input values to the new goal for any properties
mentioned in the `ignoreChanges` list. However, this pre-processing
depends on the existence of prior inputs, which by definition does not
exist for a resource being imported prior to the issuance of the
`ImportStep`. These changes add this processing to the implementation of
`ImportStep`, using the inputs read from the provider as the prior
inputs.
2020-12-17 14:46:50 -08:00
Pat Gavlin 0981df6a7a
Rename deploy.Plan to deploy.Deployment. ()
Rename deploy.Plan to deploy.Deployment.

There are two benefits to this change:

1. The name "Deployment" more accurately reflects the behavior of the
   type, which is responsible for previewing or executing a deployment.
2. Renaming this type frees up the name "Plan" for use when addressing
   .
2020-11-18 09:47:52 -08:00
Pat Gavlin 89c71bb49e
[cli] Add an import command. ()
Co-authored-by: stack72 <public@paulstack.co.uk>
2020-10-14 12:51:53 +01:00
Pat Gavlin 249140242e
Add support for provider-side preview. ()
These changes add support for provider-side previews of create and
update operations, which allows resource providers to supply output
property values for resources that are being created or updated during a
preview.

If a plugin supports provider-side preview, its create/update methods
will be invoked during previews with the `preview` property set to true.
It is the responsibility of the provider to fill in any output
properties that are known before returning. It is a best practice for
providers to only fill in property values that are guaranteed to be
identical if the preview were instead an update (i.e. only those output
properties whose values can be conclusively determined without
actually performing the create/update operation should be populated).
Providers that support previews must accept unknown values in their
create and update methods.

If a plugin does not support provider-side preview, the inputs to a
create or update operation will be propagated to the outputs as they are
today.

Fixes .
2020-10-09 13:13:55 -07:00
Paul Stack 0824fc9a8b
Revert "Narrow a few interfaces." () 2020-07-09 15:19:12 +01:00
Pat Gavlin 45d2fa95d6
Narrow a few interfaces. ()
- Remove `Info` from `Source`. This method was not used.
- Remove `Stack` from `EvalSource`. This method was not used.
- Remove `Type` and `URN` from `Step`. These values are available via
  `Res().URN.Type()` and `Res().URN`, respectively. This removes the
  possibility of inconsistencies between the type, URN, and state of the
  resource associated with a `Step`.
- Remove URN from StepEventMetadata.
2020-07-01 15:32:50 -07:00