2023-04-12 09:35:20 +00:00
|
|
|
// Copyright 2016-2023, Pulumi Corporation.
|
2018-05-22 19:43:36 +00:00
|
|
|
//
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
// you may not use this file except in compliance with the License.
|
|
|
|
// You may obtain a copy of the License at
|
|
|
|
//
|
|
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
//
|
|
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
// See the License for the specific language governing permissions and
|
|
|
|
// limitations under the License.
|
2017-06-10 01:34:37 +00:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 18:50:47 +00:00
|
|
|
package deploy
|
2017-06-10 01:34:37 +00:00
|
|
|
|
|
|
|
import (
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
"context"
|
2021-11-13 02:37:17 +00:00
|
|
|
"errors"
|
2019-04-10 20:47:54 +00:00
|
|
|
"fmt"
|
|
|
|
"strings"
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
"time"
|
2019-04-10 20:47:54 +00:00
|
|
|
|
2023-09-18 11:01:28 +00:00
|
|
|
"github.com/pulumi/pulumi/pkg/v3/display"
|
2021-03-17 13:20:05 +00:00
|
|
|
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/providers"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/diag"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/diag/colors"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
|
2017-06-10 01:34:37 +00:00
|
|
|
)
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
// StepCompleteFunc is the type of functions returned from Step.Apply. These
|
|
|
|
// functions are to be called when the engine has fully retired a step. You
|
|
|
|
// _should not_ modify the resource state in these functions -- doing so will
|
|
|
|
// race with the snapshot writing code.
|
2018-08-08 20:45:48 +00:00
|
|
|
type StepCompleteFunc func()
|
|
|
|
|
2017-06-10 01:34:37 +00:00
|
|
|
// Step is a specification for a deployment operation.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
type Step interface {
|
2024-06-14 10:19:13 +00:00
|
|
|
// Apply applies this step. It returns the status of the resource after the
|
|
|
|
// step application, a function to call to signal that this step has fully
|
|
|
|
// completed, and an error, if one occurred while applying the step.
|
2018-08-08 20:45:48 +00:00
|
|
|
//
|
2024-06-14 10:19:13 +00:00
|
|
|
// The returned StepCompleteFunc, if not nil, must be called after committing
|
|
|
|
// the results of this step into the state of the deployment.
|
|
|
|
Apply() (resource.Status, StepCompleteFunc, error)
|
|
|
|
|
|
|
|
// the operation performed by this step.
|
|
|
|
Op() display.StepOp
|
|
|
|
// the resource URN (for before and after).
|
|
|
|
URN() resource.URN
|
|
|
|
// the type of the resource affected by this step.
|
|
|
|
Type() tokens.Type
|
|
|
|
// the provider reference for the resource affected by this step.
|
|
|
|
Provider() string
|
|
|
|
// the state of the resource before performing this step.
|
|
|
|
Old() *resource.State
|
|
|
|
// the state of the resource after performing this step.
|
|
|
|
New() *resource.State
|
|
|
|
// the latest state for the resource that is known (worst case, old).
|
|
|
|
Res() *resource.State
|
|
|
|
// true if this step represents a logical operation in the program.
|
|
|
|
Logical() bool
|
|
|
|
// the deployment to which this step belongs.
|
|
|
|
Deployment() *Deployment
|
|
|
|
|
|
|
|
// Calling Fail will mark the step as failed.
|
|
|
|
Fail()
|
|
|
|
// Calling Skip will mark the step as skipped.
|
|
|
|
Skip()
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// SameStep is a mutating step that does nothing.
|
|
|
|
type SameStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the resource before this step.
|
|
|
|
new *resource.State // the state of the resource after this step.
|
2019-11-19 04:28:25 +00:00
|
|
|
|
|
|
|
// If this is a same-step for a resource being created but which was not --target'ed by the user
|
|
|
|
// (and thus was skipped).
|
|
|
|
skippedCreate bool
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
|
|
|
|
2017-11-17 02:21:41 +00:00
|
|
|
var _ Step = (*SameStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewSameStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is custom")
|
|
|
|
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
|
|
|
|
"old", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
Make sure non-targeted resources are not updated (#15476)
When the `--target` option is used, resources that already exist in the
snapshot, but aren't directly targeted should not be updated at all.
Internally in the engine, this is done by turning them into a
`SameStep`, meaning no updates will actually be preformed, and we will
make it look like the resource stayed the same.
However, we currently still write the "new" state of the resource (e.g.
updated dependencies, inputs, etc.) into the snapshot. This is mostly
fine as long as the new dependencies already exist. If a dependency on a
resource is that doesn't already exist is added however this breaks.
Since the resource that's being depended on doesn't exist in the
snapshot and isn't targeted, we won't create it. At the same time we're
adding a dependency on that virtually non-existing resource, which makes
the snapshot invalid.
Since we're in `--target` mode, we should do what we promised the user,
and only update the targeted resources, nothing else. Introduce a new
`NonTargetedSameStep` here, which does exactly that. It's essentially
the same as a `SameStep`, but we always use the *old* state instead of
the new one when writing it out. Since the resource is not targeted,
this leaves it in the same state as before.
Fixes #12096
Fixes #15382
2024-03-05 07:49:11 +00:00
|
|
|
contract.Requiref(new == old || new.ID == "", "new", "must not have an ID")
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
|
|
|
|
"new", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return &SameStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
reg: reg,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-11-19 04:28:25 +00:00
|
|
|
// NewSkippedCreateStep produces a SameStep for a resource that was created but not targeted
|
|
|
|
// by the user (and thus was skipped). These act as no-op steps (hence 'same') since we are not
|
|
|
|
// actually creating the resource, but ensure that we complete resource-registration and convey the
|
|
|
|
// right information downstream. For example, we will not write these into the checkpoint file.
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewSkippedCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID == "", "new", "must not have an ID")
|
|
|
|
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
|
|
|
|
"new", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
2019-11-19 04:28:25 +00:00
|
|
|
|
|
|
|
// Make the old state here a direct copy of the new state
|
2024-05-09 16:15:41 +00:00
|
|
|
old := new.Copy()
|
2019-11-19 04:28:25 +00:00
|
|
|
return &SameStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2019-11-19 04:28:25 +00:00
|
|
|
reg: reg,
|
2024-05-09 16:15:41 +00:00
|
|
|
old: old,
|
2019-11-19 04:28:25 +00:00
|
|
|
new: new,
|
|
|
|
skippedCreate: true,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *SameStep) Op() display.StepOp { return OpSame }
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *SameStep) Deployment() *Deployment { return s.deployment }
|
|
|
|
func (s *SameStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *SameStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *SameStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *SameStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *SameStep) New() *resource.State { return s.new }
|
|
|
|
func (s *SameStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *SameStep) Logical() bool { return true }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *SameStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2024-05-09 16:15:41 +00:00
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2021-07-28 19:12:53 +00:00
|
|
|
// Retain the ID and outputs
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 22:31:07 +00:00
|
|
|
s.new.ID = s.old.ID
|
|
|
|
s.new.Outputs = s.old.Outputs
|
2021-07-28 19:12:53 +00:00
|
|
|
|
|
|
|
// If the resource is a provider, ensure that it is present in the registry under the appropriate URNs.
|
2023-04-07 22:40:41 +00:00
|
|
|
// We can only do this if the provider is actually a same, not a skipped create.
|
|
|
|
if providers.IsProviderType(s.new.Type) && !s.skippedCreate {
|
2021-07-28 19:12:53 +00:00
|
|
|
if s.Deployment() != nil {
|
2023-04-12 09:35:20 +00:00
|
|
|
err := s.Deployment().SameProvider(s.new)
|
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil,
|
2023-12-20 15:54:06 +00:00
|
|
|
fmt.Errorf("bad provider state for resource %v: %w", s.URN(), err)
|
2023-04-12 09:35:20 +00:00
|
|
|
}
|
2021-07-28 19:12:53 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
// TODO: should this step be marked as skipped if it comes from a targeted up?
|
Propagate deleted dependencies of untargeted resources (#16247)
When using `--target` to target specific resources during an update, we
use the list of targets to decide which steps to generate given a set of
resource registrations. Specifically:
* If the registration event names a resource that is targeted, we
process it as usual.
* If the registration event names a resource that _is not_ targeted, we
emit a `SameStep` for it.
In the latter case, the emission of a `SameStep` means that the old
state for the resource will be copied across to the new state. This is
the desired behaviour -- the resource was not targeted and so the new
state should contain the resource exactly as it was prior to the update.
However, this presents a problem if the old state has references to
resources that either will not appear in the new state, or will appear
in the wrong place. Consider the following program in TypeScript-esque
pseudocode:
```typescript
const a = new Resource("a")
const b = new Resource("b", { dependency: a })
const c = new Resource("c")
```
Here, `b` depends on `a`, while `a` and `c` have no dependencies. We run
this program without specifying targets and obtain a state containing
`a`, `b` and `c`, with `a` appearing before `b` due to `b`'s dependency
on `a`. We now modify the program as follows:
```typescript
const b = new Resource("b")
const c = new Resource("c")
```
`a` has been removed from the program and consequently `b` no longer
depends on it. We once more run the program, this time with a `--target`
of `c`. That is to say, neither `a` nor `b` is targeted. The execution
proceeds as follows:
* `a` is not in the program, so no `RegisterResourceEvent` will be
emitted and processed for it.
* `b` is in the program, but it is not targeted. Its
`RegisterResourceEvent` will be turned into a `SameStep` and `b`'s _old
state will be copied as-is to the new state_.
* `c` is in the program and is targeted. It will be processed as normal.
At the end of execution when we come to write the snapshot, we take the
following actions:
* We first write the processed resources: `b`'s old state and `c`'s new
state.
* We then copy over any unprocessed resources from the base (previous)
snapshot. This includes `a` (which is again desirable since its deletion
should not be processed due to it not being targeted).
Our snapshot is now not topologically sorted and thus invalid: `b` has a
dependency on `a`, but `a` appears after it. Presently this bug will
manifest irrespective of the nature of the dependency: `.Dependencies`,
`.PropertyDependencies` and `.DeletedWith` are all affected.
This commit fixes this issue by traversing all untargeted resource
dependency relationships and ensuring that `SameStep`s (or better if
they have been targeted) are emitted before emitting the depending
resource's `SameStep`.
* Fixes #16052
* Fixes #15959
2024-05-23 12:31:03 +00:00
|
|
|
complete := func() {
|
|
|
|
// It's possible that s.reg will be nil in the case that multiple same steps
|
|
|
|
// are emitted for a single RegisterResourceEvent. This occurs when a
|
|
|
|
// resource which is not targeted by a targeted operation needs to ensure
|
|
|
|
// that old dependencies are brought forward into the new state before it
|
|
|
|
// is. In these cases the root resource will be instantiated with its event
|
|
|
|
// while its dependencies will have a nil event. This is fine since in these
|
|
|
|
// cases the only Done callback we care about is the one for the root
|
|
|
|
// resource.
|
|
|
|
if s.reg != nil {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new})
|
|
|
|
}
|
|
|
|
}
|
2018-08-08 20:45:48 +00:00
|
|
|
return resource.StatusOK, complete, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2019-11-19 04:28:25 +00:00
|
|
|
func (s *SameStep) IsSkippedCreate() bool {
|
|
|
|
return s.skippedCreate
|
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *SameStep) Fail() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *SameStep) Skip() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
|
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
// CreateStep is a mutating step that creates an entirely new resource.
|
|
|
|
type CreateStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
2019-07-01 19:34:19 +00:00
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the existing resource (only for replacements).
|
|
|
|
new *resource.State // the state of the resource after this step.
|
|
|
|
keys []resource.PropertyKey // the keys causing replacement (only for replacements).
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff (only for replacements).
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff (only for replacements).
|
|
|
|
replacing bool // true if this is a create due to a replacement.
|
|
|
|
pendingDelete bool // true if this replacement should create a pending delete.
|
2023-12-22 21:14:04 +00:00
|
|
|
provider plugin.Provider // the optional provider to use.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2017-11-17 02:21:41 +00:00
|
|
|
var _ Step = (*CreateStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(reg != nil, "reg", "must not be nil")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID == "", "new", "must not have an ID")
|
|
|
|
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
|
|
|
|
"new", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be external")
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return &CreateStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
reg: reg,
|
|
|
|
new: new,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewCreateReplacementStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
|
2023-03-03 16:36:39 +00:00
|
|
|
keys, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(reg != nil, "reg", "must not be nil")
|
|
|
|
|
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID == "", "new", "must not have an ID")
|
|
|
|
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
|
|
|
|
"new", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be external")
|
|
|
|
|
2017-08-06 17:05:51 +00:00
|
|
|
return &CreateStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2017-12-10 16:37:22 +00:00
|
|
|
reg: reg,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
keys: keys,
|
2019-03-07 00:41:19 +00:00
|
|
|
diffs: diffs,
|
2019-07-01 19:34:19 +00:00
|
|
|
detailedDiff: detailedDiff,
|
2017-12-10 16:37:22 +00:00
|
|
|
replacing: true,
|
|
|
|
pendingDelete: pendingDelete,
|
2017-08-06 17:05:51 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *CreateStep) Op() display.StepOp {
|
2017-08-06 17:05:51 +00:00
|
|
|
if s.replacing {
|
|
|
|
return OpCreateReplacement
|
|
|
|
}
|
|
|
|
return OpCreate
|
|
|
|
}
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *CreateStep) Deployment() *Deployment { return s.deployment }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *CreateStep) Type() tokens.Type { return s.new.Type }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *CreateStep) Provider() string { return s.new.Provider }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *CreateStep) URN() resource.URN { return s.new.URN }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *CreateStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *CreateStep) New() *resource.State { return s.new }
|
|
|
|
func (s *CreateStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *CreateStep) Keys() []resource.PropertyKey { return s.keys }
|
|
|
|
func (s *CreateStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *CreateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
func (s *CreateStep) Logical() bool { return !s.replacing }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *CreateStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2018-05-02 17:36:55 +00:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
2024-04-11 22:54:08 +00:00
|
|
|
|
|
|
|
id := s.new.ID
|
|
|
|
outs := s.new.Outputs
|
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
if s.new.Custom {
|
|
|
|
// Invoke the Create RPC function for this provider:
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err := getProvider(s, s.provider)
|
2020-10-09 20:13:55 +00:00
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 21:26:28 +00:00
|
|
|
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resp, err := prov.Create(context.TODO(), plugin.CreateRequest{
|
|
|
|
URN: s.URN(),
|
|
|
|
Properties: s.new.Inputs,
|
|
|
|
Timeout: s.new.CustomTimeouts.Create,
|
Clean up deployment options (#16357)
# Description
There are a number of parts of the deployment process that require
context about and configuration for the operation being executed. For
instance:
* Source evaluation -- evaluating programs in order to emit resource
registrations
* Step generation -- processing resource registrations in order to
generate steps (create this, update that, delete the other, etc.)
* Step execution -- executing steps in order to action a deployment.
Presently, these pieces all take some form of `Options` struct or pass
explicit arguments. This is problematic for a couple of reasons:
* It could be possible for different parts of the codebase to end up
operating in different contexts/with different configurations, whether
due to different values being passed explicitly or due to missed
copying/instantiation.
* Some parts need less context/configuration than others, but still
accept full `Options`, making it hard to discern what information is
actually necessary in any given part of the process.
This commit attempts to clean things up by moving deployment options
directly into the `Deployment` itself. Since step generation and
execution already refer to a `Deployment`, they get a consistent view of
the options for free. For source evaluation, we introduce an
`EvalSourceOptions` struct for configuring just the options necessary
there. At the top level, the engine configures a single set of options
to flow through the deployment steps later on.
As part of this work, a few other things have been changed:
* Preview/dry-run parameters have been incorporated into options. This
lets up lop off another argument and mitigate a bit of "boolean
blindness". We don't appear to flip this flag within a deployment
process (indeed, all options seem to be immutable) and so having it as a
separate flag doesn't seem to buy us anything.
* Several methods representing parts of the deployment process have lost
arguments in favour of state that is already being carried on (or can be
carried on) their receiver. For instance, `deployment.run` no longer
takes actions or preview configuration. While doing so means that a
`deployment` could be run multiple times with different actions/preview
arguments, we don't currently exploit this fact anywhere, so moving this
state to the point of construction both simplifies things considerably
and reduces the possibility for error (e.g. passing different values of
`preview` when instantiating a `deployment` and subsequently calling
`run`).
* Event handlers have been split out of the options object and attached
to `Deployment` separately. This means we can talk about options at a
higher level without having to `nil` out/worry about this field and
mutate it correctly later on.
* Options are no longer mutated during deployment. Presently there
appears to be only one case of this -- when handling `ContinueOnError`
in the presence of `IgnoreChanges` (e.g. when performing a refresh).
This case has been refactored so that the mutation is no longer
necessary.
# Notes
* This change is in preparation for #16146, where we'd like to add an
environment variable to control behaviour and having a single unified
`Options` struct would make it easier to pass this configuration down
with introducing (more) global state into deployments. Indeed, this
change should make it easier to factor global state into `Options` so
that it can be controlled and tested more easily/is less susceptible to
bugs, race conditions, etc.
* I've tweaked/extended some comments while I'm here and have learned
things the hard way (e.g. `Refresh` vs `isRefresh`). Feedback welcome on
this if we'd rather not conflate.
* This change does mean that if in future we wanted e.g. to be able to
run a `Deployment` in multiple different ways with multiple sets of
actions, we'd have to refactor. Pushing state to the point of object
construction reduces the flexibility of the code. However, since we are
not presently using that flexibility (nor is there an obvious [to my
mind] use case in the near future), this seems like a good trade-off to
guard against bugs/make it simpler to move that state around.
* I've left some other review comments in the code around
questions/changes that might be a bad idea; happy to receive feedback on
it all though!
2024-06-11 13:37:57 +00:00
|
|
|
Preview: s.deployment.opts.DryRun,
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
})
|
2020-10-09 20:13:55 +00:00
|
|
|
if err != nil {
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if resp.Status != resource.StatusPartialFailure {
|
|
|
|
return resp.Status, nil, err
|
2020-10-09 20:13:55 +00:00
|
|
|
}
|
2018-05-02 17:36:55 +00:00
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
resourceError = err
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resourceStatus = resp.Status
|
2018-07-19 05:52:01 +00:00
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
2017-11-29 23:05:58 +00:00
|
|
|
}
|
2020-10-09 20:13:55 +00:00
|
|
|
}
|
2018-05-02 17:36:55 +00:00
|
|
|
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
id = resp.ID
|
|
|
|
outs = resp.Properties
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
if !s.deployment.opts.DryRun && id == "" {
|
2023-12-12 12:19:42 +00:00
|
|
|
return resourceStatus, nil, errors.New("provider did not return an ID from Create")
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2024-05-09 16:15:41 +00:00
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2024-04-11 22:54:08 +00:00
|
|
|
// Copy any of the default and output properties on the live object state.
|
|
|
|
s.new.ID = id
|
|
|
|
s.new.Outputs = outs
|
|
|
|
|
2023-09-25 18:22:23 +00:00
|
|
|
// Create should set the Create and Modified timestamps as the resource state has been created.
|
|
|
|
now := time.Now().UTC()
|
|
|
|
s.new.Created = &now
|
|
|
|
s.new.Modified = &now
|
|
|
|
|
2017-10-05 21:08:35 +00:00
|
|
|
// Mark the old resource as pending deletion if necessary.
|
2017-12-10 16:37:22 +00:00
|
|
|
if s.replacing && s.pendingDelete {
|
2024-04-11 22:54:08 +00:00
|
|
|
contract.Assertf(s.old != s.new, "old and new states should not be the same")
|
2024-05-09 16:15:41 +00:00
|
|
|
s.old.Lock.Lock()
|
2017-10-05 21:08:35 +00:00
|
|
|
s.old.Delete = true
|
2024-05-09 16:15:41 +00:00
|
|
|
s.old.Lock.Unlock()
|
2017-10-05 21:08:35 +00:00
|
|
|
}
|
|
|
|
|
2023-09-25 18:22:23 +00:00
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR ensures that we only run the `StepCompleteFunc` if the step is
successful. If not, we should use the step's `Fail` method to register
the result. Previously, when a user runs a pulumi up operation with
`--continue-on-error`, the engine would register the result twice, once
in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342
and if a failure occurs, it is re-registered in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373
I suspect that the double registration is the cause of the user facing
error. Since this would only occur for create and update steps, it also
explains why the error doesn't occur during deletions.
Testing:
- Manually ran the repro identified in #16373 and
https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614
against a locally built CLI with this fix.
- Extended existing lifecycle tests to account for the different code
branches of `CreateStep.Apply()` and `UpdateStep.Apply()`
Fixes #16373
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-13 15:42:57 +00:00
|
|
|
|
|
|
|
if resourceError != nil {
|
|
|
|
// If we have a failure, we should return an empty complete function
|
|
|
|
// and let the Fail method handle the registration.
|
|
|
|
return resourceStatus, nil, resourceError
|
2018-07-06 22:17:32 +00:00
|
|
|
}
|
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR ensures that we only run the `StepCompleteFunc` if the step is
successful. If not, we should use the step's `Fail` method to register
the result. Previously, when a user runs a pulumi up operation with
`--continue-on-error`, the engine would register the result twice, once
in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342
and if a failure occurs, it is re-registered in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373
I suspect that the double registration is the cause of the user facing
error. Since this would only occur for create and update steps, it also
explains why the error doesn't occur during deletions.
Testing:
- Manually ran the repro identified in #16373 and
https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614
against a locally built CLI with this fix.
- Extended existing lifecycle tests to account for the different code
branches of `CreateStep.Apply()` and `UpdateStep.Apply()`
Fixes #16373
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-13 15:42:57 +00:00
|
|
|
|
|
|
|
return resourceStatus, complete, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *CreateStep) Fail() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *CreateStep) Skip() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
|
|
|
|
}
|
|
|
|
|
2018-08-03 21:06:00 +00:00
|
|
|
// DeleteStep is a mutating step that deletes an existing resource. If `old` is marked "External",
|
|
|
|
// DeleteStep is a no-op.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
type DeleteStep struct {
|
2022-10-20 06:15:43 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
replacing bool // true if part of a replacement.
|
|
|
|
otherDeletions map[resource.URN]bool // other resources that are planned to delete
|
2023-12-22 21:14:04 +00:00
|
|
|
provider plugin.Provider // the optional provider to use.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2017-11-17 02:21:41 +00:00
|
|
|
var _ Step = (*DeleteStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2022-10-20 06:15:43 +00:00
|
|
|
func NewDeleteStep(deployment *Deployment, otherDeletions map[resource.URN]bool, old *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
|
|
|
|
"old", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
|
2017-12-10 16:37:22 +00:00
|
|
|
return &DeleteStep{
|
2022-10-20 06:15:43 +00:00
|
|
|
deployment: deployment,
|
|
|
|
old: old,
|
|
|
|
otherDeletions: otherDeletions,
|
2017-12-10 16:37:22 +00:00
|
|
|
}
|
|
|
|
}
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 18:22:39 +00:00
|
|
|
|
2022-10-20 06:15:43 +00:00
|
|
|
func NewDeleteReplacementStep(
|
|
|
|
deployment *Deployment,
|
|
|
|
otherDeletions map[resource.URN]bool,
|
|
|
|
old *resource.State,
|
|
|
|
pendingReplace bool,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
|
|
|
|
"old", "must have or be a provider if it is a custom resource")
|
|
|
|
|
|
|
|
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
|
|
|
|
// There are two cases in which we create a delete-replacment step:
|
|
|
|
//
|
|
|
|
// 1. When creating the delete steps that occur due to a delete-before-replace
|
|
|
|
// 2. When creating the delete step that occurs due to a delete-after-replace
|
|
|
|
//
|
|
|
|
// In the former case, the persistence layer may require that the resource remain in the
|
|
|
|
// checkpoint file for purposes of checkpoint integrity. We communicate this case by means
|
|
|
|
// of the `PendingReplacement` field on `resource.State`, which we set here.
|
|
|
|
//
|
|
|
|
// In the latter case, the resource must be deleted, but the deletion may not occur if an earlier step fails.
|
|
|
|
// The engine requires that the fact that the old resource must be deleted is persisted in the checkpoint so
|
|
|
|
// that it can issue a deletion of this resource on the next update to this stack.
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Assertf(pendingReplace != old.Delete,
|
|
|
|
"resource %v cannot be pending replacement and deletion at the same time", old.URN)
|
2024-05-09 16:15:41 +00:00
|
|
|
old.Lock.Lock()
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
old.PendingReplacement = pendingReplace
|
2024-05-09 16:15:41 +00:00
|
|
|
old.Lock.Unlock()
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return &DeleteStep{
|
2022-10-20 06:15:43 +00:00
|
|
|
deployment: deployment,
|
|
|
|
otherDeletions: otherDeletions,
|
|
|
|
old: old,
|
|
|
|
replacing: true,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *DeleteStep) Op() display.StepOp {
|
2019-01-31 21:48:44 +00:00
|
|
|
if s.old.External {
|
|
|
|
if s.replacing {
|
|
|
|
return OpDiscardReplaced
|
|
|
|
}
|
|
|
|
return OpReadDiscard
|
|
|
|
}
|
|
|
|
|
2017-08-06 17:05:51 +00:00
|
|
|
if s.replacing {
|
|
|
|
return OpDeleteReplaced
|
|
|
|
}
|
|
|
|
return OpDelete
|
|
|
|
}
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *DeleteStep) Deployment() *Deployment { return s.deployment }
|
|
|
|
func (s *DeleteStep) Type() tokens.Type { return s.old.Type }
|
|
|
|
func (s *DeleteStep) Provider() string { return s.old.Provider }
|
|
|
|
func (s *DeleteStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *DeleteStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *DeleteStep) New() *resource.State { return nil }
|
|
|
|
func (s *DeleteStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *DeleteStep) Logical() bool { return !s.replacing }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2022-10-20 06:15:43 +00:00
|
|
|
func isDeletedWith(with resource.URN, otherDeletions map[resource.URN]bool) bool {
|
|
|
|
if with == "" {
|
|
|
|
return false
|
|
|
|
}
|
|
|
|
r, ok := otherDeletions[with]
|
|
|
|
if !ok {
|
|
|
|
return false
|
|
|
|
}
|
|
|
|
return r
|
|
|
|
}
|
|
|
|
|
2023-01-10 18:32:12 +00:00
|
|
|
type deleteProtectedError struct {
|
|
|
|
urn resource.URN
|
|
|
|
}
|
|
|
|
|
|
|
|
func (d deleteProtectedError) Error() string {
|
|
|
|
return fmt.Sprintf("resource %[1]q cannot be deleted\n"+
|
|
|
|
"because it is protected. To unprotect the resource, "+
|
|
|
|
"either remove the `protect` flag from the resource in your Pulumi "+
|
|
|
|
"program and run `pulumi up`, or use the command:\n"+
|
2023-06-21 20:46:36 +00:00
|
|
|
"`pulumi state unprotect %[2]s`", d.urn, d.urn.Quote())
|
2023-01-10 18:32:12 +00:00
|
|
|
}
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *DeleteStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2022-01-29 08:57:36 +00:00
|
|
|
// Refuse to delete protected resources (unless we're replacing them in
|
|
|
|
// which case we will of checked protect elsewhere)
|
|
|
|
if !s.replacing && s.old.Protect {
|
2023-01-10 18:32:12 +00:00
|
|
|
return resource.StatusOK, nil, deleteProtectedError{urn: s.old.URN}
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 22:31:07 +00:00
|
|
|
}
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
if s.deployment.opts.DryRun {
|
2022-02-16 22:11:12 +00:00
|
|
|
// Do nothing in preview
|
|
|
|
} else if s.old.External {
|
|
|
|
// Deleting an External resource is a no-op, since Pulumi does not own the lifecycle.
|
|
|
|
} else if s.old.RetainOnDelete {
|
|
|
|
// Deleting a "drop on delete" is a no-op as the user has explicitly asked us to not delete the resource.
|
2022-10-20 06:15:43 +00:00
|
|
|
} else if isDeletedWith(s.old.DeletedWith, s.otherDeletions) {
|
|
|
|
// No need to delete this resource since this resource will be deleted by the another deletion
|
2022-02-16 22:11:12 +00:00
|
|
|
} else if s.old.Custom {
|
|
|
|
// Not preview and not external and not Drop and is custom, do the actual delete
|
|
|
|
|
|
|
|
// Invoke the Delete RPC function for this provider:
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err := getProvider(s, s.provider)
|
2022-02-16 22:11:12 +00:00
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 21:26:28 +00:00
|
|
|
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if rst, err := prov.Delete(context.TODO(), plugin.DeleteRequest{
|
|
|
|
URN: s.URN(),
|
|
|
|
ID: s.old.ID,
|
|
|
|
Inputs: s.old.Inputs,
|
|
|
|
Outputs: s.old.Outputs,
|
|
|
|
Timeout: s.old.CustomTimeouts.Delete,
|
|
|
|
}); err != nil {
|
|
|
|
return rst.Status, nil, err
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
2017-11-29 23:05:58 +00:00
|
|
|
|
2018-08-08 20:45:48 +00:00
|
|
|
return resource.StatusOK, func() {}, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *DeleteStep) Fail() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *DeleteStep) Skip() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
type RemovePendingReplaceStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
|
|
|
old *resource.State // the state of the existing resource.
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewRemovePendingReplaceStep(deployment *Deployment, old *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.PendingReplacement, "old", "must be pending replacement")
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
return &RemovePendingReplaceStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
old: old,
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *RemovePendingReplaceStep) Op() display.StepOp {
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
return OpRemovePendingReplace
|
|
|
|
}
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *RemovePendingReplaceStep) Deployment() *Deployment { return s.deployment }
|
|
|
|
func (s *RemovePendingReplaceStep) Type() tokens.Type { return s.old.Type }
|
|
|
|
func (s *RemovePendingReplaceStep) Provider() string { return s.old.Provider }
|
|
|
|
func (s *RemovePendingReplaceStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *RemovePendingReplaceStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *RemovePendingReplaceStep) New() *resource.State { return nil }
|
|
|
|
func (s *RemovePendingReplaceStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *RemovePendingReplaceStep) Logical() bool { return false }
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *RemovePendingReplaceStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
return resource.StatusOK, nil, nil
|
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *RemovePendingReplaceStep) Fail() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *RemovePendingReplaceStep) Skip() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
// UpdateStep is a mutating step that updates an existing resource's state.
|
|
|
|
type UpdateStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
2019-07-31 16:39:07 +00:00
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
new *resource.State // the newly computed state of the resource after updating.
|
|
|
|
stables []resource.PropertyKey // an optional list of properties that won't change during this update.
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured diff.
|
|
|
|
ignoreChanges []string // a list of property paths to ignore when updating.
|
2023-12-22 21:14:04 +00:00
|
|
|
provider plugin.Provider // the optional provider to use.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2017-11-17 02:21:41 +00:00
|
|
|
var _ Step = (*UpdateStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewUpdateStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
|
|
|
|
stables, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff,
|
2023-03-03 16:36:39 +00:00
|
|
|
ignoreChanges []string,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
|
|
|
|
"old", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!old.External, "old", "must not be an external resource")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID == "", "new", "must not have an ID")
|
|
|
|
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
|
|
|
|
"new", "must have or be a provider if it is a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be an external resource")
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return &UpdateStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2019-07-31 16:39:07 +00:00
|
|
|
reg: reg,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
stables: stables,
|
|
|
|
diffs: diffs,
|
|
|
|
detailedDiff: detailedDiff,
|
|
|
|
ignoreChanges: ignoreChanges,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *UpdateStep) Op() display.StepOp { return OpUpdate }
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *UpdateStep) Deployment() *Deployment { return s.deployment }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *UpdateStep) Type() tokens.Type { return s.new.Type }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *UpdateStep) Provider() string { return s.new.Provider }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *UpdateStep) URN() resource.URN { return s.new.URN }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *UpdateStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *UpdateStep) New() *resource.State { return s.new }
|
|
|
|
func (s *UpdateStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *UpdateStep) Logical() bool { return true }
|
|
|
|
func (s *UpdateStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *UpdateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *UpdateStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
// Always propagate the ID and timestamps even in previews and refreshes.
|
2024-05-09 16:15:41 +00:00
|
|
|
s.new.Lock.Lock()
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 18:22:39 +00:00
|
|
|
s.new.ID = s.old.ID
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
s.new.Created = s.old.Created
|
|
|
|
s.new.Modified = s.old.Modified
|
2024-05-09 16:15:41 +00:00
|
|
|
s.new.Lock.Unlock()
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 18:22:39 +00:00
|
|
|
|
2018-05-02 17:36:55 +00:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
2020-10-09 20:13:55 +00:00
|
|
|
if s.new.Custom {
|
|
|
|
// Invoke the Update RPC function for this provider:
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err := getProvider(s, s.provider)
|
2020-10-09 20:13:55 +00:00
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
General prep work for refresh
This change includes a bunch of refactorings I made in prep for
doing refresh (first, the command, see pulumi/pulumi#1081):
* The primary change is to change the way the engine's core update
functionality works with respect to deploy.Source. This is the
way we can plug in new sources of resource information during
planning (and, soon, diffing). The way I intend to model refresh
is by having a new kind of source, deploy.RefreshSource, which
will let us do virtually everything about an update/diff the same
way with refreshes, which avoid otherwise duplicative effort.
This includes changing the planOptions (nee deployOptions) to
take a new SourceFunc callback, which is responsible for creating
a source specific to the kind of plan being requested.
Preview, Update, and Destroy now are primarily differentiated by
the kind of deploy.Source that they return, rather than sprinkling
things like `if Destroying` throughout. This tidies up some logic
and, more importantly, gives us precisely the refresh hook we need.
* Originally, we used the deploy.NullSource for Destroy operations.
This simply returns nothing, which is how Destroy works. For some
reason, we were no longer doing this, and instead had some
`if Destroying` cases sprinkled throughout the deploy.EvalSource.
I think this is a vestige of some old way we did configuration, at
least judging by a comment, which is apparently no longer relevant.
* Move diff and diff-printing logic within the engine into its own
pkg/engine/diff.go file, to prepare for upcoming work.
* I keep noticing benign diffs anytime I regenerate protobufs. I
suspect this is because we're also on different versions. I changed
generate.sh to also dump the version into grpc_version.txt. At
least we can understand where the diffs are coming from, decide
whether to take them (i.e., a newer version), and ensure that as
a team we are monotonically increasing, and not going backwards.
* I also tidied up some tiny things I noticed while in there, like
comments, incorrect types, lint suppressions, and so on.
2018-03-28 14:45:23 +00:00
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
// Update to the combination of the old "all" state, but overwritten with new inputs.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resp, upderr := prov.Update(context.TODO(), plugin.UpdateRequest{
|
|
|
|
URN: s.URN(),
|
|
|
|
ID: s.old.ID,
|
|
|
|
OldInputs: s.old.Inputs,
|
|
|
|
OldOutputs: s.old.Outputs,
|
|
|
|
NewInputs: s.new.Inputs,
|
|
|
|
Timeout: s.new.CustomTimeouts.Update,
|
|
|
|
IgnoreChanges: s.ignoreChanges,
|
Clean up deployment options (#16357)
# Description
There are a number of parts of the deployment process that require
context about and configuration for the operation being executed. For
instance:
* Source evaluation -- evaluating programs in order to emit resource
registrations
* Step generation -- processing resource registrations in order to
generate steps (create this, update that, delete the other, etc.)
* Step execution -- executing steps in order to action a deployment.
Presently, these pieces all take some form of `Options` struct or pass
explicit arguments. This is problematic for a couple of reasons:
* It could be possible for different parts of the codebase to end up
operating in different contexts/with different configurations, whether
due to different values being passed explicitly or due to missed
copying/instantiation.
* Some parts need less context/configuration than others, but still
accept full `Options`, making it hard to discern what information is
actually necessary in any given part of the process.
This commit attempts to clean things up by moving deployment options
directly into the `Deployment` itself. Since step generation and
execution already refer to a `Deployment`, they get a consistent view of
the options for free. For source evaluation, we introduce an
`EvalSourceOptions` struct for configuring just the options necessary
there. At the top level, the engine configures a single set of options
to flow through the deployment steps later on.
As part of this work, a few other things have been changed:
* Preview/dry-run parameters have been incorporated into options. This
lets up lop off another argument and mitigate a bit of "boolean
blindness". We don't appear to flip this flag within a deployment
process (indeed, all options seem to be immutable) and so having it as a
separate flag doesn't seem to buy us anything.
* Several methods representing parts of the deployment process have lost
arguments in favour of state that is already being carried on (or can be
carried on) their receiver. For instance, `deployment.run` no longer
takes actions or preview configuration. While doing so means that a
`deployment` could be run multiple times with different actions/preview
arguments, we don't currently exploit this fact anywhere, so moving this
state to the point of construction both simplifies things considerably
and reduces the possibility for error (e.g. passing different values of
`preview` when instantiating a `deployment` and subsequently calling
`run`).
* Event handlers have been split out of the options object and attached
to `Deployment` separately. This means we can talk about options at a
higher level without having to `nil` out/worry about this field and
mutate it correctly later on.
* Options are no longer mutated during deployment. Presently there
appears to be only one case of this -- when handling `ContinueOnError`
in the presence of `IgnoreChanges` (e.g. when performing a refresh).
This case has been refactored so that the mutation is no longer
necessary.
# Notes
* This change is in preparation for #16146, where we'd like to add an
environment variable to control behaviour and having a single unified
`Options` struct would make it easier to pass this configuration down
with introducing (more) global state into deployments. Indeed, this
change should make it easier to factor global state into `Options` so
that it can be controlled and tested more easily/is less susceptible to
bugs, race conditions, etc.
* I've tweaked/extended some comments while I'm here and have learned
things the hard way (e.g. `Refresh` vs `isRefresh`). Feedback welcome on
this if we'd rather not conflate.
* This change does mean that if in future we wanted e.g. to be able to
run a `Deployment` in multiple different ways with multiple sets of
actions, we'd have to refactor. Pushing state to the point of object
construction reduces the flexibility of the code. However, since we are
not presently using that flexibility (nor is there an obvious [to my
mind] use case in the near future), this seems like a good trade-off to
guard against bugs/make it simpler to move that state around.
* I've left some other review comments in the code around
questions/changes that might be a bad idea; happy to receive feedback on
it all though!
2024-06-11 13:37:57 +00:00
|
|
|
Preview: s.deployment.opts.DryRun,
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
})
|
2024-05-09 16:15:41 +00:00
|
|
|
|
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
if upderr != nil {
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if resp.Status != resource.StatusPartialFailure {
|
|
|
|
return resp.Status, nil, upderr
|
2020-10-09 20:13:55 +00:00
|
|
|
}
|
2018-05-02 17:36:55 +00:00
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
resourceError = upderr
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resourceStatus = resp.Status
|
2018-07-19 05:52:01 +00:00
|
|
|
|
2020-10-09 20:13:55 +00:00
|
|
|
if initErr, isInitErr := upderr.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
2017-12-01 00:37:44 +00:00
|
|
|
}
|
|
|
|
}
|
2020-10-09 20:13:55 +00:00
|
|
|
|
|
|
|
// Now copy any output state back in case the update triggered cascading updates to other properties.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
s.new.Outputs = resp.Properties
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
// UpdateStep doesn't create, but does modify state.
|
|
|
|
// Change the Modified timestamp.
|
|
|
|
now := time.Now().UTC()
|
|
|
|
s.new.Modified = &now
|
|
|
|
}
|
2023-09-25 18:22:23 +00:00
|
|
|
|
|
|
|
// Finally, mark this operation as complete.
|
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR ensures that we only run the `StepCompleteFunc` if the step is
successful. If not, we should use the step's `Fail` method to register
the result. Previously, when a user runs a pulumi up operation with
`--continue-on-error`, the engine would register the result twice, once
in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342
and if a failure occurs, it is re-registered in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373
I suspect that the double registration is the cause of the user facing
error. Since this would only occur for create and update steps, it also
explains why the error doesn't occur during deletions.
Testing:
- Manually ran the repro identified in #16373 and
https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614
against a locally built CLI with this fix.
- Extended existing lifecycle tests to account for the different code
branches of `CreateStep.Apply()` and `UpdateStep.Apply()`
Fixes #16373
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-13 15:42:57 +00:00
|
|
|
|
|
|
|
if resourceError != nil {
|
|
|
|
// If we have a failure, we should return an empty complete function
|
|
|
|
// and let the Fail method handle the registration.
|
|
|
|
return resourceStatus, nil, resourceError
|
2018-07-06 22:17:32 +00:00
|
|
|
}
|
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR ensures that we only run the `StepCompleteFunc` if the step is
successful. If not, we should use the step's `Fail` method to register
the result. Previously, when a user runs a pulumi up operation with
`--continue-on-error`, the engine would register the result twice, once
in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342
and if a failure occurs, it is re-registered in
https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373
I suspect that the double registration is the cause of the user facing
error. Since this would only occur for create and update steps, it also
explains why the error doesn't occur during deletions.
Testing:
- Manually ran the repro identified in #16373 and
https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614
against a locally built CLI with this fix.
- Extended existing lifecycle tests to account for the different code
branches of `CreateStep.Apply()` and `UpdateStep.Apply()`
Fixes #16373
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-13 15:42:57 +00:00
|
|
|
return resourceStatus, complete, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *UpdateStep) Fail() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *UpdateStep) Skip() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
|
|
|
|
}
|
|
|
|
|
2017-08-06 17:05:51 +00:00
|
|
|
// ReplaceStep is a logical step indicating a resource will be replaced. This is comprised of three physical steps:
|
|
|
|
// a creation of the new resource, any number of intervening updates of dependents to the new resource, and then
|
|
|
|
// a deletion of the now-replaced old resource. This logical step is primarily here for tools and visualization.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
type ReplaceStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
2019-07-01 19:34:19 +00:00
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
new *resource.State // the new state snapshot.
|
|
|
|
keys []resource.PropertyKey // the keys causing replacement.
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
|
|
|
|
pendingDelete bool // true if a pending deletion should happen.
|
2017-06-11 00:03:58 +00:00
|
|
|
}
|
|
|
|
|
2017-11-17 02:21:41 +00:00
|
|
|
var _ Step = (*ReplaceStep)(nil)
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewReplaceStep(deployment *Deployment, old, new *resource.State, keys, diffs []resource.PropertyKey,
|
2023-03-03 16:36:39 +00:00
|
|
|
detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(old.URN != "", "old", "must have a URN")
|
|
|
|
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
|
|
|
|
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
2018-08-03 21:06:00 +00:00
|
|
|
// contract.Assert(new.ID == "")
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return &ReplaceStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2017-12-10 16:37:22 +00:00
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
keys: keys,
|
2019-03-07 00:41:19 +00:00
|
|
|
diffs: diffs,
|
2019-07-01 19:34:19 +00:00
|
|
|
detailedDiff: detailedDiff,
|
2017-12-10 16:37:22 +00:00
|
|
|
pendingDelete: pendingDelete,
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *ReplaceStep) Op() display.StepOp { return OpReplace }
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *ReplaceStep) Deployment() *Deployment { return s.deployment }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *ReplaceStep) Type() tokens.Type { return s.new.Type }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *ReplaceStep) Provider() string { return s.new.Provider }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *ReplaceStep) URN() resource.URN { return s.new.URN }
|
2019-07-01 19:34:19 +00:00
|
|
|
func (s *ReplaceStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ReplaceStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ReplaceStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ReplaceStep) Keys() []resource.PropertyKey { return s.keys }
|
|
|
|
func (s *ReplaceStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *ReplaceStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
func (s *ReplaceStep) Logical() bool { return true }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *ReplaceStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2017-12-10 16:37:22 +00:00
|
|
|
// If this is a pending delete, we should have marked the old resource for deletion in the CreateReplacement step.
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Assertf(!s.pendingDelete || s.old.Delete,
|
|
|
|
"old resource %v should be marked for deletion if pending delete", s.old.URN)
|
2018-08-08 20:45:48 +00:00
|
|
|
return resource.StatusOK, func() {}, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
}
|
2017-06-10 01:34:37 +00:00
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *ReplaceStep) Fail() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ReplaceStep) Skip() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
2018-08-03 21:06:00 +00:00
|
|
|
// ReadStep is a step indicating that an existing resources will be "read" and projected into the Pulumi object
|
|
|
|
// model. Resources that are read are marked with the "External" bit which indicates to the engine that it does
|
|
|
|
// not own this resource's lifeycle.
|
|
|
|
//
|
|
|
|
// A resource with a given URN can transition freely between an "external" state and a non-external state. If
|
2020-11-18 17:47:52 +00:00
|
|
|
// a URN that was previously marked "External" (i.e. was the target of a ReadStep in a previous deployment) is the
|
|
|
|
// target of a RegisterResource in the next deployment, a CreateReplacement step will be issued to indicate the
|
|
|
|
// transition from external to owned. If a URN that was previously not marked "External" is the target of a
|
|
|
|
// ReadResource in the next deployment, a ReadReplacement step will be issued to indicate the transition from owned to
|
|
|
|
// external.
|
2018-08-03 21:06:00 +00:00
|
|
|
type ReadStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the deployment that produced this read
|
|
|
|
event ReadResourceEvent // the event that should be signaled upon completion
|
|
|
|
old *resource.State // the old resource state, if one exists for this urn
|
|
|
|
new *resource.State // the new resource state, to be used to query the provider
|
|
|
|
replacing bool // whether or not the new resource is replacing the old resource
|
2023-12-22 21:14:04 +00:00
|
|
|
provider plugin.Provider // the optional provider to use.
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// NewReadStep creates a new Read step.
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewReadStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID != "", "new", "must have an ID")
|
|
|
|
contract.Requiref(new.External, "new", "must be marked as external")
|
|
|
|
contract.Requiref(new.Custom, "new", "must be a custom resource")
|
2018-08-03 21:06:00 +00:00
|
|
|
|
|
|
|
// If Old was given, it's either an external resource or its ID is equal to the
|
|
|
|
// ID that we are preparing to read.
|
|
|
|
if old != nil {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old.ID == new.ID || old.External,
|
|
|
|
"old", "must have the same ID as new or be external")
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return &ReadStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
event: event,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
replacing: false,
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewReadReplacementStep creates a new Read step with the `replacing` flag set. When executed,
|
|
|
|
// it will pend deletion of the "old" resource, which must not be an external resource.
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewReadReplacementStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID != "", "new", "must have an ID")
|
|
|
|
contract.Requiref(new.External, "new", "must be marked as external")
|
|
|
|
contract.Requiref(new.Custom, "new", "must be a custom resource")
|
|
|
|
|
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
|
|
|
contract.Requiref(!old.External, "old", "must not be marked as external")
|
|
|
|
|
2018-08-03 21:06:00 +00:00
|
|
|
return &ReadStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
event: event,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
replacing: true,
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *ReadStep) Op() display.StepOp {
|
2018-08-03 21:06:00 +00:00
|
|
|
if s.replacing {
|
|
|
|
return OpReadReplacement
|
|
|
|
}
|
|
|
|
|
|
|
|
return OpRead
|
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *ReadStep) Deployment() *Deployment { return s.deployment }
|
|
|
|
func (s *ReadStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *ReadStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *ReadStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *ReadStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ReadStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ReadStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ReadStep) Logical() bool { return !s.replacing }
|
2018-08-03 21:06:00 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *ReadStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2018-08-03 21:06:00 +00:00
|
|
|
urn := s.new.URN
|
|
|
|
id := s.new.ID
|
|
|
|
|
2018-08-07 07:40:43 +00:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
2018-08-03 21:06:00 +00:00
|
|
|
// Unlike most steps, Read steps run during previews. The only time
|
|
|
|
// we can't run is if the ID we are given is unknown.
|
2019-08-21 17:09:02 +00:00
|
|
|
if id == plugin.UnknownStringValue {
|
2024-05-09 16:15:41 +00:00
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2018-08-08 19:06:20 +00:00
|
|
|
s.new.Outputs = resource.PropertyMap{}
|
|
|
|
} else {
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err := getProvider(s, s.provider)
|
2018-08-03 21:06:00 +00:00
|
|
|
if err != nil {
|
2018-08-08 20:45:48 +00:00
|
|
|
return resource.StatusOK, nil, err
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
2023-12-02 14:39:38 +00:00
|
|
|
// Technically the only data we have at this point is "inputs", but we've been passing that as "state" to
|
|
|
|
// providers since forever and it would probably break things to stop sending that now. Thus this strange double
|
|
|
|
// send of inputs as both "inputs" and "state". Something to break to tidy up in V4.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
result, err := prov.Read(context.TODO(), plugin.ReadRequest{
|
|
|
|
URN: urn,
|
|
|
|
ID: id,
|
|
|
|
Inputs: s.new.Inputs,
|
|
|
|
State: s.new.Inputs,
|
|
|
|
})
|
2024-05-09 16:15:41 +00:00
|
|
|
|
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2018-08-03 21:06:00 +00:00
|
|
|
if err != nil {
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if result.Status != resource.StatusPartialFailure {
|
|
|
|
return result.Status, nil, err
|
2018-08-07 07:40:43 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
resourceError = err
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resourceStatus = result.Status
|
2018-08-07 07:40:43 +00:00
|
|
|
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
}
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
2019-08-21 17:09:02 +00:00
|
|
|
// If there is no such resource, return an error indicating as such.
|
|
|
|
if result.Outputs == nil {
|
2021-11-13 02:37:17 +00:00
|
|
|
return resource.StatusOK, nil, fmt.Errorf("resource '%s' does not exist", id)
|
2019-08-21 17:09:02 +00:00
|
|
|
}
|
2019-03-11 20:50:00 +00:00
|
|
|
s.new.Outputs = result.Outputs
|
2019-08-21 17:09:02 +00:00
|
|
|
|
|
|
|
if result.ID != "" {
|
|
|
|
s.new.ID = result.ID
|
|
|
|
}
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// If we were asked to replace an existing, non-External resource, pend the
|
|
|
|
// deletion here.
|
|
|
|
if s.replacing {
|
|
|
|
s.old.Delete = true
|
|
|
|
}
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
// Propagate timestamps on Read.
|
|
|
|
if s.old != nil {
|
|
|
|
s.new.Created = s.old.Created
|
|
|
|
s.new.Modified = s.old.Modified
|
|
|
|
}
|
2023-09-25 18:22:23 +00:00
|
|
|
var inputsChange, outputsChange bool
|
|
|
|
if s.old != nil {
|
|
|
|
inputsChange = !s.new.Inputs.DeepEquals(s.old.Inputs)
|
|
|
|
outputsChange = !s.new.Outputs.DeepEquals(s.old.Outputs)
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
}
|
2023-09-25 18:22:23 +00:00
|
|
|
// Only update the Modified timestamp if read provides new values that differ
|
|
|
|
// from the old state.
|
|
|
|
if inputsChange || outputsChange {
|
|
|
|
now := time.Now().UTC()
|
|
|
|
s.new.Modified = &now
|
|
|
|
}
|
|
|
|
|
|
|
|
complete := func() { s.event.Done(&ReadResult{State: s.new}) }
|
2018-08-07 07:40:43 +00:00
|
|
|
if resourceError == nil {
|
|
|
|
return resourceStatus, complete, nil
|
|
|
|
}
|
|
|
|
return resourceStatus, complete, resourceError
|
2018-08-03 21:06:00 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *ReadStep) Fail() {
|
|
|
|
s.event.Done(&ReadResult{State: s.new, Result: ResultStateFailed})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ReadStep) Skip() {
|
|
|
|
s.event.Done(&ReadResult{State: s.new, Result: ResultStateSkipped})
|
|
|
|
}
|
|
|
|
|
2018-08-23 00:52:46 +00:00
|
|
|
// RefreshStep is a step used to track the progress of a refresh operation. A refresh operation updates the an existing
|
|
|
|
// resource by reading its current state from its provider plugin. These steps are not issued by the step generator;
|
2020-11-18 17:47:52 +00:00
|
|
|
// instead, they are issued by the deployment executor as the optional first step in deployment execution.
|
2018-08-23 00:52:46 +00:00
|
|
|
type RefreshStep struct {
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
deployment *Deployment // the deployment that produced this refresh
|
|
|
|
old *resource.State // the old resource state, if one exists for this urn
|
|
|
|
new *resource.State // the new resource state, to be used to query the provider
|
|
|
|
done chan<- bool // the channel to use to signal completion, if any
|
|
|
|
provider plugin.Provider // the optional provider to use.
|
|
|
|
diff plugin.DiffResult // the diff between the cloud provider and the state file
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// NewRefreshStep creates a new Refresh step.
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewRefreshStep(deployment *Deployment, old *resource.State, done chan<- bool) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(old != nil, "old", "must not be nil")
|
2018-08-23 00:52:46 +00:00
|
|
|
|
|
|
|
// NOTE: we set the new state to the old state by default so that we don't interpret step failures as deletes.
|
|
|
|
return &RefreshStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
old: old,
|
|
|
|
new: old,
|
|
|
|
done: done,
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
func (s *RefreshStep) Op() display.StepOp { return OpRefresh }
|
|
|
|
func (s *RefreshStep) Deployment() *Deployment { return s.deployment }
|
|
|
|
func (s *RefreshStep) Type() tokens.Type { return s.old.Type }
|
|
|
|
func (s *RefreshStep) Provider() string { return s.old.Provider }
|
|
|
|
func (s *RefreshStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *RefreshStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *RefreshStep) New() *resource.State { return s.new }
|
|
|
|
func (s *RefreshStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *RefreshStep) Logical() bool { return false }
|
|
|
|
func (s *RefreshStep) Diffs() []resource.PropertyKey { return s.diff.ChangedKeys }
|
|
|
|
func (s *RefreshStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.diff.DetailedDiff }
|
2018-08-23 00:52:46 +00:00
|
|
|
|
|
|
|
// ResultOp returns the operation that corresponds to the change to this resource after reading its current state, if
|
|
|
|
// any.
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *RefreshStep) ResultOp() display.StepOp {
|
2018-08-23 00:52:46 +00:00
|
|
|
if s.new == nil {
|
|
|
|
return OpDelete
|
|
|
|
}
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
|
Don't call `Diff` when refreshing external resources (#16544)
External resources are resources whose state is tracked but not managed.
That is, they feature in the state file, but Pulumi does not issue
create, read or update operations for them. Typically they arise as the
result of generated `.get` functions and methods in Pulumi programs
(these should not be confused with specific `getX` functions/methods,
which are backed by `Invoke`s and designed to reflect data
sources/arbitrary function calls).
In #16146, `refresh` operations were modified to use actual bonafide
`Diff` calls to compute differences in a manner that would reflect "what
will happen on the next `preview`". In general, this aligns better with
what users expect when running `refresh`, and fixes a number of issues
that have long plagued `refresh`. However, it falls down in the case of
external resources. The existing `Diff` contract takes (somewhat
strangely) _three_ key inputs:
* The old inputs ("what the program used to say")
* The old outputs ("what we got back from the last update")
* The new inputs ("what the program says now")
Intuitively, `Diff`'s job is to return the difference between the old
and new inputs. Aside from old outputs being a bit of a third wheel,
this contract makes sense for an `update` operation, where the inputs
are driving the outputs and changes there will yield changes elsewhere.
In a `refresh` operation, however, we have a different situation -- one
in which _both_ old and new inputs _and_ outputs are available. Alas, we
don't currently have a "four-argument `Diff`" (or a two-argument one we
can call twice with no fear of repercussions), and so the usage in
`refresh` is a little bit of a clever hack that ends up biasing inputs
over outputs (see #16146 or the implementation for details).
For external resources, the old behaviour (where we just compared
outputs) seems like a better fit. This commit therefore adds this
special case and a test to ensure that when we see external resources,
we simply compare outputs and don't call `Diff`.
Fixes #16401
Fixes #16502
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
|
|
|
// There are two cases in which we'll diff only resource outputs on a
|
|
|
|
// refresh:
|
|
|
|
//
|
|
|
|
// * The resource is external, that is, it is not managed by Pulumi.
|
|
|
|
// In these cases, we care somewhat equally about inputs and outputs, but
|
|
|
|
// the Diff contract we currently support forces us to bias one side
|
|
|
|
// (typically inputs). Moreover, some providers might not support
|
|
|
|
// diff/handle diffing correctly for external resources. This can lead to
|
|
|
|
// surprising results, so for now we sidestep the issue by only looking at
|
|
|
|
// outputs.
|
|
|
|
// * The user has explicitly opted into this legacy behaviour by setting
|
|
|
|
// the `UseLegacyRefreshDiff` option to true.
|
|
|
|
if s.old.External || s.deployment.opts.UseLegacyRefreshDiff {
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
if s.new == s.old || s.old.Outputs.Diff(s.new.Outputs) == nil {
|
|
|
|
return OpSame
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
if s.new == s.old || s.diff.Changes == plugin.DiffNone {
|
|
|
|
return OpSame
|
|
|
|
}
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
|
2018-08-23 00:52:46 +00:00
|
|
|
return OpUpdate
|
|
|
|
}
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *RefreshStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
2018-08-23 00:52:46 +00:00
|
|
|
var complete func()
|
|
|
|
if s.done != nil {
|
|
|
|
complete = func() { close(s.done) }
|
|
|
|
}
|
|
|
|
|
2019-08-16 18:04:03 +00:00
|
|
|
resourceID := s.old.ID
|
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
// Component, provider, and pending-replace resources never change with a refresh; just return the current state.
|
|
|
|
if !s.old.Custom || providers.IsProviderType(s.old.Type) || s.old.PendingReplacement {
|
2018-08-23 00:52:46 +00:00
|
|
|
return resource.StatusOK, complete, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// For a custom resource, fetch the resource's provider and read the resource's current state.
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err := getProvider(s, s.provider)
|
2018-08-23 00:52:46 +00:00
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
var initErrors []string
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
refreshed, err := prov.Read(context.TODO(), plugin.ReadRequest{
|
|
|
|
URN: s.old.URN,
|
|
|
|
ID: resourceID,
|
|
|
|
Inputs: s.old.Inputs,
|
|
|
|
State: s.old.Outputs,
|
|
|
|
})
|
2018-08-23 00:52:46 +00:00
|
|
|
if err != nil {
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if refreshed.Status != resource.StatusPartialFailure {
|
|
|
|
return refreshed.Status, nil, err
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
2018-09-05 21:00:28 +00:00
|
|
|
initErrors = initErr.Reasons
|
2019-04-10 20:47:54 +00:00
|
|
|
|
|
|
|
// Partial failure SHOULD NOT cause refresh to fail. Instead:
|
|
|
|
//
|
|
|
|
// 1. Warn instead that during refresh we noticed the resource has become unhealthy.
|
|
|
|
// 2. Make sure the initialization errors are persisted in the state, so that the next
|
|
|
|
// `pulumi up` will surface them to the user.
|
|
|
|
err = nil
|
2023-12-12 12:19:42 +00:00
|
|
|
msg := "Refreshed resource is in an unhealthy state:\n* " + strings.Join(initErrors, "\n* ")
|
2020-11-18 17:47:52 +00:00
|
|
|
s.Deployment().Diag().Warningf(diag.RawMessage(s.URN(), msg))
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
|
|
|
}
|
2019-03-11 20:50:00 +00:00
|
|
|
outputs := refreshed.Outputs
|
2018-08-23 00:52:46 +00:00
|
|
|
|
2019-03-11 20:50:00 +00:00
|
|
|
// If the provider specified new inputs for this resource, pick them up now. Otherwise, retain the current inputs.
|
|
|
|
inputs := s.old.Inputs
|
|
|
|
if refreshed.Inputs != nil {
|
|
|
|
inputs = refreshed.Inputs
|
|
|
|
}
|
|
|
|
|
|
|
|
if outputs != nil {
|
2019-08-16 18:04:03 +00:00
|
|
|
// There is a chance that the ID has changed. We want to allow this change to happen
|
|
|
|
// it will have changed already in the outputs, but we need to persist this change
|
|
|
|
// at a state level because the Id
|
|
|
|
if refreshed.ID != "" && refreshed.ID != resourceID {
|
|
|
|
logging.V(7).Infof("Refreshing ID; oldId=%s, newId=%s", resourceID, refreshed.ID)
|
|
|
|
resourceID = refreshed.ID
|
|
|
|
}
|
|
|
|
|
|
|
|
s.new = resource.NewState(s.old.Type, s.old.URN, s.old.Custom, s.old.Delete, resourceID, inputs, outputs,
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
s.old.Parent, s.old.Protect, s.old.External, s.old.Dependencies, initErrors, s.old.Provider,
|
2022-03-24 19:08:18 +00:00
|
|
|
s.old.PropertyDependencies, s.old.PendingReplacement, s.old.AdditionalSecretOutputs, s.old.Aliases,
|
[engine] Add support for source positions
These changes add support for passing source position information in
gRPC metadata and recording the source position that corresponds to a
resource registration in the statefile.
Enabling source position information in the resource model can provide
substantial benefits, including but not limited to:
- Better errors from the Pulumi CLI
- Go-to-defintion for resources in state
- Editor integration for errors, etc. from `pulumi preview`
Source positions are (file, line) or (file, line, column) tuples
represented as URIs. The line and column are stored in the fragment
portion of the URI as "line(,column)?". The scheme of the URI and the
form of its path component depends on the context in which it is
generated or used:
- During an active update, the URI's scheme is `file` and paths are
absolute filesystem paths. This allows consumers to easily access
arbitrary files that are available on the host.
- In a statefile, the URI's scheme is `project` and paths are relative
to the project root. This allows consumers to resolve source positions
relative to the project file in different contexts irrespective of the
location of the project itself (e.g. given a project-relative path and
the URL of the project's root on GitHub, one can build a GitHub URL for
the source position).
During an update, source position information may be attached to gRPC
calls as "source-position" metadata. This allows arbitrary calls to be
associated with source positions without changes to their protobuf
payloads. Modifying the protobuf payloads is also a viable approach, but
is somewhat more invasive than attaching metadata, and requires changes
to every call signature.
Source positions should reflect the position in user code that initiated
a resource model operation (e.g. the source position passed with
`RegisterResource` for `pet` in the example above should be the source
position in `index.ts`, _not_ the source position in the Pulumi SDK). In
general, the Pulumi SDK should be able to infer the source position of
the resource registration, as the relationship between a resource
registration and its corresponding user code should be static per SDK.
Source positions in state files will be stored as a new `registeredAt`
property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
|
|
|
&s.old.CustomTimeouts, s.old.ImportID, s.old.RetainOnDelete, s.old.DeletedWith, s.old.Created, s.old.Modified,
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
s.old.SourcePosition, s.old.IgnoreChanges,
|
[engine] Add support for source positions
These changes add support for passing source position information in
gRPC metadata and recording the source position that corresponds to a
resource registration in the statefile.
Enabling source position information in the resource model can provide
substantial benefits, including but not limited to:
- Better errors from the Pulumi CLI
- Go-to-defintion for resources in state
- Editor integration for errors, etc. from `pulumi preview`
Source positions are (file, line) or (file, line, column) tuples
represented as URIs. The line and column are stored in the fragment
portion of the URI as "line(,column)?". The scheme of the URI and the
form of its path component depends on the context in which it is
generated or used:
- During an active update, the URI's scheme is `file` and paths are
absolute filesystem paths. This allows consumers to easily access
arbitrary files that are available on the host.
- In a statefile, the URI's scheme is `project` and paths are relative
to the project root. This allows consumers to resolve source positions
relative to the project file in different contexts irrespective of the
location of the project itself (e.g. given a project-relative path and
the URL of the project's root on GitHub, one can build a GitHub URL for
the source position).
During an update, source position information may be attached to gRPC
calls as "source-position" metadata. This allows arbitrary calls to be
associated with source positions without changes to their protobuf
payloads. Modifying the protobuf payloads is also a viable approach, but
is somewhat more invasive than attaching metadata, and requires changes
to every call signature.
Source positions should reflect the position in user code that initiated
a resource model operation (e.g. the source position passed with
`RegisterResource` for `pet` in the example above should be the source
position in `index.ts`, _not_ the source position in the Pulumi SDK). In
general, the Pulumi SDK should be able to infer the source position of
the resource registration, as the relationship between a resource
registration and its corresponding user code should be static per SDK.
Source positions in state files will be stored as a new `registeredAt`
property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
|
|
|
)
|
2023-09-25 18:22:23 +00:00
|
|
|
var inputsChange, outputsChange bool
|
|
|
|
if s.old != nil {
|
Don't call `Diff` when refreshing external resources (#16544)
External resources are resources whose state is tracked but not managed.
That is, they feature in the state file, but Pulumi does not issue
create, read or update operations for them. Typically they arise as the
result of generated `.get` functions and methods in Pulumi programs
(these should not be confused with specific `getX` functions/methods,
which are backed by `Invoke`s and designed to reflect data
sources/arbitrary function calls).
In #16146, `refresh` operations were modified to use actual bonafide
`Diff` calls to compute differences in a manner that would reflect "what
will happen on the next `preview`". In general, this aligns better with
what users expect when running `refresh`, and fixes a number of issues
that have long plagued `refresh`. However, it falls down in the case of
external resources. The existing `Diff` contract takes (somewhat
strangely) _three_ key inputs:
* The old inputs ("what the program used to say")
* The old outputs ("what we got back from the last update")
* The new inputs ("what the program says now")
Intuitively, `Diff`'s job is to return the difference between the old
and new inputs. Aside from old outputs being a bit of a third wheel,
this contract makes sense for an `update` operation, where the inputs
are driving the outputs and changes there will yield changes elsewhere.
In a `refresh` operation, however, we have a different situation -- one
in which _both_ old and new inputs _and_ outputs are available. Alas, we
don't currently have a "four-argument `Diff`" (or a two-argument one we
can call twice with no fear of repercussions), and so the usage in
`refresh` is a little bit of a clever hack that ends up biasing inputs
over outputs (see #16146 or the implementation for details).
For external resources, the old behaviour (where we just compared
outputs) seems like a better fit. This commit therefore adds this
special case and a test to ensure that when we see external resources,
we simply compare outputs and don't call `Diff`.
Fixes #16401
Fixes #16502
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
|
|
|
// There are two cases in which we'll diff only resource outputs on a
|
|
|
|
// refresh:
|
|
|
|
//
|
|
|
|
// * The resource is external, that is, it is not managed by Pulumi.
|
|
|
|
// In these cases, we care somewhat equally about inputs and outputs, but
|
|
|
|
// the Diff contract we currently support forces us to bias one side
|
|
|
|
// (typically inputs). Moreover, some providers might not support
|
|
|
|
// diff/handle diffing correctly for external resources. This can lead to
|
|
|
|
// surprising results, so for now we sidestep the issue by only looking at
|
|
|
|
// outputs.
|
|
|
|
// * The user has explicitly opted into this legacy behaviour by setting
|
|
|
|
// the `UseLegacyRefreshDiff` option to true.
|
|
|
|
if s.old.External || s.deployment.opts.UseLegacyRefreshDiff {
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
inputsChange = !refreshed.Inputs.DeepEquals(s.old.Inputs)
|
|
|
|
outputsChange = !refreshed.Outputs.DeepEquals(s.old.Outputs)
|
|
|
|
} else {
|
|
|
|
inputsChange = !inputs.DeepEquals(s.old.Inputs)
|
|
|
|
outputsChange = !outputs.DeepEquals(s.old.Outputs)
|
|
|
|
}
|
2023-09-25 18:22:23 +00:00
|
|
|
}
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
|
2023-09-25 18:22:23 +00:00
|
|
|
// Only update the Modified timestamp if refresh provides new values that differ
|
|
|
|
// from the old state.
|
|
|
|
if inputsChange || outputsChange {
|
|
|
|
// The refresh has identified an incongruence between the provider and state
|
|
|
|
// updated the Modified timestamp to track this.
|
|
|
|
now := time.Now().UTC()
|
|
|
|
s.new.Modified = &now
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
}
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
|
Don't call `Diff` when refreshing external resources (#16544)
External resources are resources whose state is tracked but not managed.
That is, they feature in the state file, but Pulumi does not issue
create, read or update operations for them. Typically they arise as the
result of generated `.get` functions and methods in Pulumi programs
(these should not be confused with specific `getX` functions/methods,
which are backed by `Invoke`s and designed to reflect data
sources/arbitrary function calls).
In #16146, `refresh` operations were modified to use actual bonafide
`Diff` calls to compute differences in a manner that would reflect "what
will happen on the next `preview`". In general, this aligns better with
what users expect when running `refresh`, and fixes a number of issues
that have long plagued `refresh`. However, it falls down in the case of
external resources. The existing `Diff` contract takes (somewhat
strangely) _three_ key inputs:
* The old inputs ("what the program used to say")
* The old outputs ("what we got back from the last update")
* The new inputs ("what the program says now")
Intuitively, `Diff`'s job is to return the difference between the old
and new inputs. Aside from old outputs being a bit of a third wheel,
this contract makes sense for an `update` operation, where the inputs
are driving the outputs and changes there will yield changes elsewhere.
In a `refresh` operation, however, we have a different situation -- one
in which _both_ old and new inputs _and_ outputs are available. Alas, we
don't currently have a "four-argument `Diff`" (or a two-argument one we
can call twice with no fear of repercussions), and so the usage in
`refresh` is a little bit of a clever hack that ends up biasing inputs
over outputs (see #16146 or the implementation for details).
For external resources, the old behaviour (where we just compared
outputs) seems like a better fit. This commit therefore adds this
special case and a test to ensure that when we see external resources,
we simply compare outputs and don't call `Diff`.
Fixes #16401
Fixes #16502
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
|
|
|
if s.old.External {
|
|
|
|
logging.V(7).Infof("External resource %s; diffing outputs only", s.URN())
|
|
|
|
} else if s.deployment.opts.UseLegacyRefreshDiff {
|
|
|
|
logging.V(7).Infof("Refresh diffing disabled; diffing outputs only (%s)", s.URN())
|
|
|
|
} else {
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
// To compute refresh diffs against the desired state, we compute the diff
|
|
|
|
// that a user would see if they immediately ran an `up` operation on a
|
|
|
|
// no-change program after this refresh. However, this will return the
|
|
|
|
// _opposite_ of what we want, since the `up`'s diff is framed in terms of
|
|
|
|
// the program being the source of truth (not the provider). That is, we
|
|
|
|
// want to show the user what changes are coming from the outputs into the
|
|
|
|
// inputs, not what changes are coming from the inputs into the outputs!
|
|
|
|
// We thus invert the diff (changing adds to deletes, and so on) before
|
|
|
|
// storing it against the step.
|
|
|
|
//
|
|
|
|
// Note that to compute the diff in this manner, we pass:
|
|
|
|
//
|
|
|
|
// * newInputs where oldInputs are expected
|
|
|
|
// * newOutputs where oldOutputs are expected
|
|
|
|
// * oldInputs where newInputs are expected
|
|
|
|
diff, err := diffResource(
|
|
|
|
s.new.URN, s.new.ID,
|
|
|
|
// pass new inputs/outputs as old inputs/outputs
|
|
|
|
s.new.Inputs, s.new.Outputs,
|
|
|
|
// pass old inputs as new inputs
|
|
|
|
s.old.Inputs,
|
2024-06-14 10:19:13 +00:00
|
|
|
prov, s.deployment.opts.DryRun, s.old.IgnoreChanges,
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
)
|
|
|
|
if err != nil {
|
|
|
|
return refreshed.Status, nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
s.diff = diff.Invert()
|
Don't call `Diff` when refreshing external resources (#16544)
External resources are resources whose state is tracked but not managed.
That is, they feature in the state file, but Pulumi does not issue
create, read or update operations for them. Typically they arise as the
result of generated `.get` functions and methods in Pulumi programs
(these should not be confused with specific `getX` functions/methods,
which are backed by `Invoke`s and designed to reflect data
sources/arbitrary function calls).
In #16146, `refresh` operations were modified to use actual bonafide
`Diff` calls to compute differences in a manner that would reflect "what
will happen on the next `preview`". In general, this aligns better with
what users expect when running `refresh`, and fixes a number of issues
that have long plagued `refresh`. However, it falls down in the case of
external resources. The existing `Diff` contract takes (somewhat
strangely) _three_ key inputs:
* The old inputs ("what the program used to say")
* The old outputs ("what we got back from the last update")
* The new inputs ("what the program says now")
Intuitively, `Diff`'s job is to return the difference between the old
and new inputs. Aside from old outputs being a bit of a third wheel,
this contract makes sense for an `update` operation, where the inputs
are driving the outputs and changes there will yield changes elsewhere.
In a `refresh` operation, however, we have a different situation -- one
in which _both_ old and new inputs _and_ outputs are available. Alas, we
don't currently have a "four-argument `Diff`" (or a two-argument one we
can call twice with no fear of repercussions), and so the usage in
`refresh` is a little bit of a clever hack that ends up biasing inputs
over outputs (see #16146 or the implementation for details).
For external resources, the old behaviour (where we just compared
outputs) seems like a better fit. This commit therefore adds this
special case and a test to ensure that when we see external resources,
we simply compare outputs and don't call `Diff`.
Fixes #16401
Fixes #16502
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
|
|
|
logging.V(7).Infof("Refresh diff for %s: %v", s.URN(), s.diff)
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
}
|
2018-08-23 00:52:46 +00:00
|
|
|
} else {
|
|
|
|
s.new = nil
|
|
|
|
}
|
|
|
|
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
return refreshed.Status, nil, err
|
2018-08-23 00:52:46 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *RefreshStep) Fail() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *RefreshStep) Skip() {
|
|
|
|
// Nothing to do here.
|
|
|
|
}
|
|
|
|
|
2019-07-12 18:12:01 +00:00
|
|
|
type ImportStep struct {
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment *Deployment // the current deployment.
|
2019-07-31 16:39:07 +00:00
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
original *resource.State // the original resource, if this is an import-replace.
|
|
|
|
old *resource.State // the state of the resource fetched from the provider.
|
|
|
|
new *resource.State // the newly computed state of the resource after importing.
|
|
|
|
replacing bool // true if we are replacing a Pulumi-managed resource.
|
2020-11-18 17:47:52 +00:00
|
|
|
planned bool // true if this import is from an import deployment.
|
2019-07-31 16:39:07 +00:00
|
|
|
diffs []resource.PropertyKey // any keys that differed between the user's program and the actual state.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
|
|
|
|
ignoreChanges []string // a list of property paths to ignore when updating.
|
2022-08-09 16:40:59 +00:00
|
|
|
randomSeed []byte // the random seed to use for Check.
|
2023-12-22 21:14:04 +00:00
|
|
|
provider plugin.Provider // the optional provider to use.
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewImportStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State,
|
2023-03-03 16:36:39 +00:00
|
|
|
ignoreChanges []string, randomSeed []byte,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID != "", "new", "must have an ID")
|
|
|
|
contract.Requiref(new.Custom, "new", "must be a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be external")
|
|
|
|
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
|
2019-07-12 18:12:01 +00:00
|
|
|
|
|
|
|
return &ImportStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2019-07-31 16:39:07 +00:00
|
|
|
reg: reg,
|
|
|
|
new: new,
|
|
|
|
ignoreChanges: ignoreChanges,
|
2022-08-09 16:40:59 +00:00
|
|
|
randomSeed: randomSeed,
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func NewImportReplacementStep(deployment *Deployment, reg RegisterResourceEvent, original, new *resource.State,
|
2023-03-03 16:36:39 +00:00
|
|
|
ignoreChanges []string, randomSeed []byte,
|
|
|
|
) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(original != nil, "original", "must not be nil")
|
|
|
|
|
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
|
|
|
contract.Requiref(new.ID != "", "new", "must have an ID")
|
|
|
|
contract.Requiref(new.Custom, "new", "must be a custom resource")
|
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be external")
|
|
|
|
|
|
|
|
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
|
2019-07-12 18:12:01 +00:00
|
|
|
|
|
|
|
return &ImportStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
2019-07-31 16:39:07 +00:00
|
|
|
reg: reg,
|
|
|
|
original: original,
|
|
|
|
new: new,
|
|
|
|
replacing: true,
|
|
|
|
ignoreChanges: ignoreChanges,
|
2022-08-09 16:40:59 +00:00
|
|
|
randomSeed: randomSeed,
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-08-09 16:40:59 +00:00
|
|
|
func newImportDeploymentStep(deployment *Deployment, new *resource.State, randomSeed []byte) Step {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(new != nil, "new", "must not be nil")
|
|
|
|
contract.Requiref(new.URN != "", "new", "must have a URN")
|
2023-11-13 17:58:35 +00:00
|
|
|
contract.Requiref(!new.Custom || new.ID != "", "new", "must have an ID")
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
|
|
|
|
contract.Requiref(!new.External, "new", "must not be external")
|
2023-11-13 17:58:35 +00:00
|
|
|
contract.Requiref(!new.Custom || randomSeed != nil, "randomSeed", "must not be nil")
|
2020-10-14 11:51:53 +00:00
|
|
|
|
|
|
|
return &ImportStep{
|
2020-11-18 17:47:52 +00:00
|
|
|
deployment: deployment,
|
|
|
|
reg: noopEvent(0),
|
|
|
|
new: new,
|
|
|
|
planned: true,
|
2022-08-09 16:40:59 +00:00
|
|
|
randomSeed: randomSeed,
|
2020-10-14 11:51:53 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func (s *ImportStep) Op() display.StepOp {
|
2019-07-12 18:12:01 +00:00
|
|
|
if s.replacing {
|
|
|
|
return OpImportReplacement
|
|
|
|
}
|
|
|
|
return OpImport
|
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
func (s *ImportStep) Deployment() *Deployment { return s.deployment }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *ImportStep) Type() tokens.Type { return s.new.Type }
|
2019-07-12 18:12:01 +00:00
|
|
|
func (s *ImportStep) Provider() string { return s.new.Provider }
|
2020-07-09 14:19:12 +00:00
|
|
|
func (s *ImportStep) URN() resource.URN { return s.new.URN }
|
2019-07-12 18:12:01 +00:00
|
|
|
func (s *ImportStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ImportStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ImportStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ImportStep) Logical() bool { return !s.replacing }
|
|
|
|
func (s *ImportStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *ImportStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
func (s *ImportStep) Apply() (resource.Status, StepCompleteFunc, error) {
|
This commit adds the `Created` and `Modified` timestamps to pulumi state that are optional.
`Created`: Created tracks when the remote resource was first added to state by pulumi. Checkpoints prior to early 2023 do not include this. (Create, Import)
`Modified`: Modified tracks when the resource state was last altered. Checkpoints prior to early 2023 do not include this. (Create, Import, Read, Refresh, Update)
When serialized they will follow RFC3339 with nanoseconds captured by a test case.
https://pkg.go.dev/time#RFC3339
Note: Older versions of pulumi may strip these fields when modifying the state.
For future expansion, when we inevitably need to track other timestamps, we'll add a new "operationTimestamps" field (or something similarly named that clarified these are timestamps of the actual Pulumi operations).
operationTimestamps: {
created: ...,
updated: ...,
imported: ...,
}
Fixes https://github.com/pulumi/pulumi/issues/12022
2023-02-06 20:39:11 +00:00
|
|
|
complete := func() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new})
|
|
|
|
}
|
2019-07-12 18:12:01 +00:00
|
|
|
|
2020-10-14 11:51:53 +00:00
|
|
|
// If this is a planned import, ensure that the resource does not exist in the old state file.
|
|
|
|
if s.planned {
|
2020-11-18 17:47:52 +00:00
|
|
|
if _, ok := s.deployment.olds[s.new.URN]; ok {
|
2021-11-13 02:37:17 +00:00
|
|
|
return resource.StatusOK, nil, fmt.Errorf("resource '%v' already exists", s.new.URN)
|
2020-10-14 11:51:53 +00:00
|
|
|
}
|
2023-12-04 10:36:51 +00:00
|
|
|
if s.new.Parent.QualifiedType() != resource.RootStackType {
|
2024-04-09 10:56:25 +00:00
|
|
|
_, ok := s.deployment.news.Load(s.new.Parent)
|
2023-11-10 13:31:11 +00:00
|
|
|
if !ok {
|
2021-11-13 02:37:17 +00:00
|
|
|
return resource.StatusOK, nil, fmt.Errorf("unknown parent '%v' for resource '%v'",
|
2020-10-14 11:51:53 +00:00
|
|
|
s.new.Parent, s.new.URN)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2023-11-13 17:58:35 +00:00
|
|
|
// Only need to do anything here for custom resources, components just import as empty
|
|
|
|
inputs := resource.PropertyMap{}
|
|
|
|
outputs := resource.PropertyMap{}
|
|
|
|
var prov plugin.Provider
|
|
|
|
rst := resource.StatusOK
|
|
|
|
if s.new.Custom {
|
|
|
|
// Read the current state of the resource to import. If the provider does not hand us back any inputs for the
|
|
|
|
// resource, it probably needs to be updated. If the resource does not exist at all, fail the import.
|
|
|
|
var err error
|
2023-12-22 21:14:04 +00:00
|
|
|
prov, err = getProvider(s, s.provider)
|
2023-11-13 17:58:35 +00:00
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
read, err := prov.Read(context.TODO(), plugin.ReadRequest{
|
|
|
|
URN: s.new.URN,
|
|
|
|
ID: s.new.ID,
|
|
|
|
})
|
|
|
|
rst = read.Status
|
2024-05-09 16:15:41 +00:00
|
|
|
|
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
|
|
|
|
2023-11-13 17:58:35 +00:00
|
|
|
if err != nil {
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
} else {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if read.Outputs == nil {
|
|
|
|
return rst, nil, fmt.Errorf("resource '%v' does not exist", s.new.ID)
|
|
|
|
}
|
|
|
|
if read.Inputs == nil {
|
|
|
|
return resource.StatusOK, nil,
|
|
|
|
fmt.Errorf("provider does not support importing resources; please try updating the '%v' plugin",
|
|
|
|
s.new.URN.Type().Package())
|
|
|
|
}
|
|
|
|
if read.ID != "" {
|
|
|
|
s.new.ID = read.ID
|
|
|
|
}
|
|
|
|
inputs = read.Inputs
|
|
|
|
outputs = read.Outputs
|
2024-05-09 16:15:41 +00:00
|
|
|
} else {
|
|
|
|
s.new.Lock.Lock()
|
|
|
|
defer s.new.Lock.Unlock()
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
|
2023-11-13 17:58:35 +00:00
|
|
|
s.new.Outputs = outputs
|
2019-07-12 18:12:01 +00:00
|
|
|
// Magic up an old state so the frontend can display a proper diff. This state is the output of the just-executed
|
|
|
|
// `Read` combined with the resource identity and metadata from the desired state. This ensures that the only
|
|
|
|
// differences between the old and new states are between the inputs and outputs.
|
2023-11-13 17:58:35 +00:00
|
|
|
s.old = resource.NewState(s.new.Type, s.new.URN, s.new.Custom, false, s.new.ID, inputs, outputs,
|
2019-07-12 18:12:01 +00:00
|
|
|
s.new.Parent, s.new.Protect, false, s.new.Dependencies, s.new.InitErrors, s.new.Provider,
|
2022-10-20 06:15:43 +00:00
|
|
|
s.new.PropertyDependencies, false, nil, nil, &s.new.CustomTimeouts, s.new.ImportID, s.new.RetainOnDelete,
|
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146)
Presently, the behaviour of diffing during refresh steps is incomplete,
returning only an "output diff" that presents the changes in outputs.
This commit changes refresh steps so that:
* they compute a diff similar to the one that would be computed if a
`preview` were run immediately after the refresh, which is more
typically what users expect and want; and
* `IgnoreChanges` resource options are respected when performing the new
desired-state diffs, so that property additions or changes reported by a
refresh can be ignored.
In particular, `IgnoreChanges` can now be used to acknowledge that part
or all of a resource may change in the provider, but the user is OK with
this and doesn't want to be notified about it during a refresh.
Importantly, this means that the diff won't be reported, but also that
the changes won't be applied to state.
The implementation covers the following:
* A diff is computed using the inputs from the program and then
inverting the result, since in the case of a refresh the diff is being
driven by the provider side and not the program. This doesn't change
what is stored back into the state, but it does produce a diff that is
more aligned with the "true changes to the desired state".
* `IgnoreChanges` resource options are now stored in state, so that this
information can be used in refresh operations that do not have access
to/run the program.
* In the context of a refresh operation, `IgnoreChanges` applies to
*both* input and output properties. This differs from the behaviour of a
normal update operation, where `IgnoreChanges` only considers input
properties.
* The special `"*"` value for `IgnoreChanges` can be used to ignore all
properties. It _also_ ignores the case where the resource cannot be
found in the provider, and instead keeps the resource intact in state
with its existing input and output properties.
Because the program is not run for refresh operations, `IgnoreChanges`
options must be applied separately before a refresh takes place. This
can be accomplished using e.g. a `pulumi up` that applies the options
prior to a refresh. We should investigate perhaps providing a `pulumi
state set ...`-like CLI to make these sorts of changes directly to a
state.
For use cases relying on the legacy refresh diff provider, the
`PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which
will disable desired-state diff computation. We only need to perform
checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will
work correctly based on the presence or absence of a `DetailedDiff` in
the step.
### Notes
- https://github.com/pulumi/pulumi/issues/16144 affects some of these
cases - though its technically orthogonal
- https://github.com/pulumi/pulumi/issues/11279 is another technically
orthogonal issue that many providers (at least TFBridge ones) - do not
report back changes to input properties on Read when the input property
(or property path) was missing on the inputs. This is again technically
orthogonal - but leads to cases that appear "wrong" in terms of what is
stored back into the state still - though the same as before this
change.
- Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so
the ability to ignore changes on refresh doesn't currently work for
Azure Native.
### Fixes
* Fixes #16072
* Fixes #16278
* Fixes #16334
* Not quite #12346, but likely replaces the need for that
Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
|
|
|
s.new.DeletedWith, nil, nil, s.new.SourcePosition, s.new.IgnoreChanges)
|
2019-07-12 18:12:01 +00:00
|
|
|
|
2023-09-25 18:22:23 +00:00
|
|
|
// Import takes a resource that Pulumi did not create and imports it into pulumi state.
|
|
|
|
now := time.Now().UTC()
|
|
|
|
s.new.Modified = &now
|
|
|
|
// Set Created to now as the resource has been created in the state.
|
|
|
|
s.new.Created = &now
|
|
|
|
|
2023-11-13 17:58:35 +00:00
|
|
|
// If this is a component we don't need to do the rest of the input validation
|
|
|
|
if !s.new.Custom {
|
|
|
|
return rst, complete, nil
|
|
|
|
}
|
|
|
|
|
2020-11-18 17:47:52 +00:00
|
|
|
// If this step came from an import deployment, we need to fetch any required inputs from the state.
|
2020-10-14 11:51:53 +00:00
|
|
|
if s.planned {
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.Assertf(len(s.new.Inputs) == 0, "import resource cannot have existing inputs")
|
2020-10-14 11:51:53 +00:00
|
|
|
|
2022-02-02 13:04:59 +00:00
|
|
|
// Get the import object and see if it had properties set
|
|
|
|
var inputProperties []string
|
|
|
|
for _, imp := range s.deployment.imports {
|
|
|
|
if imp.ID == s.old.ID {
|
|
|
|
inputProperties = imp.Properties
|
|
|
|
break
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if len(inputProperties) == 0 {
|
2022-03-08 12:27:15 +00:00
|
|
|
logging.V(9).Infof("Importing %v with all properties", s.URN())
|
|
|
|
s.new.Inputs = s.old.Inputs.Copy()
|
2022-02-02 13:04:59 +00:00
|
|
|
} else {
|
|
|
|
logging.V(9).Infof("Importing %v with supplied properties: %v", s.URN(), inputProperties)
|
|
|
|
for _, p := range inputProperties {
|
|
|
|
k := resource.PropertyKey(p)
|
|
|
|
if value, has := s.old.Inputs[k]; has {
|
|
|
|
s.new.Inputs[k] = value
|
|
|
|
}
|
2020-10-14 11:51:53 +00:00
|
|
|
}
|
|
|
|
}
|
2022-03-08 12:27:15 +00:00
|
|
|
|
|
|
|
// Check the provider inputs for consistency. If the inputs fail validation, the import will still succeed, but
|
|
|
|
// we will display the validation failures and a message informing the user that the failures are almost
|
|
|
|
// definitely a provider bug.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resp, err := prov.Check(context.TODO(), plugin.CheckRequest{
|
|
|
|
URN: s.new.URN,
|
|
|
|
Olds: s.old.Inputs,
|
|
|
|
News: s.new.Inputs,
|
2024-06-14 10:19:13 +00:00
|
|
|
AllowUnknowns: s.deployment.opts.DryRun,
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
RandomSeed: s.randomSeed,
|
|
|
|
})
|
2022-03-08 12:27:15 +00:00
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
2020-12-17 22:46:50 +00:00
|
|
|
}
|
2022-03-08 12:27:15 +00:00
|
|
|
|
|
|
|
// Print this warning before printing all the check failures to give better context.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if len(resp.Failures) != 0 {
|
2022-03-08 12:27:15 +00:00
|
|
|
|
|
|
|
// Based on if the user passed 'properties' or not we want to change the error message here.
|
|
|
|
var errorMessage string
|
|
|
|
if len(inputProperties) == 0 {
|
|
|
|
ref, err := providers.ParseReference(s.Provider())
|
2023-02-17 20:05:48 +00:00
|
|
|
contract.AssertNoErrorf(err, "failed to parse provider reference %q", s.Provider())
|
2022-03-08 12:27:15 +00:00
|
|
|
|
|
|
|
pkgName := ref.URN().Type().Name()
|
|
|
|
errorMessage = fmt.Sprintf("This is almost certainly a bug in the `%s` provider.", pkgName)
|
|
|
|
} else {
|
|
|
|
errorMessage = "Try specifying a different set of properties to import with in the future."
|
|
|
|
}
|
|
|
|
|
|
|
|
s.deployment.Diag().Warningf(diag.Message(s.new.URN,
|
|
|
|
"One or more imported inputs failed to validate. %s "+
|
|
|
|
"The import will still proceed, but you will need to edit the generated code after copying it into your program."),
|
|
|
|
errorMessage)
|
|
|
|
}
|
|
|
|
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
issueCheckFailures(s.deployment.Diag().Warningf, s.new, s.new.URN, resp.Failures)
|
2022-03-08 12:27:15 +00:00
|
|
|
|
|
|
|
s.diffs, s.detailedDiff = []resource.PropertyKey{}, map[string]plugin.PropertyDiff{}
|
2023-09-25 18:22:23 +00:00
|
|
|
|
|
|
|
return rst, complete, nil
|
2020-10-14 11:51:53 +00:00
|
|
|
}
|
|
|
|
|
2022-03-08 12:27:15 +00:00
|
|
|
// Set inputs back to their old values (if any) for any "ignored" properties
|
2023-09-25 12:25:23 +00:00
|
|
|
processedInputs, err := processIgnoreChanges(s.new.Inputs, s.old.Inputs, s.ignoreChanges)
|
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
2022-03-08 12:27:15 +00:00
|
|
|
}
|
|
|
|
s.new.Inputs = processedInputs
|
|
|
|
|
2020-10-14 11:51:53 +00:00
|
|
|
// Check the inputs using the provider inputs for defaults.
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
resp, err := prov.Check(context.TODO(), plugin.CheckRequest{
|
|
|
|
URN: s.new.URN,
|
|
|
|
Olds: s.old.Inputs,
|
|
|
|
News: s.new.Inputs,
|
2024-06-14 10:19:13 +00:00
|
|
|
AllowUnknowns: s.deployment.opts.DryRun,
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
RandomSeed: s.randomSeed,
|
|
|
|
})
|
2019-07-12 18:12:01 +00:00
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
if issueCheckErrors(s.deployment, s.new, s.new.URN, resp.Failures) {
|
2019-07-12 18:12:01 +00:00
|
|
|
return rst, nil, errors.New("one or more inputs failed to validate")
|
|
|
|
}
|
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302)
Normalize methods on plugin.Provider to the form:
```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```
This provides a more consistent and forwards compatible interface for
each of our methods.
---
I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
|
|
|
s.new.Inputs = resp.Properties
|
2019-07-12 18:12:01 +00:00
|
|
|
|
2020-10-14 11:51:53 +00:00
|
|
|
// Diff the user inputs against the provider inputs. If there are any differences, fail the import unless this step
|
2020-11-18 17:47:52 +00:00
|
|
|
// is from an import deployment.
|
2024-06-14 10:19:13 +00:00
|
|
|
diff, err := diffResource(
|
|
|
|
s.new.URN, s.new.ID,
|
|
|
|
s.old.Inputs, s.old.Outputs,
|
|
|
|
s.new.Inputs,
|
|
|
|
prov,
|
|
|
|
s.deployment.opts.DryRun,
|
|
|
|
s.ignoreChanges,
|
|
|
|
)
|
2019-07-12 18:12:01 +00:00
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
|
2022-03-08 12:27:15 +00:00
|
|
|
s.diffs, s.detailedDiff = diff.ChangedKeys, diff.DetailedDiff
|
2020-10-14 11:51:53 +00:00
|
|
|
|
2022-03-08 12:27:15 +00:00
|
|
|
if diff.Changes != plugin.DiffNone {
|
|
|
|
const message = "inputs to import do not match the existing resource"
|
2019-07-12 18:12:01 +00:00
|
|
|
|
2024-06-14 10:19:13 +00:00
|
|
|
if s.deployment.opts.DryRun {
|
2022-03-08 12:27:15 +00:00
|
|
|
s.deployment.ctx.Diag.Warningf(diag.StreamMessage(s.new.URN,
|
|
|
|
message+"; importing this resource will fail", 0))
|
|
|
|
} else {
|
|
|
|
err = errors.New(message)
|
2020-10-14 11:51:53 +00:00
|
|
|
}
|
2022-03-08 12:27:15 +00:00
|
|
|
}
|
2020-10-14 11:51:53 +00:00
|
|
|
|
2022-03-08 12:27:15 +00:00
|
|
|
// If we were asked to replace an existing, non-External resource, pend the deletion here.
|
|
|
|
if err == nil && s.replacing {
|
|
|
|
s.original.Delete = true
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
|
2024-07-03 10:24:26 +00:00
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
return rst, complete, nil
|
2019-07-12 18:12:01 +00:00
|
|
|
}
|
|
|
|
|
2024-04-22 11:12:45 +00:00
|
|
|
func (s *ImportStep) Fail() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ImportStep) Skip() {
|
|
|
|
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
|
|
|
|
}
|
|
|
|
|
2017-06-10 01:34:37 +00:00
|
|
|
const (
|
2022-06-27 14:08:06 +00:00
|
|
|
OpSame display.StepOp = "same" // nothing to do.
|
|
|
|
OpCreate display.StepOp = "create" // creating a new resource.
|
|
|
|
OpUpdate display.StepOp = "update" // updating an existing resource.
|
|
|
|
OpDelete display.StepOp = "delete" // deleting an existing resource.
|
|
|
|
OpReplace display.StepOp = "replace" // replacing a resource with a new one.
|
|
|
|
OpCreateReplacement display.StepOp = "create-replacement" // creating a new resource for a replacement.
|
|
|
|
OpDeleteReplaced display.StepOp = "delete-replaced" // deleting an existing resource after replacement.
|
|
|
|
OpRead display.StepOp = "read" // reading an existing resource.
|
|
|
|
OpReadReplacement display.StepOp = "read-replacement" // reading an existing resource for a replacement.
|
|
|
|
OpRefresh display.StepOp = "refresh" // refreshing an existing resource.
|
|
|
|
OpReadDiscard display.StepOp = "discard" // removing a resource that was read.
|
|
|
|
OpDiscardReplaced display.StepOp = "discard-replaced" // discarding a read resource that was replaced.
|
|
|
|
OpRemovePendingReplace display.StepOp = "remove-pending-replace" // removing a pending replace resource.
|
|
|
|
OpImport display.StepOp = "import" // import an existing resource.
|
2024-05-06 17:34:24 +00:00
|
|
|
OpImportReplacement display.StepOp = "import-replacement" // replace an existing resource
|
|
|
|
// with an imported resource.
|
2017-06-10 01:34:37 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
// StepOps contains the full set of step operation types.
|
2022-06-27 14:08:06 +00:00
|
|
|
var StepOps = []display.StepOp{
|
2017-06-11 15:09:20 +00:00
|
|
|
OpSame,
|
2017-06-10 01:34:37 +00:00
|
|
|
OpCreate,
|
|
|
|
OpUpdate,
|
|
|
|
OpDelete,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
OpReplace,
|
2017-08-06 17:05:51 +00:00
|
|
|
OpCreateReplacement,
|
|
|
|
OpDeleteReplaced,
|
2018-08-03 21:06:00 +00:00
|
|
|
OpRead,
|
|
|
|
OpReadReplacement,
|
2018-08-23 00:52:46 +00:00
|
|
|
OpRefresh,
|
2019-01-31 21:48:44 +00:00
|
|
|
OpReadDiscard,
|
|
|
|
OpDiscardReplaced,
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 17:46:30 +00:00
|
|
|
OpRemovePendingReplace,
|
2019-07-12 18:12:01 +00:00
|
|
|
OpImport,
|
|
|
|
OpImportReplacement,
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
|
|
|
|
display: only hide replacement steps in diff (#16065)
When displaying diff events, we currently try to hide non-logical
replacement steps unless we specifically enable showing them. However we
currently do that for all non-logical operations, regardless whether
they are replacement steps or not.
In particular a RefreshStep is non-logical, but it's also not a
replacement step. We still want to show them during the diff because
their output can be important. Especially if the user just requested a
diff it doesn't make sense to hide the diff from them at the same time.
The intention here is to only hide replacement steps, so do that.
The full diff with the display tests is here:
https://gist.github.com/tgummerer/fcd012f13669a9cdc39530cde7770260 It's
unedited, so it includes some flakyness which isn't interesting.
I looked it over, and I think it looks like what we want, but I'm
curious to hear what others think. E.g.
https://gist.github.com/tgummerer/fcd012f13669a9cdc39530cde7770260#file-testdata-diff-L558
looks more correct now, as it shows the two delete operation that
actually happened, that it didn't show before, and it still shows the
same operation (Calling this one out in particular, since it took me a
bit to understand that we still have the same operation in the diff)
Fixes https://github.com/pulumi/pulumi/issues/7665
2024-04-26 14:54:21 +00:00
|
|
|
func IsReplacementStep(op display.StepOp) bool {
|
|
|
|
if op == OpReplace || op == OpCreateReplacement || op == OpDeleteReplaced ||
|
|
|
|
op == OpReadReplacement || op == OpDiscardReplaced || op == OpRemovePendingReplace ||
|
|
|
|
op == OpImportReplacement {
|
|
|
|
return true
|
|
|
|
}
|
|
|
|
return false
|
|
|
|
}
|
|
|
|
|
2017-06-10 01:34:37 +00:00
|
|
|
// Color returns a suggested color for lines of this op type.
|
2022-06-27 14:08:06 +00:00
|
|
|
func Color(op display.StepOp) string {
|
2017-06-10 01:34:37 +00:00
|
|
|
switch op {
|
2017-06-11 15:09:20 +00:00
|
|
|
case OpSame:
|
2017-11-17 02:21:41 +00:00
|
|
|
return colors.SpecUnimportant
|
2019-07-12 18:12:01 +00:00
|
|
|
case OpCreate, OpImport:
|
2017-08-06 17:05:51 +00:00
|
|
|
return colors.SpecCreate
|
2017-06-11 00:03:58 +00:00
|
|
|
case OpDelete:
|
2017-08-06 17:05:51 +00:00
|
|
|
return colors.SpecDelete
|
2017-06-10 01:34:37 +00:00
|
|
|
case OpUpdate:
|
2017-08-06 17:05:51 +00:00
|
|
|
return colors.SpecUpdate
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
case OpReplace:
|
2017-08-06 17:05:51 +00:00
|
|
|
return colors.SpecReplace
|
|
|
|
case OpCreateReplacement:
|
|
|
|
return colors.SpecCreateReplacement
|
|
|
|
case OpDeleteReplaced:
|
|
|
|
return colors.SpecDeleteReplaced
|
2018-08-03 21:06:00 +00:00
|
|
|
case OpRead:
|
2019-09-18 16:49:13 +00:00
|
|
|
return colors.SpecRead
|
2019-07-12 18:12:01 +00:00
|
|
|
case OpReadReplacement, OpImportReplacement:
|
2018-08-03 21:06:00 +00:00
|
|
|
return colors.SpecReplace
|
2018-08-23 00:52:46 +00:00
|
|
|
case OpRefresh:
|
|
|
|
return colors.SpecUpdate
|
2019-01-31 21:48:44 +00:00
|
|
|
case OpReadDiscard, OpDiscardReplaced:
|
|
|
|
return colors.SpecDelete
|
Don't re-delete resources that are `PendingReplacement` (#16510)
As well as indicating that a resource's state has changes, a diff can
also indicate that those changes require the _replacement_ of the
resource, meaning that it must be recreated and not just updated. In
this scenario, there are two possible ways to replace the resource -- by
first creating another new resource before deleting the old one
("create-before-replace"), or by first deleting the old resource before
creating its replacement ("delete-before-replace").
Create-before-replace is the default since generally, if possible to
implement, it should result in fewer instances of "downtime", where a
desired resource does not exist in the system.
Should delete-before-replace be chosen, Pulumi implements this under the
hood as three steps: delete for replacement, replace, and create
replacement. To track things consistently, as well as enable resumption
of an interrupted operation, Pulumi writes a flag, `PendingReplacement`
to the state of a deleted resource that will later be cleaned up by a
completed replacement.
Should an interrupted operation be resumed, Pulumi does not currently
take `PendingReplacement` into account, and always enqueues a(nother)
delete operation. This is typically fine (albeit wasteful) since deletes
are (should) be idempotent, but unnecessary. This commit adds
@jesse-triplewhale's fix for this behaviour whereby the
`PendingReplacement` flag is simply removed before the remainder of the
required steps (replace, create replacement) are actioned as normal. It
also extends this work with some lifecycle tests for this scenario and a
few others that may arise as a result of an interrupted replacement.
Fixes #16288
Closes #16303
Co-authored-by: Jesse Grodman <jesse@triplewhale.com>
2024-06-28 23:16:20 +00:00
|
|
|
case OpRemovePendingReplace:
|
|
|
|
return colors.SpecUnimportant
|
2017-06-10 01:34:37 +00:00
|
|
|
default:
|
2018-04-10 19:03:11 +00:00
|
|
|
contract.Failf("Unrecognized resource step op: '%v'", op)
|
2017-06-10 01:34:37 +00:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-09-28 22:16:09 +00:00
|
|
|
// ColorProgress returns a suggested coloring for lines of this of type which
|
|
|
|
// are progressing.
|
2022-06-27 14:08:06 +00:00
|
|
|
func ColorProgress(op display.StepOp) string {
|
|
|
|
return colors.Bold + Color(op)
|
2021-09-28 22:16:09 +00:00
|
|
|
}
|
|
|
|
|
2017-06-10 01:34:37 +00:00
|
|
|
// Prefix returns a suggested prefix for lines of this op type.
|
2022-06-27 14:08:06 +00:00
|
|
|
func Prefix(op display.StepOp, done bool) string {
|
2021-09-28 22:16:09 +00:00
|
|
|
var color string
|
|
|
|
if done {
|
2022-06-27 14:08:06 +00:00
|
|
|
color = Color(op)
|
2021-09-28 22:16:09 +00:00
|
|
|
} else {
|
2022-06-27 14:08:06 +00:00
|
|
|
color = ColorProgress(op)
|
2021-09-28 22:16:09 +00:00
|
|
|
}
|
2022-06-27 14:08:06 +00:00
|
|
|
return color + RawPrefix(op)
|
2017-11-25 23:54:01 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// RawPrefix returns the uncolorized prefix text.
|
2022-06-27 14:08:06 +00:00
|
|
|
func RawPrefix(op display.StepOp) string {
|
2017-06-10 01:34:37 +00:00
|
|
|
switch op {
|
2017-08-30 01:24:12 +00:00
|
|
|
case OpSame:
|
2018-09-22 20:07:43 +00:00
|
|
|
return " "
|
2017-06-10 01:34:37 +00:00
|
|
|
case OpCreate:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "+ "
|
2017-06-10 01:34:37 +00:00
|
|
|
case OpDelete:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "- "
|
2017-06-10 01:34:37 +00:00
|
|
|
case OpUpdate:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "~ "
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
case OpReplace:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "+-"
|
2017-08-06 17:05:51 +00:00
|
|
|
case OpCreateReplacement:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "++"
|
2017-08-06 17:05:51 +00:00
|
|
|
case OpDeleteReplaced:
|
2017-11-25 23:54:01 +00:00
|
|
|
return "--"
|
2018-08-03 21:06:00 +00:00
|
|
|
case OpRead:
|
2019-01-31 21:48:44 +00:00
|
|
|
return "> "
|
2018-08-03 21:06:00 +00:00
|
|
|
case OpReadReplacement:
|
2019-01-31 21:48:44 +00:00
|
|
|
return ">>"
|
2018-08-23 00:52:46 +00:00
|
|
|
case OpRefresh:
|
|
|
|
return "~ "
|
2019-01-31 21:48:44 +00:00
|
|
|
case OpReadDiscard:
|
|
|
|
return "< "
|
|
|
|
case OpDiscardReplaced:
|
|
|
|
return "<<"
|
2019-07-12 18:12:01 +00:00
|
|
|
case OpImport:
|
|
|
|
return "= "
|
|
|
|
case OpImportReplacement:
|
|
|
|
return "=>"
|
Don't re-delete resources that are `PendingReplacement` (#16510)
As well as indicating that a resource's state has changes, a diff can
also indicate that those changes require the _replacement_ of the
resource, meaning that it must be recreated and not just updated. In
this scenario, there are two possible ways to replace the resource -- by
first creating another new resource before deleting the old one
("create-before-replace"), or by first deleting the old resource before
creating its replacement ("delete-before-replace").
Create-before-replace is the default since generally, if possible to
implement, it should result in fewer instances of "downtime", where a
desired resource does not exist in the system.
Should delete-before-replace be chosen, Pulumi implements this under the
hood as three steps: delete for replacement, replace, and create
replacement. To track things consistently, as well as enable resumption
of an interrupted operation, Pulumi writes a flag, `PendingReplacement`
to the state of a deleted resource that will later be cleaned up by a
completed replacement.
Should an interrupted operation be resumed, Pulumi does not currently
take `PendingReplacement` into account, and always enqueues a(nother)
delete operation. This is typically fine (albeit wasteful) since deletes
are (should) be idempotent, but unnecessary. This commit adds
@jesse-triplewhale's fix for this behaviour whereby the
`PendingReplacement` flag is simply removed before the remainder of the
required steps (replace, create replacement) are actioned as normal. It
also extends this work with some lifecycle tests for this scenario and a
few others that may arise as a result of an interrupted replacement.
Fixes #16288
Closes #16303
Co-authored-by: Jesse Grodman <jesse@triplewhale.com>
2024-06-28 23:16:20 +00:00
|
|
|
case OpRemovePendingReplace:
|
|
|
|
return "~ "
|
2017-06-10 01:34:37 +00:00
|
|
|
default:
|
|
|
|
contract.Failf("Unrecognized resource step op: %v", op)
|
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-06-27 14:08:06 +00:00
|
|
|
func PastTense(op display.StepOp) string {
|
2017-09-27 23:58:34 +00:00
|
|
|
switch op {
|
2024-05-06 17:34:24 +00:00
|
|
|
case OpSame, OpCreate, OpReplace, OpCreateReplacement, OpUpdate, OpReadReplacement:
|
2017-09-27 23:58:34 +00:00
|
|
|
return string(op) + "d"
|
2018-08-23 00:52:46 +00:00
|
|
|
case OpRefresh:
|
|
|
|
return "refreshed"
|
2018-08-03 21:06:00 +00:00
|
|
|
case OpRead:
|
|
|
|
return "read"
|
2019-01-31 21:48:44 +00:00
|
|
|
case OpReadDiscard, OpDiscardReplaced:
|
|
|
|
return "discarded"
|
2022-03-07 11:43:07 +00:00
|
|
|
case OpDelete, OpDeleteReplaced:
|
|
|
|
return "deleted"
|
2019-07-12 18:12:01 +00:00
|
|
|
case OpImport, OpImportReplacement:
|
|
|
|
return "imported"
|
2017-09-27 23:58:34 +00:00
|
|
|
default:
|
2017-11-29 18:06:51 +00:00
|
|
|
contract.Failf("Unexpected resource step op: %v", op)
|
2017-09-27 23:58:34 +00:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-06-10 01:34:37 +00:00
|
|
|
// Suffix returns a suggested suffix for lines of this op type.
|
2022-06-27 14:08:06 +00:00
|
|
|
func Suffix(op display.StepOp) string {
|
2019-07-12 18:12:01 +00:00
|
|
|
switch op {
|
|
|
|
case OpCreateReplacement, OpUpdate, OpReplace, OpReadReplacement, OpRefresh, OpImportReplacement:
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
|
|
|
return colors.Reset // updates and replacements colorize individual lines; get has none
|
2017-06-10 01:34:37 +00:00
|
|
|
}
|
|
|
|
return ""
|
|
|
|
}
|
2017-11-21 01:38:09 +00:00
|
|
|
|
2022-01-31 10:31:51 +00:00
|
|
|
// ConstrainedTo returns true if this operation is no more impactful than the constraint.
|
2022-06-27 14:08:06 +00:00
|
|
|
func ConstrainedTo(op display.StepOp, constraint display.StepOp) bool {
|
|
|
|
var allowed []display.StepOp
|
2022-01-31 10:31:51 +00:00
|
|
|
switch constraint {
|
|
|
|
case OpSame, OpDelete, OpRead, OpReadReplacement, OpRefresh, OpReadDiscard, OpDiscardReplaced,
|
|
|
|
OpRemovePendingReplace, OpImport, OpImportReplacement:
|
2022-06-27 14:08:06 +00:00
|
|
|
allowed = []display.StepOp{constraint}
|
2022-01-31 10:31:51 +00:00
|
|
|
case OpCreate:
|
2022-06-27 14:08:06 +00:00
|
|
|
allowed = []display.StepOp{OpSame, OpCreate}
|
2022-01-31 10:31:51 +00:00
|
|
|
case OpUpdate:
|
2022-06-27 14:08:06 +00:00
|
|
|
allowed = []display.StepOp{OpSame, OpUpdate}
|
2022-01-31 10:31:51 +00:00
|
|
|
case OpReplace, OpCreateReplacement, OpDeleteReplaced:
|
2022-06-27 14:08:06 +00:00
|
|
|
allowed = []display.StepOp{OpSame, OpUpdate, constraint}
|
2022-01-31 10:31:51 +00:00
|
|
|
}
|
|
|
|
for _, candidate := range allowed {
|
|
|
|
if candidate == op {
|
|
|
|
return true
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return false
|
|
|
|
}
|
|
|
|
|
2017-11-21 01:38:09 +00:00
|
|
|
// getProvider fetches the provider for the given step.
|
2023-12-22 21:14:04 +00:00
|
|
|
func getProvider(s Step, override plugin.Provider) (plugin.Provider, error) {
|
|
|
|
if override != nil {
|
|
|
|
return override, nil
|
|
|
|
}
|
2020-07-09 14:19:12 +00:00
|
|
|
if providers.IsProviderType(s.Type()) {
|
2020-11-18 17:47:52 +00:00
|
|
|
return s.Deployment().providers, nil
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
|
|
|
}
|
|
|
|
ref, err := providers.ParseReference(s.Provider())
|
|
|
|
if err != nil {
|
2023-12-20 15:54:06 +00:00
|
|
|
return nil, fmt.Errorf("bad provider reference '%v' for resource %v: %w", s.Provider(), s.URN(), err)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
|
|
|
}
|
2022-01-26 17:08:36 +00:00
|
|
|
if providers.IsDenyDefaultsProvider(ref) {
|
|
|
|
pkg := providers.GetDeniedDefaultProviderPkg(ref)
|
|
|
|
msg := diag.GetDefaultProviderDenied(s.URN()).Message
|
|
|
|
return nil, fmt.Errorf(msg, pkg, s.URN())
|
|
|
|
}
|
2020-11-18 17:47:52 +00:00
|
|
|
provider, ok := s.Deployment().GetProvider(ref)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
|
|
|
if !ok {
|
2021-11-13 02:37:17 +00:00
|
|
|
return nil, fmt.Errorf("unknown provider '%v' for resource %v", s.Provider(), s.URN())
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
|
|
|
}
|
|
|
|
return provider, nil
|
2017-11-21 01:38:09 +00:00
|
|
|
}
|