pulumi/pkg/resource/deploy/step.go

1669 lines
63 KiB
Go
Raw Permalink Normal View History

Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
// Copyright 2016-2023, Pulumi Corporation.
2018-05-22 19:43:36 +00:00
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package deploy
import (
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/pulumi/pulumi/pkg/v3/display"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/providers"
"github.com/pulumi/pulumi/sdk/v3/go/common/diag"
"github.com/pulumi/pulumi/sdk/v3/go/common/diag/colors"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
)
// StepCompleteFunc is the type of functions returned from Step.Apply. These
// functions are to be called when the engine has fully retired a step. You
// _should not_ modify the resource state in these functions -- doing so will
// race with the snapshot writing code.
type StepCompleteFunc func()
// Step is a specification for a deployment operation.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type Step interface {
// Apply applies this step. It returns the status of the resource after the
// step application, a function to call to signal that this step has fully
// completed, and an error, if one occurred while applying the step.
//
// The returned StepCompleteFunc, if not nil, must be called after committing
// the results of this step into the state of the deployment.
Apply() (resource.Status, StepCompleteFunc, error)
// the operation performed by this step.
Op() display.StepOp
// the resource URN (for before and after).
URN() resource.URN
// the type of the resource affected by this step.
Type() tokens.Type
// the provider reference for the resource affected by this step.
Provider() string
// the state of the resource before performing this step.
Old() *resource.State
// the state of the resource after performing this step.
New() *resource.State
// the latest state for the resource that is known (worst case, old).
Res() *resource.State
// true if this step represents a logical operation in the program.
Logical() bool
// the deployment to which this step belongs.
Deployment() *Deployment
// Calling Fail will mark the step as failed.
Fail()
// Calling Skip will mark the step as skipped.
Skip()
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
// SameStep is a mutating step that does nothing.
type SameStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the resource before this step.
new *resource.State // the state of the resource after this step.
// If this is a same-step for a resource being created but which was not --target'ed by the user
// (and thus was skipped).
skippedCreate bool
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*SameStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewSameStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is custom")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new == old || new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &SameStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
// NewSkippedCreateStep produces a SameStep for a resource that was created but not targeted
// by the user (and thus was skipped). These act as no-op steps (hence 'same') since we are not
// actually creating the resource, but ensure that we complete resource-registration and convey the
// right information downstream. For example, we will not write these into the checkpoint file.
func NewSkippedCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
// Make the old state here a direct copy of the new state
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old := new.Copy()
return &SameStep{
deployment: deployment,
reg: reg,
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old: old,
new: new,
skippedCreate: true,
}
}
func (s *SameStep) Op() display.StepOp { return OpSame }
func (s *SameStep) Deployment() *Deployment { return s.deployment }
func (s *SameStep) Type() tokens.Type { return s.new.Type }
func (s *SameStep) Provider() string { return s.new.Provider }
func (s *SameStep) URN() resource.URN { return s.new.URN }
func (s *SameStep) Old() *resource.State { return s.old }
func (s *SameStep) New() *resource.State { return s.new }
func (s *SameStep) Res() *resource.State { return s.new }
func (s *SameStep) Logical() bool { return true }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *SameStep) Apply() (resource.Status, StepCompleteFunc, error) {
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
2021-07-28 19:12:53 +00:00
// Retain the ID and outputs
s.new.ID = s.old.ID
s.new.Outputs = s.old.Outputs
2021-07-28 19:12:53 +00:00
// If the resource is a provider, ensure that it is present in the registry under the appropriate URNs.
// We can only do this if the provider is actually a same, not a skipped create.
if providers.IsProviderType(s.new.Type) && !s.skippedCreate {
2021-07-28 19:12:53 +00:00
if s.Deployment() != nil {
Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
err := s.Deployment().SameProvider(s.new)
if err != nil {
return resource.StatusOK, nil,
fmt.Errorf("bad provider state for resource %v: %w", s.URN(), err)
Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
}
2021-07-28 19:12:53 +00:00
}
}
// TODO: should this step be marked as skipped if it comes from a targeted up?
Propagate deleted dependencies of untargeted resources (#16247) When using `--target` to target specific resources during an update, we use the list of targets to decide which steps to generate given a set of resource registrations. Specifically: * If the registration event names a resource that is targeted, we process it as usual. * If the registration event names a resource that _is not_ targeted, we emit a `SameStep` for it. In the latter case, the emission of a `SameStep` means that the old state for the resource will be copied across to the new state. This is the desired behaviour -- the resource was not targeted and so the new state should contain the resource exactly as it was prior to the update. However, this presents a problem if the old state has references to resources that either will not appear in the new state, or will appear in the wrong place. Consider the following program in TypeScript-esque pseudocode: ```typescript const a = new Resource("a") const b = new Resource("b", { dependency: a }) const c = new Resource("c") ``` Here, `b` depends on `a`, while `a` and `c` have no dependencies. We run this program without specifying targets and obtain a state containing `a`, `b` and `c`, with `a` appearing before `b` due to `b`'s dependency on `a`. We now modify the program as follows: ```typescript const b = new Resource("b") const c = new Resource("c") ``` `a` has been removed from the program and consequently `b` no longer depends on it. We once more run the program, this time with a `--target` of `c`. That is to say, neither `a` nor `b` is targeted. The execution proceeds as follows: * `a` is not in the program, so no `RegisterResourceEvent` will be emitted and processed for it. * `b` is in the program, but it is not targeted. Its `RegisterResourceEvent` will be turned into a `SameStep` and `b`'s _old state will be copied as-is to the new state_. * `c` is in the program and is targeted. It will be processed as normal. At the end of execution when we come to write the snapshot, we take the following actions: * We first write the processed resources: `b`'s old state and `c`'s new state. * We then copy over any unprocessed resources from the base (previous) snapshot. This includes `a` (which is again desirable since its deletion should not be processed due to it not being targeted). Our snapshot is now not topologically sorted and thus invalid: `b` has a dependency on `a`, but `a` appears after it. Presently this bug will manifest irrespective of the nature of the dependency: `.Dependencies`, `.PropertyDependencies` and `.DeletedWith` are all affected. This commit fixes this issue by traversing all untargeted resource dependency relationships and ensuring that `SameStep`s (or better if they have been targeted) are emitted before emitting the depending resource's `SameStep`. * Fixes #16052 * Fixes #15959
2024-05-23 12:31:03 +00:00
complete := func() {
// It's possible that s.reg will be nil in the case that multiple same steps
// are emitted for a single RegisterResourceEvent. This occurs when a
// resource which is not targeted by a targeted operation needs to ensure
// that old dependencies are brought forward into the new state before it
// is. In these cases the root resource will be instantiated with its event
// while its dependencies will have a nil event. This is fine since in these
// cases the only Done callback we care about is the one for the root
// resource.
if s.reg != nil {
s.reg.Done(&RegisterResult{State: s.new})
}
}
return resource.StatusOK, complete, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *SameStep) IsSkippedCreate() bool {
return s.skippedCreate
}
func (s *SameStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *SameStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// CreateStep is a mutating step that creates an entirely new resource.
type CreateStep struct {
deployment *Deployment // the current deployment.
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the existing resource (only for replacements).
new *resource.State // the state of the resource after this step.
keys []resource.PropertyKey // the keys causing replacement (only for replacements).
diffs []resource.PropertyKey // the keys causing a diff (only for replacements).
detailedDiff map[string]plugin.PropertyDiff // the structured property diff (only for replacements).
replacing bool // true if this is a create due to a replacement.
pendingDelete bool // true if this replacement should create a pending delete.
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*CreateStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
contract.Requiref(reg != nil, "reg", "must not be nil")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &CreateStep{
deployment: deployment,
reg: reg,
new: new,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func NewCreateReplacementStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
keys, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
) Step {
contract.Requiref(reg != nil, "reg", "must not be nil")
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
return &CreateStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
keys: keys,
diffs: diffs,
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
detailedDiff: detailedDiff,
replacing: true,
pendingDelete: pendingDelete,
}
}
func (s *CreateStep) Op() display.StepOp {
if s.replacing {
return OpCreateReplacement
}
return OpCreate
}
func (s *CreateStep) Deployment() *Deployment { return s.deployment }
func (s *CreateStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *CreateStep) Provider() string { return s.new.Provider }
func (s *CreateStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *CreateStep) Old() *resource.State { return s.old }
func (s *CreateStep) New() *resource.State { return s.new }
func (s *CreateStep) Res() *resource.State { return s.new }
func (s *CreateStep) Keys() []resource.PropertyKey { return s.keys }
func (s *CreateStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *CreateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *CreateStep) Logical() bool { return !s.replacing }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *CreateStep) Apply() (resource.Status, StepCompleteFunc, error) {
var resourceError error
resourceStatus := resource.StatusOK
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
id := s.new.ID
outs := s.new.Outputs
if s.new.Custom {
// Invoke the Create RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
2019-07-15 21:26:28 +00:00
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resp, err := prov.Create(context.TODO(), plugin.CreateRequest{
URN: s.URN(),
Properties: s.new.Inputs,
Timeout: s.new.CustomTimeouts.Create,
Clean up deployment options (#16357) # Description There are a number of parts of the deployment process that require context about and configuration for the operation being executed. For instance: * Source evaluation -- evaluating programs in order to emit resource registrations * Step generation -- processing resource registrations in order to generate steps (create this, update that, delete the other, etc.) * Step execution -- executing steps in order to action a deployment. Presently, these pieces all take some form of `Options` struct or pass explicit arguments. This is problematic for a couple of reasons: * It could be possible for different parts of the codebase to end up operating in different contexts/with different configurations, whether due to different values being passed explicitly or due to missed copying/instantiation. * Some parts need less context/configuration than others, but still accept full `Options`, making it hard to discern what information is actually necessary in any given part of the process. This commit attempts to clean things up by moving deployment options directly into the `Deployment` itself. Since step generation and execution already refer to a `Deployment`, they get a consistent view of the options for free. For source evaluation, we introduce an `EvalSourceOptions` struct for configuring just the options necessary there. At the top level, the engine configures a single set of options to flow through the deployment steps later on. As part of this work, a few other things have been changed: * Preview/dry-run parameters have been incorporated into options. This lets up lop off another argument and mitigate a bit of "boolean blindness". We don't appear to flip this flag within a deployment process (indeed, all options seem to be immutable) and so having it as a separate flag doesn't seem to buy us anything. * Several methods representing parts of the deployment process have lost arguments in favour of state that is already being carried on (or can be carried on) their receiver. For instance, `deployment.run` no longer takes actions or preview configuration. While doing so means that a `deployment` could be run multiple times with different actions/preview arguments, we don't currently exploit this fact anywhere, so moving this state to the point of construction both simplifies things considerably and reduces the possibility for error (e.g. passing different values of `preview` when instantiating a `deployment` and subsequently calling `run`). * Event handlers have been split out of the options object and attached to `Deployment` separately. This means we can talk about options at a higher level without having to `nil` out/worry about this field and mutate it correctly later on. * Options are no longer mutated during deployment. Presently there appears to be only one case of this -- when handling `ContinueOnError` in the presence of `IgnoreChanges` (e.g. when performing a refresh). This case has been refactored so that the mutation is no longer necessary. # Notes * This change is in preparation for #16146, where we'd like to add an environment variable to control behaviour and having a single unified `Options` struct would make it easier to pass this configuration down with introducing (more) global state into deployments. Indeed, this change should make it easier to factor global state into `Options` so that it can be controlled and tested more easily/is less susceptible to bugs, race conditions, etc. * I've tweaked/extended some comments while I'm here and have learned things the hard way (e.g. `Refresh` vs `isRefresh`). Feedback welcome on this if we'd rather not conflate. * This change does mean that if in future we wanted e.g. to be able to run a `Deployment` in multiple different ways with multiple sets of actions, we'd have to refactor. Pushing state to the point of object construction reduces the flexibility of the code. However, since we are not presently using that flexibility (nor is there an obvious [to my mind] use case in the near future), this seems like a good trade-off to guard against bugs/make it simpler to move that state around. * I've left some other review comments in the code around questions/changes that might be a bad idea; happy to receive feedback on it all though!
2024-06-11 13:37:57 +00:00
Preview: s.deployment.opts.DryRun,
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
})
if err != nil {
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if resp.Status != resource.StatusPartialFailure {
return resp.Status, nil, err
}
resourceError = err
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resourceStatus = resp.Status
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
id = resp.ID
outs = resp.Properties
if !s.deployment.opts.DryRun && id == "" {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return resourceStatus, nil, errors.New("provider did not return an ID from Create")
Implement components This change implements core support for "components" in the Pulumi Fabric. This work is described further in pulumi/pulumi#340, where we are still discussing some of the finer points. In a nutshell, resources no longer imply external providers. It's entirely possible to have a resource that logically represents something but without having a physical manifestation that needs to be tracked and managed by our typical CRUD operations. For example, the aws/serverless/Function helper is one such type. It aggregates Lambda-related resources and exposes a nice interface. All of the Pulumi Cloud Framework resources are also examples. To indicate that a resource does participate in the usual CRUD resource provider, it simply derives from ExternalResource instead of Resource. All resources now have the ability to adopt children. This is purely a metadata/tagging thing, and will help us roll up displays, provide attribution to the developer, and even hide aspects of the resource graph as appropriate (e.g., when they are implementation details). Our use of this capability is ultra limited right now; in fact, the only place we display children is in the CLI output. For instance: + aws:serverless:Function: (create) [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda] => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0 => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda The bit indicating whether a resource is external or not is tracked in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
// Copy any of the default and output properties on the live object state.
s.new.ID = id
s.new.Outputs = outs
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Create should set the Create and Modified timestamps as the resource state has been created.
now := time.Now().UTC()
s.new.Created = &now
s.new.Modified = &now
// Mark the old resource as pending deletion if necessary.
if s.replacing && s.pendingDelete {
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
contract.Assertf(s.old != s.new, "old and new states should not be the same")
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.old.Lock.Lock()
s.old.Delete = true
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.old.Lock.Unlock()
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR ensures that we only run the `StepCompleteFunc` if the step is successful. If not, we should use the step's `Fail` method to register the result. Previously, when a user runs a pulumi up operation with `--continue-on-error`, the engine would register the result twice, once in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342 and if a failure occurs, it is re-registered in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373 I suspect that the double registration is the cause of the user facing error. Since this would only occur for create and update steps, it also explains why the error doesn't occur during deletions. Testing: - Manually ran the repro identified in #16373 and https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614 against a locally built CLI with this fix. - Extended existing lifecycle tests to account for the different code branches of `CreateStep.Apply()` and `UpdateStep.Apply()` Fixes #16373 ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-13 15:42:57 +00:00
if resourceError != nil {
// If we have a failure, we should return an empty complete function
// and let the Fail method handle the registration.
return resourceStatus, nil, resourceError
2018-07-06 22:17:32 +00:00
}
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR ensures that we only run the `StepCompleteFunc` if the step is successful. If not, we should use the step's `Fail` method to register the result. Previously, when a user runs a pulumi up operation with `--continue-on-error`, the engine would register the result twice, once in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342 and if a failure occurs, it is re-registered in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373 I suspect that the double registration is the cause of the user facing error. Since this would only occur for create and update steps, it also explains why the error doesn't occur during deletions. Testing: - Manually ran the repro identified in #16373 and https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614 against a locally built CLI with this fix. - Extended existing lifecycle tests to account for the different code branches of `CreateStep.Apply()` and `UpdateStep.Apply()` Fixes #16373 ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-13 15:42:57 +00:00
return resourceStatus, complete, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *CreateStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *CreateStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
// DeleteStep is a mutating step that deletes an existing resource. If `old` is marked "External",
// DeleteStep is a no-op.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type DeleteStep struct {
deployment *Deployment // the current deployment.
old *resource.State // the state of the existing resource.
replacing bool // true if part of a replacement.
otherDeletions map[resource.URN]bool // other resources that are planned to delete
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*DeleteStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewDeleteStep(deployment *Deployment, otherDeletions map[resource.URN]bool, old *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
return &DeleteStep{
deployment: deployment,
old: old,
otherDeletions: otherDeletions,
}
}
func NewDeleteReplacementStep(
deployment *Deployment,
otherDeletions map[resource.URN]bool,
old *resource.State,
pendingReplace bool,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
// There are two cases in which we create a delete-replacment step:
//
// 1. When creating the delete steps that occur due to a delete-before-replace
// 2. When creating the delete step that occurs due to a delete-after-replace
//
// In the former case, the persistence layer may require that the resource remain in the
// checkpoint file for purposes of checkpoint integrity. We communicate this case by means
// of the `PendingReplacement` field on `resource.State`, which we set here.
//
// In the latter case, the resource must be deleted, but the deletion may not occur if an earlier step fails.
// The engine requires that the fact that the old resource must be deleted is persisted in the checkpoint so
// that it can issue a deletion of this resource on the next update to this stack.
contract.Assertf(pendingReplace != old.Delete,
"resource %v cannot be pending replacement and deletion at the same time", old.URN)
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old.Lock.Lock()
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
old.PendingReplacement = pendingReplace
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old.Lock.Unlock()
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &DeleteStep{
deployment: deployment,
otherDeletions: otherDeletions,
old: old,
replacing: true,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func (s *DeleteStep) Op() display.StepOp {
if s.old.External {
if s.replacing {
return OpDiscardReplaced
}
return OpReadDiscard
}
if s.replacing {
return OpDeleteReplaced
}
return OpDelete
}
func (s *DeleteStep) Deployment() *Deployment { return s.deployment }
func (s *DeleteStep) Type() tokens.Type { return s.old.Type }
func (s *DeleteStep) Provider() string { return s.old.Provider }
func (s *DeleteStep) URN() resource.URN { return s.old.URN }
func (s *DeleteStep) Old() *resource.State { return s.old }
func (s *DeleteStep) New() *resource.State { return nil }
func (s *DeleteStep) Res() *resource.State { return s.old }
func (s *DeleteStep) Logical() bool { return !s.replacing }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func isDeletedWith(with resource.URN, otherDeletions map[resource.URN]bool) bool {
if with == "" {
return false
}
r, ok := otherDeletions[with]
if !ok {
return false
}
return r
}
2023-01-10 18:32:12 +00:00
type deleteProtectedError struct {
urn resource.URN
}
func (d deleteProtectedError) Error() string {
return fmt.Sprintf("resource %[1]q cannot be deleted\n"+
"because it is protected. To unprotect the resource, "+
"either remove the `protect` flag from the resource in your Pulumi "+
"program and run `pulumi up`, or use the command:\n"+
"`pulumi state unprotect %[2]s`", d.urn, d.urn.Quote())
2023-01-10 18:32:12 +00:00
}
func (s *DeleteStep) Apply() (resource.Status, StepCompleteFunc, error) {
// Refuse to delete protected resources (unless we're replacing them in
// which case we will of checked protect elsewhere)
if !s.replacing && s.old.Protect {
2023-01-10 18:32:12 +00:00
return resource.StatusOK, nil, deleteProtectedError{urn: s.old.URN}
}
if s.deployment.opts.DryRun {
// Do nothing in preview
} else if s.old.External {
// Deleting an External resource is a no-op, since Pulumi does not own the lifecycle.
} else if s.old.RetainOnDelete {
// Deleting a "drop on delete" is a no-op as the user has explicitly asked us to not delete the resource.
} else if isDeletedWith(s.old.DeletedWith, s.otherDeletions) {
// No need to delete this resource since this resource will be deleted by the another deletion
} else if s.old.Custom {
// Not preview and not external and not Drop and is custom, do the actual delete
// Invoke the Delete RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
2019-07-15 21:26:28 +00:00
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if rst, err := prov.Delete(context.TODO(), plugin.DeleteRequest{
URN: s.URN(),
ID: s.old.ID,
Inputs: s.old.Inputs,
Outputs: s.old.Outputs,
Timeout: s.old.CustomTimeouts.Delete,
}); err != nil {
return rst.Status, nil, err
Implement components This change implements core support for "components" in the Pulumi Fabric. This work is described further in pulumi/pulumi#340, where we are still discussing some of the finer points. In a nutshell, resources no longer imply external providers. It's entirely possible to have a resource that logically represents something but without having a physical manifestation that needs to be tracked and managed by our typical CRUD operations. For example, the aws/serverless/Function helper is one such type. It aggregates Lambda-related resources and exposes a nice interface. All of the Pulumi Cloud Framework resources are also examples. To indicate that a resource does participate in the usual CRUD resource provider, it simply derives from ExternalResource instead of Resource. All resources now have the ability to adopt children. This is purely a metadata/tagging thing, and will help us roll up displays, provide attribution to the developer, and even hide aspects of the resource graph as appropriate (e.g., when they are implementation details). Our use of this capability is ultra limited right now; in fact, the only place we display children is in the CLI output. For instance: + aws:serverless:Function: (create) [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda] => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0 => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda The bit indicating whether a resource is external or not is tracked in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
return resource.StatusOK, func() {}, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *DeleteStep) Fail() {
// Nothing to do here.
}
func (s *DeleteStep) Skip() {
// Nothing to do here.
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
type RemovePendingReplaceStep struct {
deployment *Deployment // the current deployment.
old *resource.State // the state of the existing resource.
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
}
func NewRemovePendingReplaceStep(deployment *Deployment, old *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.PendingReplacement, "old", "must be pending replacement")
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
return &RemovePendingReplaceStep{
deployment: deployment,
old: old,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
}
}
func (s *RemovePendingReplaceStep) Op() display.StepOp {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
return OpRemovePendingReplace
}
func (s *RemovePendingReplaceStep) Deployment() *Deployment { return s.deployment }
func (s *RemovePendingReplaceStep) Type() tokens.Type { return s.old.Type }
func (s *RemovePendingReplaceStep) Provider() string { return s.old.Provider }
func (s *RemovePendingReplaceStep) URN() resource.URN { return s.old.URN }
func (s *RemovePendingReplaceStep) Old() *resource.State { return s.old }
func (s *RemovePendingReplaceStep) New() *resource.State { return nil }
func (s *RemovePendingReplaceStep) Res() *resource.State { return s.old }
func (s *RemovePendingReplaceStep) Logical() bool { return false }
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
func (s *RemovePendingReplaceStep) Apply() (resource.Status, StepCompleteFunc, error) {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
return resource.StatusOK, nil, nil
}
func (s *RemovePendingReplaceStep) Fail() {
// Nothing to do here.
}
func (s *RemovePendingReplaceStep) Skip() {
// Nothing to do here.
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// UpdateStep is a mutating step that updates an existing resource's state.
type UpdateStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the existing resource.
new *resource.State // the newly computed state of the resource after updating.
stables []resource.PropertyKey // an optional list of properties that won't change during this update.
diffs []resource.PropertyKey // the keys causing a diff.
detailedDiff map[string]plugin.PropertyDiff // the structured diff.
ignoreChanges []string // a list of property paths to ignore when updating.
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*UpdateStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewUpdateStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
stables, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(!old.External, "old", "must not be an external resource")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be an external resource")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &UpdateStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
stables: stables,
diffs: diffs,
detailedDiff: detailedDiff,
ignoreChanges: ignoreChanges,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func (s *UpdateStep) Op() display.StepOp { return OpUpdate }
func (s *UpdateStep) Deployment() *Deployment { return s.deployment }
func (s *UpdateStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *UpdateStep) Provider() string { return s.new.Provider }
func (s *UpdateStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *UpdateStep) Old() *resource.State { return s.old }
func (s *UpdateStep) New() *resource.State { return s.new }
func (s *UpdateStep) Res() *resource.State { return s.new }
func (s *UpdateStep) Logical() bool { return true }
func (s *UpdateStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *UpdateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *UpdateStep) Apply() (resource.Status, StepCompleteFunc, error) {
// Always propagate the ID and timestamps even in previews and refreshes.
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
s.new.ID = s.old.ID
s.new.Created = s.old.Created
s.new.Modified = s.old.Modified
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Unlock()
var resourceError error
resourceStatus := resource.StatusOK
if s.new.Custom {
// Invoke the Update RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
General prep work for refresh This change includes a bunch of refactorings I made in prep for doing refresh (first, the command, see pulumi/pulumi#1081): * The primary change is to change the way the engine's core update functionality works with respect to deploy.Source. This is the way we can plug in new sources of resource information during planning (and, soon, diffing). The way I intend to model refresh is by having a new kind of source, deploy.RefreshSource, which will let us do virtually everything about an update/diff the same way with refreshes, which avoid otherwise duplicative effort. This includes changing the planOptions (nee deployOptions) to take a new SourceFunc callback, which is responsible for creating a source specific to the kind of plan being requested. Preview, Update, and Destroy now are primarily differentiated by the kind of deploy.Source that they return, rather than sprinkling things like `if Destroying` throughout. This tidies up some logic and, more importantly, gives us precisely the refresh hook we need. * Originally, we used the deploy.NullSource for Destroy operations. This simply returns nothing, which is how Destroy works. For some reason, we were no longer doing this, and instead had some `if Destroying` cases sprinkled throughout the deploy.EvalSource. I think this is a vestige of some old way we did configuration, at least judging by a comment, which is apparently no longer relevant. * Move diff and diff-printing logic within the engine into its own pkg/engine/diff.go file, to prepare for upcoming work. * I keep noticing benign diffs anytime I regenerate protobufs. I suspect this is because we're also on different versions. I changed generate.sh to also dump the version into grpc_version.txt. At least we can understand where the diffs are coming from, decide whether to take them (i.e., a newer version), and ensure that as a team we are monotonically increasing, and not going backwards. * I also tidied up some tiny things I noticed while in there, like comments, incorrect types, lint suppressions, and so on.
2018-03-28 14:45:23 +00:00
// Update to the combination of the old "all" state, but overwritten with new inputs.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resp, upderr := prov.Update(context.TODO(), plugin.UpdateRequest{
URN: s.URN(),
ID: s.old.ID,
OldInputs: s.old.Inputs,
OldOutputs: s.old.Outputs,
NewInputs: s.new.Inputs,
Timeout: s.new.CustomTimeouts.Update,
IgnoreChanges: s.ignoreChanges,
Clean up deployment options (#16357) # Description There are a number of parts of the deployment process that require context about and configuration for the operation being executed. For instance: * Source evaluation -- evaluating programs in order to emit resource registrations * Step generation -- processing resource registrations in order to generate steps (create this, update that, delete the other, etc.) * Step execution -- executing steps in order to action a deployment. Presently, these pieces all take some form of `Options` struct or pass explicit arguments. This is problematic for a couple of reasons: * It could be possible for different parts of the codebase to end up operating in different contexts/with different configurations, whether due to different values being passed explicitly or due to missed copying/instantiation. * Some parts need less context/configuration than others, but still accept full `Options`, making it hard to discern what information is actually necessary in any given part of the process. This commit attempts to clean things up by moving deployment options directly into the `Deployment` itself. Since step generation and execution already refer to a `Deployment`, they get a consistent view of the options for free. For source evaluation, we introduce an `EvalSourceOptions` struct for configuring just the options necessary there. At the top level, the engine configures a single set of options to flow through the deployment steps later on. As part of this work, a few other things have been changed: * Preview/dry-run parameters have been incorporated into options. This lets up lop off another argument and mitigate a bit of "boolean blindness". We don't appear to flip this flag within a deployment process (indeed, all options seem to be immutable) and so having it as a separate flag doesn't seem to buy us anything. * Several methods representing parts of the deployment process have lost arguments in favour of state that is already being carried on (or can be carried on) their receiver. For instance, `deployment.run` no longer takes actions or preview configuration. While doing so means that a `deployment` could be run multiple times with different actions/preview arguments, we don't currently exploit this fact anywhere, so moving this state to the point of construction both simplifies things considerably and reduces the possibility for error (e.g. passing different values of `preview` when instantiating a `deployment` and subsequently calling `run`). * Event handlers have been split out of the options object and attached to `Deployment` separately. This means we can talk about options at a higher level without having to `nil` out/worry about this field and mutate it correctly later on. * Options are no longer mutated during deployment. Presently there appears to be only one case of this -- when handling `ContinueOnError` in the presence of `IgnoreChanges` (e.g. when performing a refresh). This case has been refactored so that the mutation is no longer necessary. # Notes * This change is in preparation for #16146, where we'd like to add an environment variable to control behaviour and having a single unified `Options` struct would make it easier to pass this configuration down with introducing (more) global state into deployments. Indeed, this change should make it easier to factor global state into `Options` so that it can be controlled and tested more easily/is less susceptible to bugs, race conditions, etc. * I've tweaked/extended some comments while I'm here and have learned things the hard way (e.g. `Refresh` vs `isRefresh`). Feedback welcome on this if we'd rather not conflate. * This change does mean that if in future we wanted e.g. to be able to run a `Deployment` in multiple different ways with multiple sets of actions, we'd have to refactor. Pushing state to the point of object construction reduces the flexibility of the code. However, since we are not presently using that flexibility (nor is there an obvious [to my mind] use case in the near future), this seems like a good trade-off to guard against bugs/make it simpler to move that state around. * I've left some other review comments in the code around questions/changes that might be a bad idea; happy to receive feedback on it all though!
2024-06-11 13:37:57 +00:00
Preview: s.deployment.opts.DryRun,
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
})
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
if upderr != nil {
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if resp.Status != resource.StatusPartialFailure {
return resp.Status, nil, upderr
}
resourceError = upderr
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resourceStatus = resp.Status
if initErr, isInitErr := upderr.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
// Now copy any output state back in case the update triggered cascading updates to other properties.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
s.new.Outputs = resp.Properties
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// UpdateStep doesn't create, but does modify state.
// Change the Modified timestamp.
now := time.Now().UTC()
s.new.Modified = &now
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Finally, mark this operation as complete.
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR ensures that we only run the `StepCompleteFunc` if the step is successful. If not, we should use the step's `Fail` method to register the result. Previously, when a user runs a pulumi up operation with `--continue-on-error`, the engine would register the result twice, once in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342 and if a failure occurs, it is re-registered in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373 I suspect that the double registration is the cause of the user facing error. Since this would only occur for create and update steps, it also explains why the error doesn't occur during deletions. Testing: - Manually ran the repro identified in #16373 and https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614 against a locally built CLI with this fix. - Extended existing lifecycle tests to account for the different code branches of `CreateStep.Apply()` and `UpdateStep.Apply()` Fixes #16373 ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-13 15:42:57 +00:00
if resourceError != nil {
// If we have a failure, we should return an empty complete function
// and let the Fail method handle the registration.
return resourceStatus, nil, resourceError
2018-07-06 22:17:32 +00:00
}
Fix --continue-on-error running indefinitely when a resource fails to be created or updated (#16371) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR ensures that we only run the `StepCompleteFunc` if the step is successful. If not, we should use the step's `Fail` method to register the result. Previously, when a user runs a pulumi up operation with `--continue-on-error`, the engine would register the result twice, once in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L342 and if a failure occurs, it is re-registered in https://github.com/pulumi/pulumi/blob/ee6ec150d8237d472196da9e87265b6b0f24e6af/pkg/resource/deploy/step_executor.go#L373 I suspect that the double registration is the cause of the user facing error. Since this would only occur for create and update steps, it also explains why the error doesn't occur during deletions. Testing: - Manually ran the repro identified in #16373 and https://github.com/pulumi/pulumi-command/issues/435#issuecomment-2114250614 against a locally built CLI with this fix. - Extended existing lifecycle tests to account for the different code branches of `CreateStep.Apply()` and `UpdateStep.Apply()` Fixes #16373 ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-13 15:42:57 +00:00
return resourceStatus, complete, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *UpdateStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *UpdateStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
// ReplaceStep is a logical step indicating a resource will be replaced. This is comprised of three physical steps:
// a creation of the new resource, any number of intervening updates of dependents to the new resource, and then
// a deletion of the now-replaced old resource. This logical step is primarily here for tools and visualization.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type ReplaceStep struct {
deployment *Deployment // the current deployment.
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
old *resource.State // the state of the existing resource.
new *resource.State // the new state snapshot.
keys []resource.PropertyKey // the keys causing replacement.
diffs []resource.PropertyKey // the keys causing a diff.
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
pendingDelete bool // true if a pending deletion should happen.
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*ReplaceStep)(nil)
func NewReplaceStep(deployment *Deployment, old, new *resource.State, keys, diffs []resource.PropertyKey,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
// contract.Assert(new.ID == "")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &ReplaceStep{
deployment: deployment,
old: old,
new: new,
keys: keys,
diffs: diffs,
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
detailedDiff: detailedDiff,
pendingDelete: pendingDelete,
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *ReplaceStep) Op() display.StepOp { return OpReplace }
func (s *ReplaceStep) Deployment() *Deployment { return s.deployment }
func (s *ReplaceStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *ReplaceStep) Provider() string { return s.new.Provider }
func (s *ReplaceStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *ReplaceStep) Old() *resource.State { return s.old }
func (s *ReplaceStep) New() *resource.State { return s.new }
func (s *ReplaceStep) Res() *resource.State { return s.new }
func (s *ReplaceStep) Keys() []resource.PropertyKey { return s.keys }
func (s *ReplaceStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *ReplaceStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *ReplaceStep) Logical() bool { return true }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *ReplaceStep) Apply() (resource.Status, StepCompleteFunc, error) {
// If this is a pending delete, we should have marked the old resource for deletion in the CreateReplacement step.
contract.Assertf(!s.pendingDelete || s.old.Delete,
"old resource %v should be marked for deletion if pending delete", s.old.URN)
return resource.StatusOK, func() {}, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *ReplaceStep) Fail() {
// Nothing to do here.
}
func (s *ReplaceStep) Skip() {
// Nothing to do here.
}
// ReadStep is a step indicating that an existing resources will be "read" and projected into the Pulumi object
// model. Resources that are read are marked with the "External" bit which indicates to the engine that it does
// not own this resource's lifeycle.
//
// A resource with a given URN can transition freely between an "external" state and a non-external state. If
// a URN that was previously marked "External" (i.e. was the target of a ReadStep in a previous deployment) is the
// target of a RegisterResource in the next deployment, a CreateReplacement step will be issued to indicate the
// transition from external to owned. If a URN that was previously not marked "External" is the target of a
// ReadResource in the next deployment, a ReadReplacement step will be issued to indicate the transition from owned to
// external.
type ReadStep struct {
deployment *Deployment // the deployment that produced this read
event ReadResourceEvent // the event that should be signaled upon completion
old *resource.State // the old resource state, if one exists for this urn
new *resource.State // the new resource state, to be used to query the provider
replacing bool // whether or not the new resource is replacing the old resource
provider plugin.Provider // the optional provider to use.
}
// NewReadStep creates a new Read step.
func NewReadStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.External, "new", "must be marked as external")
contract.Requiref(new.Custom, "new", "must be a custom resource")
// If Old was given, it's either an external resource or its ID is equal to the
// ID that we are preparing to read.
if old != nil {
contract.Requiref(old.ID == new.ID || old.External,
"old", "must have the same ID as new or be external")
}
return &ReadStep{
deployment: deployment,
event: event,
old: old,
new: new,
replacing: false,
}
}
// NewReadReplacementStep creates a new Read step with the `replacing` flag set. When executed,
// it will pend deletion of the "old" resource, which must not be an external resource.
func NewReadReplacementStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.External, "new", "must be marked as external")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(!old.External, "old", "must not be marked as external")
return &ReadStep{
deployment: deployment,
event: event,
old: old,
new: new,
replacing: true,
}
}
func (s *ReadStep) Op() display.StepOp {
if s.replacing {
return OpReadReplacement
}
return OpRead
}
func (s *ReadStep) Deployment() *Deployment { return s.deployment }
func (s *ReadStep) Type() tokens.Type { return s.new.Type }
func (s *ReadStep) Provider() string { return s.new.Provider }
func (s *ReadStep) URN() resource.URN { return s.new.URN }
func (s *ReadStep) Old() *resource.State { return s.old }
func (s *ReadStep) New() *resource.State { return s.new }
func (s *ReadStep) Res() *resource.State { return s.new }
func (s *ReadStep) Logical() bool { return !s.replacing }
func (s *ReadStep) Apply() (resource.Status, StepCompleteFunc, error) {
urn := s.new.URN
id := s.new.ID
var resourceError error
resourceStatus := resource.StatusOK
// Unlike most steps, Read steps run during previews. The only time
// we can't run is if the ID we are given is unknown.
if id == plugin.UnknownStringValue {
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
s.new.Outputs = resource.PropertyMap{}
} else {
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
Pass inputs to read as inputs and state (#14683) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Also adds some more checks to the import tests to give confidence that the import machinery was behaving as expected. I noticed this inconsistency of Read when looking at https://github.com/pulumi/pulumi/pull/14678. Comment in the code at the change site, but just to repeat here. I don't think we can stop sending inputs as state because providers have probably come to depend on that, but it's a little more consistent if we pass them as inputs as well. Means in the stackref code we can _just_ look at the inputs set. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-02 14:39:38 +00:00
// Technically the only data we have at this point is "inputs", but we've been passing that as "state" to
// providers since forever and it would probably break things to stop sending that now. Thus this strange double
// send of inputs as both "inputs" and "state". Something to break to tidy up in V4.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
result, err := prov.Read(context.TODO(), plugin.ReadRequest{
URN: urn,
ID: id,
Inputs: s.new.Inputs,
State: s.new.Inputs,
})
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
if err != nil {
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if result.Status != resource.StatusPartialFailure {
return result.Status, nil, err
}
resourceError = err
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resourceStatus = result.Status
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
// If there is no such resource, return an error indicating as such.
if result.Outputs == nil {
return resource.StatusOK, nil, fmt.Errorf("resource '%s' does not exist", id)
}
s.new.Outputs = result.Outputs
if result.ID != "" {
s.new.ID = result.ID
}
}
// If we were asked to replace an existing, non-External resource, pend the
// deletion here.
if s.replacing {
s.old.Delete = true
}
// Propagate timestamps on Read.
if s.old != nil {
s.new.Created = s.old.Created
s.new.Modified = s.old.Modified
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
var inputsChange, outputsChange bool
if s.old != nil {
inputsChange = !s.new.Inputs.DeepEquals(s.old.Inputs)
outputsChange = !s.new.Outputs.DeepEquals(s.old.Outputs)
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Only update the Modified timestamp if read provides new values that differ
// from the old state.
if inputsChange || outputsChange {
now := time.Now().UTC()
s.new.Modified = &now
}
complete := func() { s.event.Done(&ReadResult{State: s.new}) }
if resourceError == nil {
return resourceStatus, complete, nil
}
return resourceStatus, complete, resourceError
}
func (s *ReadStep) Fail() {
s.event.Done(&ReadResult{State: s.new, Result: ResultStateFailed})
}
func (s *ReadStep) Skip() {
s.event.Done(&ReadResult{State: s.new, Result: ResultStateSkipped})
}
// RefreshStep is a step used to track the progress of a refresh operation. A refresh operation updates the an existing
// resource by reading its current state from its provider plugin. These steps are not issued by the step generator;
// instead, they are issued by the deployment executor as the optional first step in deployment execution.
type RefreshStep struct {
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
deployment *Deployment // the deployment that produced this refresh
old *resource.State // the old resource state, if one exists for this urn
new *resource.State // the new resource state, to be used to query the provider
done chan<- bool // the channel to use to signal completion, if any
provider plugin.Provider // the optional provider to use.
diff plugin.DiffResult // the diff between the cloud provider and the state file
}
// NewRefreshStep creates a new Refresh step.
func NewRefreshStep(deployment *Deployment, old *resource.State, done chan<- bool) Step {
contract.Requiref(old != nil, "old", "must not be nil")
// NOTE: we set the new state to the old state by default so that we don't interpret step failures as deletes.
return &RefreshStep{
deployment: deployment,
old: old,
new: old,
done: done,
}
}
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
func (s *RefreshStep) Op() display.StepOp { return OpRefresh }
func (s *RefreshStep) Deployment() *Deployment { return s.deployment }
func (s *RefreshStep) Type() tokens.Type { return s.old.Type }
func (s *RefreshStep) Provider() string { return s.old.Provider }
func (s *RefreshStep) URN() resource.URN { return s.old.URN }
func (s *RefreshStep) Old() *resource.State { return s.old }
func (s *RefreshStep) New() *resource.State { return s.new }
func (s *RefreshStep) Res() *resource.State { return s.old }
func (s *RefreshStep) Logical() bool { return false }
func (s *RefreshStep) Diffs() []resource.PropertyKey { return s.diff.ChangedKeys }
func (s *RefreshStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.diff.DetailedDiff }
// ResultOp returns the operation that corresponds to the change to this resource after reading its current state, if
// any.
func (s *RefreshStep) ResultOp() display.StepOp {
if s.new == nil {
return OpDelete
}
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
Don't call `Diff` when refreshing external resources (#16544) External resources are resources whose state is tracked but not managed. That is, they feature in the state file, but Pulumi does not issue create, read or update operations for them. Typically they arise as the result of generated `.get` functions and methods in Pulumi programs (these should not be confused with specific `getX` functions/methods, which are backed by `Invoke`s and designed to reflect data sources/arbitrary function calls). In #16146, `refresh` operations were modified to use actual bonafide `Diff` calls to compute differences in a manner that would reflect "what will happen on the next `preview`". In general, this aligns better with what users expect when running `refresh`, and fixes a number of issues that have long plagued `refresh`. However, it falls down in the case of external resources. The existing `Diff` contract takes (somewhat strangely) _three_ key inputs: * The old inputs ("what the program used to say") * The old outputs ("what we got back from the last update") * The new inputs ("what the program says now") Intuitively, `Diff`'s job is to return the difference between the old and new inputs. Aside from old outputs being a bit of a third wheel, this contract makes sense for an `update` operation, where the inputs are driving the outputs and changes there will yield changes elsewhere. In a `refresh` operation, however, we have a different situation -- one in which _both_ old and new inputs _and_ outputs are available. Alas, we don't currently have a "four-argument `Diff`" (or a two-argument one we can call twice with no fear of repercussions), and so the usage in `refresh` is a little bit of a clever hack that ends up biasing inputs over outputs (see #16146 or the implementation for details). For external resources, the old behaviour (where we just compared outputs) seems like a better fit. This commit therefore adds this special case and a test to ensure that when we see external resources, we simply compare outputs and don't call `Diff`. Fixes #16401 Fixes #16502 --------- Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
// There are two cases in which we'll diff only resource outputs on a
// refresh:
//
// * The resource is external, that is, it is not managed by Pulumi.
// In these cases, we care somewhat equally about inputs and outputs, but
// the Diff contract we currently support forces us to bias one side
// (typically inputs). Moreover, some providers might not support
// diff/handle diffing correctly for external resources. This can lead to
// surprising results, so for now we sidestep the issue by only looking at
// outputs.
// * The user has explicitly opted into this legacy behaviour by setting
// the `UseLegacyRefreshDiff` option to true.
if s.old.External || s.deployment.opts.UseLegacyRefreshDiff {
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
if s.new == s.old || s.old.Outputs.Diff(s.new.Outputs) == nil {
return OpSame
}
} else {
if s.new == s.old || s.diff.Changes == plugin.DiffNone {
return OpSame
}
}
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
return OpUpdate
}
func (s *RefreshStep) Apply() (resource.Status, StepCompleteFunc, error) {
var complete func()
if s.done != nil {
complete = func() { close(s.done) }
}
resourceID := s.old.ID
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
// Component, provider, and pending-replace resources never change with a refresh; just return the current state.
if !s.old.Custom || providers.IsProviderType(s.old.Type) || s.old.PendingReplacement {
return resource.StatusOK, complete, nil
}
// For a custom resource, fetch the resource's provider and read the resource's current state.
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
var initErrors []string
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
refreshed, err := prov.Read(context.TODO(), plugin.ReadRequest{
URN: s.old.URN,
ID: resourceID,
Inputs: s.old.Inputs,
State: s.old.Outputs,
})
if err != nil {
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if refreshed.Status != resource.StatusPartialFailure {
return refreshed.Status, nil, err
}
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
initErrors = initErr.Reasons
// Partial failure SHOULD NOT cause refresh to fail. Instead:
//
// 1. Warn instead that during refresh we noticed the resource has become unhealthy.
// 2. Make sure the initialization errors are persisted in the state, so that the next
// `pulumi up` will surface them to the user.
err = nil
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
msg := "Refreshed resource is in an unhealthy state:\n* " + strings.Join(initErrors, "\n* ")
s.Deployment().Diag().Warningf(diag.RawMessage(s.URN(), msg))
}
}
outputs := refreshed.Outputs
// If the provider specified new inputs for this resource, pick them up now. Otherwise, retain the current inputs.
inputs := s.old.Inputs
if refreshed.Inputs != nil {
inputs = refreshed.Inputs
}
if outputs != nil {
// There is a chance that the ID has changed. We want to allow this change to happen
// it will have changed already in the outputs, but we need to persist this change
// at a state level because the Id
if refreshed.ID != "" && refreshed.ID != resourceID {
logging.V(7).Infof("Refreshing ID; oldId=%s, newId=%s", resourceID, refreshed.ID)
resourceID = refreshed.ID
}
s.new = resource.NewState(s.old.Type, s.old.URN, s.old.Custom, s.old.Delete, resourceID, inputs, outputs,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
s.old.Parent, s.old.Protect, s.old.External, s.old.Dependencies, initErrors, s.old.Provider,
s.old.PropertyDependencies, s.old.PendingReplacement, s.old.AdditionalSecretOutputs, s.old.Aliases,
[engine] Add support for source positions These changes add support for passing source position information in gRPC metadata and recording the source position that corresponds to a resource registration in the statefile. Enabling source position information in the resource model can provide substantial benefits, including but not limited to: - Better errors from the Pulumi CLI - Go-to-defintion for resources in state - Editor integration for errors, etc. from `pulumi preview` Source positions are (file, line) or (file, line, column) tuples represented as URIs. The line and column are stored in the fragment portion of the URI as "line(,column)?". The scheme of the URI and the form of its path component depends on the context in which it is generated or used: - During an active update, the URI's scheme is `file` and paths are absolute filesystem paths. This allows consumers to easily access arbitrary files that are available on the host. - In a statefile, the URI's scheme is `project` and paths are relative to the project root. This allows consumers to resolve source positions relative to the project file in different contexts irrespective of the location of the project itself (e.g. given a project-relative path and the URL of the project's root on GitHub, one can build a GitHub URL for the source position). During an update, source position information may be attached to gRPC calls as "source-position" metadata. This allows arbitrary calls to be associated with source positions without changes to their protobuf payloads. Modifying the protobuf payloads is also a viable approach, but is somewhat more invasive than attaching metadata, and requires changes to every call signature. Source positions should reflect the position in user code that initiated a resource model operation (e.g. the source position passed with `RegisterResource` for `pet` in the example above should be the source position in `index.ts`, _not_ the source position in the Pulumi SDK). In general, the Pulumi SDK should be able to infer the source position of the resource registration, as the relationship between a resource registration and its corresponding user code should be static per SDK. Source positions in state files will be stored as a new `registeredAt` property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
&s.old.CustomTimeouts, s.old.ImportID, s.old.RetainOnDelete, s.old.DeletedWith, s.old.Created, s.old.Modified,
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
s.old.SourcePosition, s.old.IgnoreChanges,
[engine] Add support for source positions These changes add support for passing source position information in gRPC metadata and recording the source position that corresponds to a resource registration in the statefile. Enabling source position information in the resource model can provide substantial benefits, including but not limited to: - Better errors from the Pulumi CLI - Go-to-defintion for resources in state - Editor integration for errors, etc. from `pulumi preview` Source positions are (file, line) or (file, line, column) tuples represented as URIs. The line and column are stored in the fragment portion of the URI as "line(,column)?". The scheme of the URI and the form of its path component depends on the context in which it is generated or used: - During an active update, the URI's scheme is `file` and paths are absolute filesystem paths. This allows consumers to easily access arbitrary files that are available on the host. - In a statefile, the URI's scheme is `project` and paths are relative to the project root. This allows consumers to resolve source positions relative to the project file in different contexts irrespective of the location of the project itself (e.g. given a project-relative path and the URL of the project's root on GitHub, one can build a GitHub URL for the source position). During an update, source position information may be attached to gRPC calls as "source-position" metadata. This allows arbitrary calls to be associated with source positions without changes to their protobuf payloads. Modifying the protobuf payloads is also a viable approach, but is somewhat more invasive than attaching metadata, and requires changes to every call signature. Source positions should reflect the position in user code that initiated a resource model operation (e.g. the source position passed with `RegisterResource` for `pet` in the example above should be the source position in `index.ts`, _not_ the source position in the Pulumi SDK). In general, the Pulumi SDK should be able to infer the source position of the resource registration, as the relationship between a resource registration and its corresponding user code should be static per SDK. Source positions in state files will be stored as a new `registeredAt` property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
)
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
var inputsChange, outputsChange bool
if s.old != nil {
Don't call `Diff` when refreshing external resources (#16544) External resources are resources whose state is tracked but not managed. That is, they feature in the state file, but Pulumi does not issue create, read or update operations for them. Typically they arise as the result of generated `.get` functions and methods in Pulumi programs (these should not be confused with specific `getX` functions/methods, which are backed by `Invoke`s and designed to reflect data sources/arbitrary function calls). In #16146, `refresh` operations were modified to use actual bonafide `Diff` calls to compute differences in a manner that would reflect "what will happen on the next `preview`". In general, this aligns better with what users expect when running `refresh`, and fixes a number of issues that have long plagued `refresh`. However, it falls down in the case of external resources. The existing `Diff` contract takes (somewhat strangely) _three_ key inputs: * The old inputs ("what the program used to say") * The old outputs ("what we got back from the last update") * The new inputs ("what the program says now") Intuitively, `Diff`'s job is to return the difference between the old and new inputs. Aside from old outputs being a bit of a third wheel, this contract makes sense for an `update` operation, where the inputs are driving the outputs and changes there will yield changes elsewhere. In a `refresh` operation, however, we have a different situation -- one in which _both_ old and new inputs _and_ outputs are available. Alas, we don't currently have a "four-argument `Diff`" (or a two-argument one we can call twice with no fear of repercussions), and so the usage in `refresh` is a little bit of a clever hack that ends up biasing inputs over outputs (see #16146 or the implementation for details). For external resources, the old behaviour (where we just compared outputs) seems like a better fit. This commit therefore adds this special case and a test to ensure that when we see external resources, we simply compare outputs and don't call `Diff`. Fixes #16401 Fixes #16502 --------- Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
// There are two cases in which we'll diff only resource outputs on a
// refresh:
//
// * The resource is external, that is, it is not managed by Pulumi.
// In these cases, we care somewhat equally about inputs and outputs, but
// the Diff contract we currently support forces us to bias one side
// (typically inputs). Moreover, some providers might not support
// diff/handle diffing correctly for external resources. This can lead to
// surprising results, so for now we sidestep the issue by only looking at
// outputs.
// * The user has explicitly opted into this legacy behaviour by setting
// the `UseLegacyRefreshDiff` option to true.
if s.old.External || s.deployment.opts.UseLegacyRefreshDiff {
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
inputsChange = !refreshed.Inputs.DeepEquals(s.old.Inputs)
outputsChange = !refreshed.Outputs.DeepEquals(s.old.Outputs)
} else {
inputsChange = !inputs.DeepEquals(s.old.Inputs)
outputsChange = !outputs.DeepEquals(s.old.Outputs)
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Only update the Modified timestamp if refresh provides new values that differ
// from the old state.
if inputsChange || outputsChange {
// The refresh has identified an incongruence between the provider and state
// updated the Modified timestamp to track this.
now := time.Now().UTC()
s.new.Modified = &now
}
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
Don't call `Diff` when refreshing external resources (#16544) External resources are resources whose state is tracked but not managed. That is, they feature in the state file, but Pulumi does not issue create, read or update operations for them. Typically they arise as the result of generated `.get` functions and methods in Pulumi programs (these should not be confused with specific `getX` functions/methods, which are backed by `Invoke`s and designed to reflect data sources/arbitrary function calls). In #16146, `refresh` operations were modified to use actual bonafide `Diff` calls to compute differences in a manner that would reflect "what will happen on the next `preview`". In general, this aligns better with what users expect when running `refresh`, and fixes a number of issues that have long plagued `refresh`. However, it falls down in the case of external resources. The existing `Diff` contract takes (somewhat strangely) _three_ key inputs: * The old inputs ("what the program used to say") * The old outputs ("what we got back from the last update") * The new inputs ("what the program says now") Intuitively, `Diff`'s job is to return the difference between the old and new inputs. Aside from old outputs being a bit of a third wheel, this contract makes sense for an `update` operation, where the inputs are driving the outputs and changes there will yield changes elsewhere. In a `refresh` operation, however, we have a different situation -- one in which _both_ old and new inputs _and_ outputs are available. Alas, we don't currently have a "four-argument `Diff`" (or a two-argument one we can call twice with no fear of repercussions), and so the usage in `refresh` is a little bit of a clever hack that ends up biasing inputs over outputs (see #16146 or the implementation for details). For external resources, the old behaviour (where we just compared outputs) seems like a better fit. This commit therefore adds this special case and a test to ensure that when we see external resources, we simply compare outputs and don't call `Diff`. Fixes #16401 Fixes #16502 --------- Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
if s.old.External {
logging.V(7).Infof("External resource %s; diffing outputs only", s.URN())
} else if s.deployment.opts.UseLegacyRefreshDiff {
logging.V(7).Infof("Refresh diffing disabled; diffing outputs only (%s)", s.URN())
} else {
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
// To compute refresh diffs against the desired state, we compute the diff
// that a user would see if they immediately ran an `up` operation on a
// no-change program after this refresh. However, this will return the
// _opposite_ of what we want, since the `up`'s diff is framed in terms of
// the program being the source of truth (not the provider). That is, we
// want to show the user what changes are coming from the outputs into the
// inputs, not what changes are coming from the inputs into the outputs!
// We thus invert the diff (changing adds to deletes, and so on) before
// storing it against the step.
//
// Note that to compute the diff in this manner, we pass:
//
// * newInputs where oldInputs are expected
// * newOutputs where oldOutputs are expected
// * oldInputs where newInputs are expected
diff, err := diffResource(
s.new.URN, s.new.ID,
// pass new inputs/outputs as old inputs/outputs
s.new.Inputs, s.new.Outputs,
// pass old inputs as new inputs
s.old.Inputs,
prov, s.deployment.opts.DryRun, s.old.IgnoreChanges,
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
)
if err != nil {
return refreshed.Status, nil, err
}
s.diff = diff.Invert()
Don't call `Diff` when refreshing external resources (#16544) External resources are resources whose state is tracked but not managed. That is, they feature in the state file, but Pulumi does not issue create, read or update operations for them. Typically they arise as the result of generated `.get` functions and methods in Pulumi programs (these should not be confused with specific `getX` functions/methods, which are backed by `Invoke`s and designed to reflect data sources/arbitrary function calls). In #16146, `refresh` operations were modified to use actual bonafide `Diff` calls to compute differences in a manner that would reflect "what will happen on the next `preview`". In general, this aligns better with what users expect when running `refresh`, and fixes a number of issues that have long plagued `refresh`. However, it falls down in the case of external resources. The existing `Diff` contract takes (somewhat strangely) _three_ key inputs: * The old inputs ("what the program used to say") * The old outputs ("what we got back from the last update") * The new inputs ("what the program says now") Intuitively, `Diff`'s job is to return the difference between the old and new inputs. Aside from old outputs being a bit of a third wheel, this contract makes sense for an `update` operation, where the inputs are driving the outputs and changes there will yield changes elsewhere. In a `refresh` operation, however, we have a different situation -- one in which _both_ old and new inputs _and_ outputs are available. Alas, we don't currently have a "four-argument `Diff`" (or a two-argument one we can call twice with no fear of repercussions), and so the usage in `refresh` is a little bit of a clever hack that ends up biasing inputs over outputs (see #16146 or the implementation for details). For external resources, the old behaviour (where we just compared outputs) seems like a better fit. This commit therefore adds this special case and a test to ensure that when we see external resources, we simply compare outputs and don't call `Diff`. Fixes #16401 Fixes #16502 --------- Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-02 10:11:10 +00:00
logging.V(7).Infof("Refresh diff for %s: %v", s.URN(), s.diff)
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
}
} else {
s.new = nil
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
return refreshed.Status, nil, err
}
func (s *RefreshStep) Fail() {
// Nothing to do here.
}
func (s *RefreshStep) Skip() {
// Nothing to do here.
}
type ImportStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
original *resource.State // the original resource, if this is an import-replace.
old *resource.State // the state of the resource fetched from the provider.
new *resource.State // the newly computed state of the resource after importing.
replacing bool // true if we are replacing a Pulumi-managed resource.
planned bool // true if this import is from an import deployment.
diffs []resource.PropertyKey // any keys that differed between the user's program and the actual state.
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
ignoreChanges []string // a list of property paths to ignore when updating.
randomSeed []byte // the random seed to use for Check.
provider plugin.Provider // the optional provider to use.
}
func NewImportStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string, randomSeed []byte,
) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: reg,
new: new,
ignoreChanges: ignoreChanges,
randomSeed: randomSeed,
}
}
func NewImportReplacementStep(deployment *Deployment, reg RegisterResourceEvent, original, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string, randomSeed []byte,
) Step {
contract.Requiref(original != nil, "original", "must not be nil")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: reg,
original: original,
new: new,
replacing: true,
ignoreChanges: ignoreChanges,
randomSeed: randomSeed,
}
}
func newImportDeploymentStep(deployment *Deployment, new *resource.State, randomSeed []byte) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
contract.Requiref(!new.Custom || new.ID != "", "new", "must have an ID")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
contract.Requiref(!new.Custom || randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: noopEvent(0),
new: new,
planned: true,
randomSeed: randomSeed,
}
}
func (s *ImportStep) Op() display.StepOp {
if s.replacing {
return OpImportReplacement
}
return OpImport
}
func (s *ImportStep) Deployment() *Deployment { return s.deployment }
func (s *ImportStep) Type() tokens.Type { return s.new.Type }
func (s *ImportStep) Provider() string { return s.new.Provider }
func (s *ImportStep) URN() resource.URN { return s.new.URN }
func (s *ImportStep) Old() *resource.State { return s.old }
func (s *ImportStep) New() *resource.State { return s.new }
func (s *ImportStep) Res() *resource.State { return s.new }
func (s *ImportStep) Logical() bool { return !s.replacing }
func (s *ImportStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *ImportStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *ImportStep) Apply() (resource.Status, StepCompleteFunc, error) {
complete := func() {
s.reg.Done(&RegisterResult{State: s.new})
}
// If this is a planned import, ensure that the resource does not exist in the old state file.
if s.planned {
if _, ok := s.deployment.olds[s.new.URN]; ok {
return resource.StatusOK, nil, fmt.Errorf("resource '%v' already exists", s.new.URN)
}
if s.new.Parent.QualifiedType() != resource.RootStackType {
_, ok := s.deployment.news.Load(s.new.Parent)
Allow importing a parent and child resource at the same time (#14461) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Allow `pulumi import` to import one resource and then use that resource as a parent for another imported resource. Currently parents can only refer to resources that already exist in the deployment meaning if you want to do this today you have to do two imports. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-10 13:31:11 +00:00
if !ok {
return resource.StatusOK, nil, fmt.Errorf("unknown parent '%v' for resource '%v'",
s.new.Parent, s.new.URN)
}
}
}
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
// Only need to do anything here for custom resources, components just import as empty
inputs := resource.PropertyMap{}
outputs := resource.PropertyMap{}
var prov plugin.Provider
rst := resource.StatusOK
if s.new.Custom {
// Read the current state of the resource to import. If the provider does not hand us back any inputs for the
// resource, it probably needs to be updated. If the resource does not exist at all, fail the import.
var err error
prov, err = getProvider(s, s.provider)
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
if err != nil {
return resource.StatusOK, nil, err
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
read, err := prov.Read(context.TODO(), plugin.ReadRequest{
URN: s.new.URN,
ID: s.new.ID,
})
rst = read.Status
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
if err != nil {
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
} else {
return rst, nil, err
}
}
if read.Outputs == nil {
return rst, nil, fmt.Errorf("resource '%v' does not exist", s.new.ID)
}
if read.Inputs == nil {
return resource.StatusOK, nil,
fmt.Errorf("provider does not support importing resources; please try updating the '%v' plugin",
s.new.URN.Type().Package())
}
if read.ID != "" {
s.new.ID = read.ID
}
inputs = read.Inputs
outputs = read.Outputs
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
} else {
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
}
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
s.new.Outputs = outputs
// Magic up an old state so the frontend can display a proper diff. This state is the output of the just-executed
// `Read` combined with the resource identity and metadata from the desired state. This ensures that the only
// differences between the old and new states are between the inputs and outputs.
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
s.old = resource.NewState(s.new.Type, s.new.URN, s.new.Custom, false, s.new.ID, inputs, outputs,
s.new.Parent, s.new.Protect, false, s.new.Dependencies, s.new.InitErrors, s.new.Provider,
s.new.PropertyDependencies, false, nil, nil, &s.new.CustomTimeouts, s.new.ImportID, s.new.RetainOnDelete,
Change `pulumi refresh` to report diff relative to desired state instead of relative to only output changes (#16146) Presently, the behaviour of diffing during refresh steps is incomplete, returning only an "output diff" that presents the changes in outputs. This commit changes refresh steps so that: * they compute a diff similar to the one that would be computed if a `preview` were run immediately after the refresh, which is more typically what users expect and want; and * `IgnoreChanges` resource options are respected when performing the new desired-state diffs, so that property additions or changes reported by a refresh can be ignored. In particular, `IgnoreChanges` can now be used to acknowledge that part or all of a resource may change in the provider, but the user is OK with this and doesn't want to be notified about it during a refresh. Importantly, this means that the diff won't be reported, but also that the changes won't be applied to state. The implementation covers the following: * A diff is computed using the inputs from the program and then inverting the result, since in the case of a refresh the diff is being driven by the provider side and not the program. This doesn't change what is stored back into the state, but it does produce a diff that is more aligned with the "true changes to the desired state". * `IgnoreChanges` resource options are now stored in state, so that this information can be used in refresh operations that do not have access to/run the program. * In the context of a refresh operation, `IgnoreChanges` applies to *both* input and output properties. This differs from the behaviour of a normal update operation, where `IgnoreChanges` only considers input properties. * The special `"*"` value for `IgnoreChanges` can be used to ignore all properties. It _also_ ignores the case where the resource cannot be found in the provider, and instead keeps the resource intact in state with its existing input and output properties. Because the program is not run for refresh operations, `IgnoreChanges` options must be applied separately before a refresh takes place. This can be accomplished using e.g. a `pulumi up` that applies the options prior to a refresh. We should investigate perhaps providing a `pulumi state set ...`-like CLI to make these sorts of changes directly to a state. For use cases relying on the legacy refresh diff provider, the `PULUMI_USE_LEGACY_REFRESH_DIFF` environment variable can be set, which will disable desired-state diff computation. We only need to perform checks in `RefreshStep.{ResultOp,Apply}`, since downstream code will work correctly based on the presence or absence of a `DetailedDiff` in the step. ### Notes - https://github.com/pulumi/pulumi/issues/16144 affects some of these cases - though its technically orthogonal - https://github.com/pulumi/pulumi/issues/11279 is another technically orthogonal issue that many providers (at least TFBridge ones) - do not report back changes to input properties on Read when the input property (or property path) was missing on the inputs. This is again technically orthogonal - but leads to cases that appear "wrong" in terms of what is stored back into the state still - though the same as before this change. - Azure Native doesn't seem to handle `ignoreChanges` passed to Diff, so the ability to ignore changes on refresh doesn't currently work for Azure Native. ### Fixes * Fixes #16072 * Fixes #16278 * Fixes #16334 * Not quite #12346, but likely replaces the need for that Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-06-12 16:17:05 +00:00
s.new.DeletedWith, nil, nil, s.new.SourcePosition, s.new.IgnoreChanges)
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Import takes a resource that Pulumi did not create and imports it into pulumi state.
now := time.Now().UTC()
s.new.Modified = &now
// Set Created to now as the resource has been created in the state.
s.new.Created = &now
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
// If this is a component we don't need to do the rest of the input validation
if !s.new.Custom {
return rst, complete, nil
}
// If this step came from an import deployment, we need to fetch any required inputs from the state.
if s.planned {
contract.Assertf(len(s.new.Inputs) == 0, "import resource cannot have existing inputs")
// Get the import object and see if it had properties set
var inputProperties []string
for _, imp := range s.deployment.imports {
if imp.ID == s.old.ID {
inputProperties = imp.Properties
break
}
}
if len(inputProperties) == 0 {
logging.V(9).Infof("Importing %v with all properties", s.URN())
s.new.Inputs = s.old.Inputs.Copy()
} else {
logging.V(9).Infof("Importing %v with supplied properties: %v", s.URN(), inputProperties)
for _, p := range inputProperties {
k := resource.PropertyKey(p)
if value, has := s.old.Inputs[k]; has {
s.new.Inputs[k] = value
}
}
}
// Check the provider inputs for consistency. If the inputs fail validation, the import will still succeed, but
// we will display the validation failures and a message informing the user that the failures are almost
// definitely a provider bug.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resp, err := prov.Check(context.TODO(), plugin.CheckRequest{
URN: s.new.URN,
Olds: s.old.Inputs,
News: s.new.Inputs,
AllowUnknowns: s.deployment.opts.DryRun,
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
RandomSeed: s.randomSeed,
})
if err != nil {
return rst, nil, err
}
// Print this warning before printing all the check failures to give better context.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if len(resp.Failures) != 0 {
// Based on if the user passed 'properties' or not we want to change the error message here.
var errorMessage string
if len(inputProperties) == 0 {
ref, err := providers.ParseReference(s.Provider())
contract.AssertNoErrorf(err, "failed to parse provider reference %q", s.Provider())
pkgName := ref.URN().Type().Name()
errorMessage = fmt.Sprintf("This is almost certainly a bug in the `%s` provider.", pkgName)
} else {
errorMessage = "Try specifying a different set of properties to import with in the future."
}
s.deployment.Diag().Warningf(diag.Message(s.new.URN,
"One or more imported inputs failed to validate. %s "+
"The import will still proceed, but you will need to edit the generated code after copying it into your program."),
errorMessage)
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
issueCheckFailures(s.deployment.Diag().Warningf, s.new, s.new.URN, resp.Failures)
s.diffs, s.detailedDiff = []resource.PropertyKey{}, map[string]plugin.PropertyDiff{}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
return rst, complete, nil
}
// Set inputs back to their old values (if any) for any "ignored" properties
processedInputs, err := processIgnoreChanges(s.new.Inputs, s.old.Inputs, s.ignoreChanges)
if err != nil {
return resource.StatusOK, nil, err
}
s.new.Inputs = processedInputs
// Check the inputs using the provider inputs for defaults.
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
resp, err := prov.Check(context.TODO(), plugin.CheckRequest{
URN: s.new.URN,
Olds: s.old.Inputs,
News: s.new.Inputs,
AllowUnknowns: s.deployment.opts.DryRun,
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
RandomSeed: s.randomSeed,
})
if err != nil {
return rst, nil, err
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
if issueCheckErrors(s.deployment, s.new, s.new.URN, resp.Failures) {
return rst, nil, errors.New("one or more inputs failed to validate")
}
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) (#16302) Normalize methods on plugin.Provider to the form: ```go Method(context.Context, MethodRequest) (MethodResponse, error) ``` This provides a more consistent and forwards compatible interface for each of our methods. --- I'm motivated to work on this because the bridge maintains a copy of this interface: `ProviderWithContext`. This doubles the pain of dealing with any breaking change and this PR would allow me to remove the extra interface. I'm willing to fix consumers of `plugin.Provider` in `pulumi/pulumi`, but I wanted to make sure that we would be willing to merge this PR if I get it green. <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes # (issue) ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-06-07 19:47:49 +00:00
s.new.Inputs = resp.Properties
// Diff the user inputs against the provider inputs. If there are any differences, fail the import unless this step
// is from an import deployment.
diff, err := diffResource(
s.new.URN, s.new.ID,
s.old.Inputs, s.old.Outputs,
s.new.Inputs,
prov,
s.deployment.opts.DryRun,
s.ignoreChanges,
)
if err != nil {
return rst, nil, err
}
s.diffs, s.detailedDiff = diff.ChangedKeys, diff.DetailedDiff
if diff.Changes != plugin.DiffNone {
const message = "inputs to import do not match the existing resource"
if s.deployment.opts.DryRun {
s.deployment.ctx.Diag.Warningf(diag.StreamMessage(s.new.URN,
message+"; importing this resource will fail", 0))
} else {
err = errors.New(message)
}
}
// If we were asked to replace an existing, non-External resource, pend the deletion here.
if err == nil && s.replacing {
s.original.Delete = true
}
if err != nil {
return rst, nil, err
}
return rst, complete, nil
}
func (s *ImportStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *ImportStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
const (
OpSame display.StepOp = "same" // nothing to do.
OpCreate display.StepOp = "create" // creating a new resource.
OpUpdate display.StepOp = "update" // updating an existing resource.
OpDelete display.StepOp = "delete" // deleting an existing resource.
OpReplace display.StepOp = "replace" // replacing a resource with a new one.
OpCreateReplacement display.StepOp = "create-replacement" // creating a new resource for a replacement.
OpDeleteReplaced display.StepOp = "delete-replaced" // deleting an existing resource after replacement.
OpRead display.StepOp = "read" // reading an existing resource.
OpReadReplacement display.StepOp = "read-replacement" // reading an existing resource for a replacement.
OpRefresh display.StepOp = "refresh" // refreshing an existing resource.
OpReadDiscard display.StepOp = "discard" // removing a resource that was read.
OpDiscardReplaced display.StepOp = "discard-replaced" // discarding a read resource that was replaced.
OpRemovePendingReplace display.StepOp = "remove-pending-replace" // removing a pending replace resource.
OpImport display.StepOp = "import" // import an existing resource.
OpImportReplacement display.StepOp = "import-replacement" // replace an existing resource
// with an imported resource.
)
// StepOps contains the full set of step operation types.
var StepOps = []display.StepOp{
OpSame,
OpCreate,
OpUpdate,
OpDelete,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
OpReplace,
OpCreateReplacement,
OpDeleteReplaced,
OpRead,
OpReadReplacement,
OpRefresh,
OpReadDiscard,
OpDiscardReplaced,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
OpRemovePendingReplace,
OpImport,
OpImportReplacement,
}
func IsReplacementStep(op display.StepOp) bool {
if op == OpReplace || op == OpCreateReplacement || op == OpDeleteReplaced ||
op == OpReadReplacement || op == OpDiscardReplaced || op == OpRemovePendingReplace ||
op == OpImportReplacement {
return true
}
return false
}
// Color returns a suggested color for lines of this op type.
func Color(op display.StepOp) string {
switch op {
case OpSame:
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
return colors.SpecUnimportant
case OpCreate, OpImport:
return colors.SpecCreate
case OpDelete:
return colors.SpecDelete
case OpUpdate:
return colors.SpecUpdate
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
case OpReplace:
return colors.SpecReplace
case OpCreateReplacement:
return colors.SpecCreateReplacement
case OpDeleteReplaced:
return colors.SpecDeleteReplaced
case OpRead:
return colors.SpecRead
case OpReadReplacement, OpImportReplacement:
return colors.SpecReplace
case OpRefresh:
return colors.SpecUpdate
case OpReadDiscard, OpDiscardReplaced:
return colors.SpecDelete
Don't re-delete resources that are `PendingReplacement` (#16510) As well as indicating that a resource's state has changes, a diff can also indicate that those changes require the _replacement_ of the resource, meaning that it must be recreated and not just updated. In this scenario, there are two possible ways to replace the resource -- by first creating another new resource before deleting the old one ("create-before-replace"), or by first deleting the old resource before creating its replacement ("delete-before-replace"). Create-before-replace is the default since generally, if possible to implement, it should result in fewer instances of "downtime", where a desired resource does not exist in the system. Should delete-before-replace be chosen, Pulumi implements this under the hood as three steps: delete for replacement, replace, and create replacement. To track things consistently, as well as enable resumption of an interrupted operation, Pulumi writes a flag, `PendingReplacement` to the state of a deleted resource that will later be cleaned up by a completed replacement. Should an interrupted operation be resumed, Pulumi does not currently take `PendingReplacement` into account, and always enqueues a(nother) delete operation. This is typically fine (albeit wasteful) since deletes are (should) be idempotent, but unnecessary. This commit adds @jesse-triplewhale's fix for this behaviour whereby the `PendingReplacement` flag is simply removed before the remainder of the required steps (replace, create replacement) are actioned as normal. It also extends this work with some lifecycle tests for this scenario and a few others that may arise as a result of an interrupted replacement. Fixes #16288 Closes #16303 Co-authored-by: Jesse Grodman <jesse@triplewhale.com>
2024-06-28 23:16:20 +00:00
case OpRemovePendingReplace:
return colors.SpecUnimportant
default:
contract.Failf("Unrecognized resource step op: '%v'", op)
return ""
}
}
// ColorProgress returns a suggested coloring for lines of this of type which
// are progressing.
func ColorProgress(op display.StepOp) string {
return colors.Bold + Color(op)
}
// Prefix returns a suggested prefix for lines of this op type.
func Prefix(op display.StepOp, done bool) string {
var color string
if done {
color = Color(op)
} else {
color = ColorProgress(op)
}
return color + RawPrefix(op)
}
// RawPrefix returns the uncolorized prefix text.
func RawPrefix(op display.StepOp) string {
switch op {
case OpSame:
return " "
case OpCreate:
return "+ "
case OpDelete:
return "- "
case OpUpdate:
return "~ "
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
case OpReplace:
return "+-"
case OpCreateReplacement:
return "++"
case OpDeleteReplaced:
return "--"
case OpRead:
return "> "
case OpReadReplacement:
return ">>"
case OpRefresh:
return "~ "
case OpReadDiscard:
return "< "
case OpDiscardReplaced:
return "<<"
case OpImport:
return "= "
case OpImportReplacement:
return "=>"
Don't re-delete resources that are `PendingReplacement` (#16510) As well as indicating that a resource's state has changes, a diff can also indicate that those changes require the _replacement_ of the resource, meaning that it must be recreated and not just updated. In this scenario, there are two possible ways to replace the resource -- by first creating another new resource before deleting the old one ("create-before-replace"), or by first deleting the old resource before creating its replacement ("delete-before-replace"). Create-before-replace is the default since generally, if possible to implement, it should result in fewer instances of "downtime", where a desired resource does not exist in the system. Should delete-before-replace be chosen, Pulumi implements this under the hood as three steps: delete for replacement, replace, and create replacement. To track things consistently, as well as enable resumption of an interrupted operation, Pulumi writes a flag, `PendingReplacement` to the state of a deleted resource that will later be cleaned up by a completed replacement. Should an interrupted operation be resumed, Pulumi does not currently take `PendingReplacement` into account, and always enqueues a(nother) delete operation. This is typically fine (albeit wasteful) since deletes are (should) be idempotent, but unnecessary. This commit adds @jesse-triplewhale's fix for this behaviour whereby the `PendingReplacement` flag is simply removed before the remainder of the required steps (replace, create replacement) are actioned as normal. It also extends this work with some lifecycle tests for this scenario and a few others that may arise as a result of an interrupted replacement. Fixes #16288 Closes #16303 Co-authored-by: Jesse Grodman <jesse@triplewhale.com>
2024-06-28 23:16:20 +00:00
case OpRemovePendingReplace:
return "~ "
default:
contract.Failf("Unrecognized resource step op: %v", op)
return ""
}
}
func PastTense(op display.StepOp) string {
switch op {
case OpSame, OpCreate, OpReplace, OpCreateReplacement, OpUpdate, OpReadReplacement:
return string(op) + "d"
case OpRefresh:
return "refreshed"
case OpRead:
return "read"
case OpReadDiscard, OpDiscardReplaced:
return "discarded"
case OpDelete, OpDeleteReplaced:
return "deleted"
case OpImport, OpImportReplacement:
return "imported"
default:
2017-11-29 18:06:51 +00:00
contract.Failf("Unexpected resource step op: %v", op)
return ""
}
}
// Suffix returns a suggested suffix for lines of this op type.
func Suffix(op display.StepOp) string {
switch op {
case OpCreateReplacement, OpUpdate, OpReplace, OpReadReplacement, OpRefresh, OpImportReplacement:
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return colors.Reset // updates and replacements colorize individual lines; get has none
}
return ""
}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
// ConstrainedTo returns true if this operation is no more impactful than the constraint.
func ConstrainedTo(op display.StepOp, constraint display.StepOp) bool {
var allowed []display.StepOp
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
switch constraint {
case OpSame, OpDelete, OpRead, OpReadReplacement, OpRefresh, OpReadDiscard, OpDiscardReplaced,
OpRemovePendingReplace, OpImport, OpImportReplacement:
allowed = []display.StepOp{constraint}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpCreate:
allowed = []display.StepOp{OpSame, OpCreate}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpUpdate:
allowed = []display.StepOp{OpSame, OpUpdate}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpReplace, OpCreateReplacement, OpDeleteReplaced:
allowed = []display.StepOp{OpSame, OpUpdate, constraint}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
for _, candidate := range allowed {
if candidate == op {
return true
}
}
return false
}
// getProvider fetches the provider for the given step.
func getProvider(s Step, override plugin.Provider) (plugin.Provider, error) {
if override != nil {
return override, nil
}
if providers.IsProviderType(s.Type()) {
return s.Deployment().providers, nil
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
ref, err := providers.ParseReference(s.Provider())
if err != nil {
return nil, fmt.Errorf("bad provider reference '%v' for resource %v: %w", s.Provider(), s.URN(), err)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
if providers.IsDenyDefaultsProvider(ref) {
pkg := providers.GetDeniedDefaultProviderPkg(ref)
msg := diag.GetDefaultProviderDenied(s.URN()).Message
return nil, fmt.Errorf(msg, pkg, s.URN())
}
provider, ok := s.Deployment().GetProvider(ref)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
if !ok {
return nil, fmt.Errorf("unknown provider '%v' for resource %v", s.Provider(), s.URN())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
return provider, nil
}