pulumi/pkg/resource/deploy/step.go

1501 lines
57 KiB
Go
Raw Normal View History

Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
// Copyright 2016-2023, Pulumi Corporation.
2018-05-22 19:43:36 +00:00
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package deploy
import (
"errors"
"fmt"
"strings"
"time"
"github.com/pulumi/pulumi/pkg/v3/display"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/providers"
"github.com/pulumi/pulumi/sdk/v3/go/common/diag"
"github.com/pulumi/pulumi/sdk/v3/go/common/diag/colors"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
)
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// StepCompleteFunc is the type of functions returned from Step.Apply. These functions are to be called when the engine
// has fully retired a step. You _should not_ modify the resource state in these functions, that will race with the
// snapshot writing code.
type StepCompleteFunc func()
// Step is a specification for a deployment operation.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type Step interface {
// Apply applies or previews this step. It returns the status of the resource after the step application,
// a function to call to signal that this step has fully completed, and an error, if one occurred while applying
// the step.
//
// The returned StepCompleteFunc, if not nil, must be called after committing the results of this step into
// the state of the deployment.
Apply(preview bool) (resource.Status, StepCompleteFunc, error) // applies or previews this step.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
Op() display.StepOp // the operation performed by this step.
URN() resource.URN // the resource URN (for before and after).
Type() tokens.Type // the type affected by this step.
Provider() string // the provider reference for this step.
Old() *resource.State // the state of the resource before performing this step.
New() *resource.State // the state of the resource after performing this step.
Res() *resource.State // the latest state for the resource that is known (worst case, old).
Logical() bool // true if this step represents a logical operation in the program.
Deployment() *Deployment // the owning deployment.
Fail() // mark a step as failed.
Skip() // mark a step as skipped
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
// SameStep is a mutating step that does nothing.
type SameStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the resource before this step.
new *resource.State // the state of the resource after this step.
// If this is a same-step for a resource being created but which was not --target'ed by the user
// (and thus was skipped).
skippedCreate bool
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*SameStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewSameStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is custom")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new == old || new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &SameStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
// NewSkippedCreateStep produces a SameStep for a resource that was created but not targeted
// by the user (and thus was skipped). These act as no-op steps (hence 'same') since we are not
// actually creating the resource, but ensure that we complete resource-registration and convey the
// right information downstream. For example, we will not write these into the checkpoint file.
func NewSkippedCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
// Make the old state here a direct copy of the new state
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old := new.Copy()
return &SameStep{
deployment: deployment,
reg: reg,
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old: old,
new: new,
skippedCreate: true,
}
}
func (s *SameStep) Op() display.StepOp { return OpSame }
func (s *SameStep) Deployment() *Deployment { return s.deployment }
func (s *SameStep) Type() tokens.Type { return s.new.Type }
func (s *SameStep) Provider() string { return s.new.Provider }
func (s *SameStep) URN() resource.URN { return s.new.URN }
func (s *SameStep) Old() *resource.State { return s.old }
func (s *SameStep) New() *resource.State { return s.new }
func (s *SameStep) Res() *resource.State { return s.new }
func (s *SameStep) Logical() bool { return true }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *SameStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
2021-07-28 19:12:53 +00:00
// Retain the ID and outputs
s.new.ID = s.old.ID
s.new.Outputs = s.old.Outputs
2021-07-28 19:12:53 +00:00
// If the resource is a provider, ensure that it is present in the registry under the appropriate URNs.
// We can only do this if the provider is actually a same, not a skipped create.
if providers.IsProviderType(s.new.Type) && !s.skippedCreate {
2021-07-28 19:12:53 +00:00
if s.Deployment() != nil {
Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
err := s.Deployment().SameProvider(s.new)
if err != nil {
return resource.StatusOK, nil,
fmt.Errorf("bad provider state for resource %v: %w", s.URN(), err)
Don't load providers at startup This changes the provider registry to no longer load all the providers from the old state on startup (in `NewRegistry`) instead the load logic has been moved to the `Same` method. The step_executor and step_generator have been fixed up to ensure that for cases where a resource might not have had it's provider created yet (i.e. for DBR'ing the old version of a resource, for refreshes or deletes) they ask the `Deployment` to look up the provider in the old state and `Same` it in the registry. All of the above means we only load providers we're going to use (even taking --targets into account). One fix mot done in this change is to auto-update providers for deletes. That is given a program state with two resources both using V1 of a provider, if you run the program to update one of those resource to use V2 of the provider but to delete the other resource currently we'll still load V1 to do that delete. It _might_ be possible (although this is definitly questionable) to see that another resource changed it's provider from V1 to V2 and to just assume the same change should have happened to the deleted resource. This could be helpful for not loading old provider versions at all, but can be done in two passes now pretty easily. Just run `up` without any program changes except for the SDK version bump to update all the provider references to V2 of the provider, then do another `up` that deletes the second resource. Fixes https://github.com/pulumi/pulumi/issues/12177.
2023-04-12 09:35:20 +00:00
}
2021-07-28 19:12:53 +00:00
}
}
// TODO: should this step be marked as skipped if it comes from a targeted up?
Propagate inputs to outputs during preview. (#3327) These changes restore a more-correct version of the behavior that was disabled with #3014. The original implementation of this behavior was done in the SDKs, which do not have access to the complete inputs for a resource (in particular, default values filled in by the provider during `Check` are not exposed to the SDK). This lack of information meant that the resolved output values could disagree with the typings present in a provider SDK. Exacerbating this problem was the fact that unknown values were dropped entirely, causing `undefined` values to appear in unexpected places. By doing this in the engine and allowing unknown values to be represented in a first-class manner in the SDK, we can attack both of these issues. Although this behavior is not _strictly_ consistent with respect to the resource model--in an update, a resource's output properties will come from its provider and may differ from its input properties--this behavior was present in the product for a fairly long time without significant issues. In the future, we may be able to improve the accuracy of resource outputs during a preview by allowing the provider to dry-run CRUD operations and return partially-known values where possible. These changes also introduce new APIs in the Node and Python SDKs that work with unknown values in a first-class fashion: - A new parameter to the `apply` function that indicates that the callback should be run even if the result of the apply contains unknown values - `containsUnknowns` and `isUnknown`, which return true if a value either contains nested unknown values or is exactly an unknown value - The `Unknown` type, which represents unknown values The primary use case for these APIs is to allow nested, properties with known values to be accessed via the lifted property accessor even when the containing property is not fully know. A common example of this pattern is the `metadata.name` property of a Kubernetes `Namespace` object: while other properties of the `metadata` bag may be unknown, `name` is often known. These APIs allow `ns.metadata.name` to return a known value in this case. In order to avoid exposing downlevel SDKs to unknown values--a change which could break user code by exposing it to unexpected values--a language SDK must indicate whether or not it supports first-class unknown values as part of each `RegisterResourceRequest`. These changes also allow us to avoid breaking user code with the new behavior introduced by the prior commit. Fixes #3190.
2019-11-11 20:09:34 +00:00
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
return resource.StatusOK, complete, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *SameStep) IsSkippedCreate() bool {
return s.skippedCreate
}
func (s *SameStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *SameStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// CreateStep is a mutating step that creates an entirely new resource.
type CreateStep struct {
deployment *Deployment // the current deployment.
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the existing resource (only for replacements).
new *resource.State // the state of the resource after this step.
keys []resource.PropertyKey // the keys causing replacement (only for replacements).
diffs []resource.PropertyKey // the keys causing a diff (only for replacements).
detailedDiff map[string]plugin.PropertyDiff // the structured property diff (only for replacements).
replacing bool // true if this is a create due to a replacement.
pendingDelete bool // true if this replacement should create a pending delete.
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*CreateStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewCreateStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State) Step {
contract.Requiref(reg != nil, "reg", "must not be nil")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &CreateStep{
deployment: deployment,
reg: reg,
new: new,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func NewCreateReplacementStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
keys, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
) Step {
contract.Requiref(reg != nil, "reg", "must not be nil")
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
return &CreateStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
keys: keys,
diffs: diffs,
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
detailedDiff: detailedDiff,
replacing: true,
pendingDelete: pendingDelete,
}
}
func (s *CreateStep) Op() display.StepOp {
if s.replacing {
return OpCreateReplacement
}
return OpCreate
}
func (s *CreateStep) Deployment() *Deployment { return s.deployment }
func (s *CreateStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *CreateStep) Provider() string { return s.new.Provider }
func (s *CreateStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *CreateStep) Old() *resource.State { return s.old }
func (s *CreateStep) New() *resource.State { return s.new }
func (s *CreateStep) Res() *resource.State { return s.new }
func (s *CreateStep) Keys() []resource.PropertyKey { return s.keys }
func (s *CreateStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *CreateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *CreateStep) Logical() bool { return !s.replacing }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *CreateStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
var resourceError error
resourceStatus := resource.StatusOK
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
id := s.new.ID
outs := s.new.Outputs
if s.new.Custom {
// Invoke the Create RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
2019-07-15 21:26:28 +00:00
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
var rst resource.Status
id, outs, rst, err = prov.Create(s.URN(), s.new.Inputs, s.new.CustomTimeouts.Create, s.deployment.preview)
if err != nil {
if rst != resource.StatusPartialFailure {
return rst, nil, err
}
resourceError = err
resourceStatus = rst
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
if !preview && id == "" {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return resourceStatus, nil, errors.New("provider did not return an ID from Create")
Implement components This change implements core support for "components" in the Pulumi Fabric. This work is described further in pulumi/pulumi#340, where we are still discussing some of the finer points. In a nutshell, resources no longer imply external providers. It's entirely possible to have a resource that logically represents something but without having a physical manifestation that needs to be tracked and managed by our typical CRUD operations. For example, the aws/serverless/Function helper is one such type. It aggregates Lambda-related resources and exposes a nice interface. All of the Pulumi Cloud Framework resources are also examples. To indicate that a resource does participate in the usual CRUD resource provider, it simply derives from ExternalResource instead of Resource. All resources now have the ability to adopt children. This is purely a metadata/tagging thing, and will help us roll up displays, provide attribution to the developer, and even hide aspects of the resource graph as appropriate (e.g., when they are implementation details). Our use of this capability is ultra limited right now; in fact, the only place we display children is in the CLI output. For instance: + aws:serverless:Function: (create) [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda] => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0 => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda The bit indicating whether a resource is external or not is tracked in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
// Copy any of the default and output properties on the live object state.
s.new.ID = id
s.new.Outputs = outs
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Create should set the Create and Modified timestamps as the resource state has been created.
now := time.Now().UTC()
s.new.Created = &now
s.new.Modified = &now
// Mark the old resource as pending deletion if necessary.
if s.replacing && s.pendingDelete {
Test SnapshotManager and Journal in engine tests (#15871) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This tests the production snapshot manager against the journalling snapshot system in engine tests. This should ensure that both systems produce similar (we ignore exact resource ordering, as long as both are valid) results. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-04-11 22:54:08 +00:00
contract.Assertf(s.old != s.new, "old and new states should not be the same")
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.old.Lock.Lock()
s.old.Delete = true
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.old.Lock.Unlock()
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
2018-07-06 22:17:32 +00:00
if resourceError == nil {
return resourceStatus, complete, nil
2018-07-06 22:17:32 +00:00
}
return resourceStatus, complete, resourceError
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *CreateStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *CreateStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
// DeleteStep is a mutating step that deletes an existing resource. If `old` is marked "External",
// DeleteStep is a no-op.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type DeleteStep struct {
deployment *Deployment // the current deployment.
old *resource.State // the state of the existing resource.
replacing bool // true if part of a replacement.
otherDeletions map[resource.URN]bool // other resources that are planned to delete
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*DeleteStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewDeleteStep(deployment *Deployment, otherDeletions map[resource.URN]bool, old *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
return &DeleteStep{
deployment: deployment,
old: old,
otherDeletions: otherDeletions,
}
}
func NewDeleteReplacementStep(
deployment *Deployment,
otherDeletions map[resource.URN]bool,
old *resource.State,
pendingReplace bool,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(otherDeletions != nil, "otherDeletions", "must not be nil")
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
// There are two cases in which we create a delete-replacment step:
//
// 1. When creating the delete steps that occur due to a delete-before-replace
// 2. When creating the delete step that occurs due to a delete-after-replace
//
// In the former case, the persistence layer may require that the resource remain in the
// checkpoint file for purposes of checkpoint integrity. We communicate this case by means
// of the `PendingReplacement` field on `resource.State`, which we set here.
//
// In the latter case, the resource must be deleted, but the deletion may not occur if an earlier step fails.
// The engine requires that the fact that the old resource must be deleted is persisted in the checkpoint so
// that it can issue a deletion of this resource on the next update to this stack.
contract.Assertf(pendingReplace != old.Delete,
"resource %v cannot be pending replacement and deletion at the same time", old.URN)
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old.Lock.Lock()
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
old.PendingReplacement = pendingReplace
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
old.Lock.Unlock()
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &DeleteStep{
deployment: deployment,
otherDeletions: otherDeletions,
old: old,
replacing: true,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func (s *DeleteStep) Op() display.StepOp {
if s.old.External {
if s.replacing {
return OpDiscardReplaced
}
return OpReadDiscard
}
if s.replacing {
return OpDeleteReplaced
}
return OpDelete
}
func (s *DeleteStep) Deployment() *Deployment { return s.deployment }
func (s *DeleteStep) Type() tokens.Type { return s.old.Type }
func (s *DeleteStep) Provider() string { return s.old.Provider }
func (s *DeleteStep) URN() resource.URN { return s.old.URN }
func (s *DeleteStep) Old() *resource.State { return s.old }
func (s *DeleteStep) New() *resource.State { return nil }
func (s *DeleteStep) Res() *resource.State { return s.old }
func (s *DeleteStep) Logical() bool { return !s.replacing }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func isDeletedWith(with resource.URN, otherDeletions map[resource.URN]bool) bool {
if with == "" {
return false
}
r, ok := otherDeletions[with]
if !ok {
return false
}
return r
}
2023-01-10 18:32:12 +00:00
type deleteProtectedError struct {
urn resource.URN
}
func (d deleteProtectedError) Error() string {
return fmt.Sprintf("resource %[1]q cannot be deleted\n"+
"because it is protected. To unprotect the resource, "+
"either remove the `protect` flag from the resource in your Pulumi "+
"program and run `pulumi up`, or use the command:\n"+
"`pulumi state unprotect %[2]s`", d.urn, d.urn.Quote())
2023-01-10 18:32:12 +00:00
}
func (s *DeleteStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
// Refuse to delete protected resources (unless we're replacing them in
// which case we will of checked protect elsewhere)
if !s.replacing && s.old.Protect {
2023-01-10 18:32:12 +00:00
return resource.StatusOK, nil, deleteProtectedError{urn: s.old.URN}
}
if preview {
// Do nothing in preview
} else if s.old.External {
// Deleting an External resource is a no-op, since Pulumi does not own the lifecycle.
} else if s.old.RetainOnDelete {
// Deleting a "drop on delete" is a no-op as the user has explicitly asked us to not delete the resource.
} else if isDeletedWith(s.old.DeletedWith, s.otherDeletions) {
// No need to delete this resource since this resource will be deleted by the another deletion
} else if s.old.Custom {
// Not preview and not external and not Drop and is custom, do the actual delete
// Invoke the Delete RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
2019-07-15 21:26:28 +00:00
Send old inputs to Delete (#14051) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14115. This was missed as part of https://github.com/pulumi/pulumi/pull/13139. Adds a new configure flag (sends_old_inputs_to_delete) which the engine will now always set to true. If that's set providers can rely on the old inputs being sent to delete, otherwise they'll get nil. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-13 14:12:26 +00:00
if rst, err := prov.Delete(s.URN(), s.old.ID, s.old.Inputs, s.old.Outputs, s.old.CustomTimeouts.Delete); err != nil {
return rst, nil, err
Implement components This change implements core support for "components" in the Pulumi Fabric. This work is described further in pulumi/pulumi#340, where we are still discussing some of the finer points. In a nutshell, resources no longer imply external providers. It's entirely possible to have a resource that logically represents something but without having a physical manifestation that needs to be tracked and managed by our typical CRUD operations. For example, the aws/serverless/Function helper is one such type. It aggregates Lambda-related resources and exposes a nice interface. All of the Pulumi Cloud Framework resources are also examples. To indicate that a resource does participate in the usual CRUD resource provider, it simply derives from ExternalResource instead of Resource. All resources now have the ability to adopt children. This is purely a metadata/tagging thing, and will help us roll up displays, provide attribution to the developer, and even hide aspects of the resource graph as appropriate (e.g., when they are implementation details). Our use of this capability is ultra limited right now; in fact, the only place we display children is in the CLI output. For instance: + aws:serverless:Function: (create) [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda] => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0 => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda The bit indicating whether a resource is external or not is tracked in the resulting checkpoint file, along with any of its children.
2017-10-14 21:18:43 +00:00
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
return resource.StatusOK, func() {}, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *DeleteStep) Fail() {
// Nothing to do here.
}
func (s *DeleteStep) Skip() {
// Nothing to do here.
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
type RemovePendingReplaceStep struct {
deployment *Deployment // the current deployment.
old *resource.State // the state of the existing resource.
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
}
func NewRemovePendingReplaceStep(deployment *Deployment, old *resource.State) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.PendingReplacement, "old", "must be pending replacement")
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
return &RemovePendingReplaceStep{
deployment: deployment,
old: old,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
}
}
func (s *RemovePendingReplaceStep) Op() display.StepOp {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
return OpRemovePendingReplace
}
func (s *RemovePendingReplaceStep) Deployment() *Deployment { return s.deployment }
func (s *RemovePendingReplaceStep) Type() tokens.Type { return s.old.Type }
func (s *RemovePendingReplaceStep) Provider() string { return s.old.Provider }
func (s *RemovePendingReplaceStep) URN() resource.URN { return s.old.URN }
func (s *RemovePendingReplaceStep) Old() *resource.State { return s.old }
func (s *RemovePendingReplaceStep) New() *resource.State { return nil }
func (s *RemovePendingReplaceStep) Res() *resource.State { return s.old }
func (s *RemovePendingReplaceStep) Logical() bool { return false }
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
func (s *RemovePendingReplaceStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
return resource.StatusOK, nil, nil
}
func (s *RemovePendingReplaceStep) Fail() {
// Nothing to do here.
}
func (s *RemovePendingReplaceStep) Skip() {
// Nothing to do here.
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// UpdateStep is a mutating step that updates an existing resource's state.
type UpdateStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
old *resource.State // the state of the existing resource.
new *resource.State // the newly computed state of the resource after updating.
stables []resource.PropertyKey // an optional list of properties that won't change during this update.
diffs []resource.PropertyKey // the keys causing a diff.
detailedDiff map[string]plugin.PropertyDiff // the structured diff.
ignoreChanges []string // a list of property paths to ignore when updating.
provider plugin.Provider // the optional provider to use.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*UpdateStep)(nil)
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func NewUpdateStep(deployment *Deployment, reg RegisterResourceEvent, old, new *resource.State,
stables, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type),
"old", "must have or be a provider if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(!old.External, "old", "must not be an external resource")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID == "", "new", "must not have an ID")
contract.Requiref(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type),
"new", "must have or be a provider if it is a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be an external resource")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &UpdateStep{
deployment: deployment,
reg: reg,
old: old,
new: new,
stables: stables,
diffs: diffs,
detailedDiff: detailedDiff,
ignoreChanges: ignoreChanges,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
}
func (s *UpdateStep) Op() display.StepOp { return OpUpdate }
func (s *UpdateStep) Deployment() *Deployment { return s.deployment }
func (s *UpdateStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *UpdateStep) Provider() string { return s.new.Provider }
func (s *UpdateStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *UpdateStep) Old() *resource.State { return s.old }
func (s *UpdateStep) New() *resource.State { return s.new }
func (s *UpdateStep) Res() *resource.State { return s.new }
func (s *UpdateStep) Logical() bool { return true }
func (s *UpdateStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *UpdateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *UpdateStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
// Always propagate the ID and timestamps even in previews and refreshes.
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
s.new.ID = s.old.ID
s.new.Created = s.old.Created
s.new.Modified = s.old.Modified
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Unlock()
var resourceError error
resourceStatus := resource.StatusOK
if s.new.Custom {
// Invoke the Update RPC function for this provider:
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
General prep work for refresh This change includes a bunch of refactorings I made in prep for doing refresh (first, the command, see pulumi/pulumi#1081): * The primary change is to change the way the engine's core update functionality works with respect to deploy.Source. This is the way we can plug in new sources of resource information during planning (and, soon, diffing). The way I intend to model refresh is by having a new kind of source, deploy.RefreshSource, which will let us do virtually everything about an update/diff the same way with refreshes, which avoid otherwise duplicative effort. This includes changing the planOptions (nee deployOptions) to take a new SourceFunc callback, which is responsible for creating a source specific to the kind of plan being requested. Preview, Update, and Destroy now are primarily differentiated by the kind of deploy.Source that they return, rather than sprinkling things like `if Destroying` throughout. This tidies up some logic and, more importantly, gives us precisely the refresh hook we need. * Originally, we used the deploy.NullSource for Destroy operations. This simply returns nothing, which is how Destroy works. For some reason, we were no longer doing this, and instead had some `if Destroying` cases sprinkled throughout the deploy.EvalSource. I think this is a vestige of some old way we did configuration, at least judging by a comment, which is apparently no longer relevant. * Move diff and diff-printing logic within the engine into its own pkg/engine/diff.go file, to prepare for upcoming work. * I keep noticing benign diffs anytime I regenerate protobufs. I suspect this is because we're also on different versions. I changed generate.sh to also dump the version into grpc_version.txt. At least we can understand where the diffs are coming from, decide whether to take them (i.e., a newer version), and ensure that as a team we are monotonically increasing, and not going backwards. * I also tidied up some tiny things I noticed while in there, like comments, incorrect types, lint suppressions, and so on.
2018-03-28 14:45:23 +00:00
// Update to the combination of the old "all" state, but overwritten with new inputs.
outs, rst, upderr := prov.Update(s.URN(), s.old.ID, s.old.Inputs, s.old.Outputs, s.new.Inputs,
s.new.CustomTimeouts.Update, s.ignoreChanges, s.deployment.preview)
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
if upderr != nil {
if rst != resource.StatusPartialFailure {
return rst, nil, upderr
}
resourceError = upderr
resourceStatus = rst
if initErr, isInitErr := upderr.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
// Now copy any output state back in case the update triggered cascading updates to other properties.
s.new.Outputs = outs
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
// UpdateStep doesn't create, but does modify state.
// Change the Modified timestamp.
now := time.Now().UTC()
s.new.Modified = &now
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Finally, mark this operation as complete.
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
2018-07-06 22:17:32 +00:00
if resourceError == nil {
return resourceStatus, complete, nil
2018-07-06 22:17:32 +00:00
}
return resourceStatus, complete, resourceError
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *UpdateStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *UpdateStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
// ReplaceStep is a logical step indicating a resource will be replaced. This is comprised of three physical steps:
// a creation of the new resource, any number of intervening updates of dependents to the new resource, and then
// a deletion of the now-replaced old resource. This logical step is primarily here for tools and visualization.
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
type ReplaceStep struct {
deployment *Deployment // the current deployment.
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
old *resource.State // the state of the existing resource.
new *resource.State // the new state snapshot.
keys []resource.PropertyKey // the keys causing replacement.
diffs []resource.PropertyKey // the keys causing a diff.
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
pendingDelete bool // true if a pending deletion should happen.
}
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
var _ Step = (*ReplaceStep)(nil)
func NewReplaceStep(deployment *Deployment, old, new *resource.State, keys, diffs []resource.PropertyKey,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool,
) Step {
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(old.URN != "", "old", "must have a URN")
contract.Requiref(old.ID != "" || !old.Custom, "old", "must have an ID if it is a custom resource")
contract.Requiref(!old.Delete, "old", "must not be marked for deletion")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
// contract.Assert(new.ID == "")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return &ReplaceStep{
deployment: deployment,
old: old,
new: new,
keys: keys,
diffs: diffs,
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
detailedDiff: detailedDiff,
pendingDelete: pendingDelete,
}
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *ReplaceStep) Op() display.StepOp { return OpReplace }
func (s *ReplaceStep) Deployment() *Deployment { return s.deployment }
func (s *ReplaceStep) Type() tokens.Type { return s.new.Type }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *ReplaceStep) Provider() string { return s.new.Provider }
func (s *ReplaceStep) URN() resource.URN { return s.new.URN }
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 19:34:19 +00:00
func (s *ReplaceStep) Old() *resource.State { return s.old }
func (s *ReplaceStep) New() *resource.State { return s.new }
func (s *ReplaceStep) Res() *resource.State { return s.new }
func (s *ReplaceStep) Keys() []resource.PropertyKey { return s.keys }
func (s *ReplaceStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *ReplaceStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *ReplaceStep) Logical() bool { return true }
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
func (s *ReplaceStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
// If this is a pending delete, we should have marked the old resource for deletion in the CreateReplacement step.
contract.Assertf(!s.pendingDelete || s.old.Delete,
"old resource %v should be marked for deletion if pending delete", s.old.URN)
return resource.StatusOK, func() {}, nil
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
}
func (s *ReplaceStep) Fail() {
// Nothing to do here.
}
func (s *ReplaceStep) Skip() {
// Nothing to do here.
}
// ReadStep is a step indicating that an existing resources will be "read" and projected into the Pulumi object
// model. Resources that are read are marked with the "External" bit which indicates to the engine that it does
// not own this resource's lifeycle.
//
// A resource with a given URN can transition freely between an "external" state and a non-external state. If
// a URN that was previously marked "External" (i.e. was the target of a ReadStep in a previous deployment) is the
// target of a RegisterResource in the next deployment, a CreateReplacement step will be issued to indicate the
// transition from external to owned. If a URN that was previously not marked "External" is the target of a
// ReadResource in the next deployment, a ReadReplacement step will be issued to indicate the transition from owned to
// external.
type ReadStep struct {
deployment *Deployment // the deployment that produced this read
event ReadResourceEvent // the event that should be signaled upon completion
old *resource.State // the old resource state, if one exists for this urn
new *resource.State // the new resource state, to be used to query the provider
replacing bool // whether or not the new resource is replacing the old resource
provider plugin.Provider // the optional provider to use.
}
// NewReadStep creates a new Read step.
func NewReadStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.External, "new", "must be marked as external")
contract.Requiref(new.Custom, "new", "must be a custom resource")
// If Old was given, it's either an external resource or its ID is equal to the
// ID that we are preparing to read.
if old != nil {
contract.Requiref(old.ID == new.ID || old.External,
"old", "must have the same ID as new or be external")
}
return &ReadStep{
deployment: deployment,
event: event,
old: old,
new: new,
replacing: false,
}
}
// NewReadReplacementStep creates a new Read step with the `replacing` flag set. When executed,
// it will pend deletion of the "old" resource, which must not be an external resource.
func NewReadReplacementStep(deployment *Deployment, event ReadResourceEvent, old, new *resource.State) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.External, "new", "must be marked as external")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(old != nil, "old", "must not be nil")
contract.Requiref(!old.External, "old", "must not be marked as external")
return &ReadStep{
deployment: deployment,
event: event,
old: old,
new: new,
replacing: true,
}
}
func (s *ReadStep) Op() display.StepOp {
if s.replacing {
return OpReadReplacement
}
return OpRead
}
func (s *ReadStep) Deployment() *Deployment { return s.deployment }
func (s *ReadStep) Type() tokens.Type { return s.new.Type }
func (s *ReadStep) Provider() string { return s.new.Provider }
func (s *ReadStep) URN() resource.URN { return s.new.URN }
func (s *ReadStep) Old() *resource.State { return s.old }
func (s *ReadStep) New() *resource.State { return s.new }
func (s *ReadStep) Res() *resource.State { return s.new }
func (s *ReadStep) Logical() bool { return !s.replacing }
func (s *ReadStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
urn := s.new.URN
id := s.new.ID
var resourceError error
resourceStatus := resource.StatusOK
// Unlike most steps, Read steps run during previews. The only time
// we can't run is if the ID we are given is unknown.
if id == plugin.UnknownStringValue {
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
s.new.Outputs = resource.PropertyMap{}
} else {
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
Pass inputs to read as inputs and state (#14683) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Also adds some more checks to the import tests to give confidence that the import machinery was behaving as expected. I noticed this inconsistency of Read when looking at https://github.com/pulumi/pulumi/pull/14678. Comment in the code at the change site, but just to repeat here. I don't think we can stop sending inputs as state because providers have probably come to depend on that, but it's a little more consistent if we pass them as inputs as well. Means in the stackref code we can _just_ look at the inputs set. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-02 14:39:38 +00:00
// Technically the only data we have at this point is "inputs", but we've been passing that as "state" to
// providers since forever and it would probably break things to stop sending that now. Thus this strange double
// send of inputs as both "inputs" and "state". Something to break to tidy up in V4.
result, rst, err := prov.Read(urn, id, s.new.Inputs, s.new.Inputs)
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
if err != nil {
if rst != resource.StatusPartialFailure {
return rst, nil, err
}
resourceError = err
resourceStatus = rst
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
}
}
// If there is no such resource, return an error indicating as such.
if result.Outputs == nil {
return resource.StatusOK, nil, fmt.Errorf("resource '%s' does not exist", id)
}
s.new.Outputs = result.Outputs
if result.ID != "" {
s.new.ID = result.ID
}
}
// If we were asked to replace an existing, non-External resource, pend the
// deletion here.
if s.replacing {
s.old.Delete = true
}
// Propagate timestamps on Read.
if s.old != nil {
s.new.Created = s.old.Created
s.new.Modified = s.old.Modified
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
var inputsChange, outputsChange bool
if s.old != nil {
inputsChange = !s.new.Inputs.DeepEquals(s.old.Inputs)
outputsChange = !s.new.Outputs.DeepEquals(s.old.Outputs)
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Only update the Modified timestamp if read provides new values that differ
// from the old state.
if inputsChange || outputsChange {
now := time.Now().UTC()
s.new.Modified = &now
}
complete := func() { s.event.Done(&ReadResult{State: s.new}) }
if resourceError == nil {
return resourceStatus, complete, nil
}
return resourceStatus, complete, resourceError
}
func (s *ReadStep) Fail() {
s.event.Done(&ReadResult{State: s.new, Result: ResultStateFailed})
}
func (s *ReadStep) Skip() {
s.event.Done(&ReadResult{State: s.new, Result: ResultStateSkipped})
}
// RefreshStep is a step used to track the progress of a refresh operation. A refresh operation updates the an existing
// resource by reading its current state from its provider plugin. These steps are not issued by the step generator;
// instead, they are issued by the deployment executor as the optional first step in deployment execution.
type RefreshStep struct {
deployment *Deployment // the deployment that produced this refresh
old *resource.State // the old resource state, if one exists for this urn
new *resource.State // the new resource state, to be used to query the provider
done chan<- bool // the channel to use to signal completion, if any
provider plugin.Provider // the optional provider to use.
}
// NewRefreshStep creates a new Refresh step.
func NewRefreshStep(deployment *Deployment, old *resource.State, done chan<- bool) Step {
contract.Requiref(old != nil, "old", "must not be nil")
// NOTE: we set the new state to the old state by default so that we don't interpret step failures as deletes.
return &RefreshStep{
deployment: deployment,
old: old,
new: old,
done: done,
}
}
func (s *RefreshStep) Op() display.StepOp { return OpRefresh }
func (s *RefreshStep) Deployment() *Deployment { return s.deployment }
func (s *RefreshStep) Type() tokens.Type { return s.old.Type }
func (s *RefreshStep) Provider() string { return s.old.Provider }
func (s *RefreshStep) URN() resource.URN { return s.old.URN }
func (s *RefreshStep) Old() *resource.State { return s.old }
func (s *RefreshStep) New() *resource.State { return s.new }
func (s *RefreshStep) Res() *resource.State { return s.old }
func (s *RefreshStep) Logical() bool { return false }
// ResultOp returns the operation that corresponds to the change to this resource after reading its current state, if
// any.
func (s *RefreshStep) ResultOp() display.StepOp {
if s.new == nil {
return OpDelete
}
if s.new == s.old || s.old.Outputs.Diff(s.new.Outputs) == nil {
return OpSame
}
return OpUpdate
}
func (s *RefreshStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
var complete func()
if s.done != nil {
complete = func() { close(s.done) }
}
resourceID := s.old.ID
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
// Component, provider, and pending-replace resources never change with a refresh; just return the current state.
if !s.old.Custom || providers.IsProviderType(s.old.Type) || s.old.PendingReplacement {
return resource.StatusOK, complete, nil
}
// For a custom resource, fetch the resource's provider and read the resource's current state.
prov, err := getProvider(s, s.provider)
if err != nil {
return resource.StatusOK, nil, err
}
var initErrors []string
refreshed, rst, err := prov.Read(s.old.URN, resourceID, s.old.Inputs, s.old.Outputs)
if err != nil {
if rst != resource.StatusPartialFailure {
return rst, nil, err
}
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
initErrors = initErr.Reasons
// Partial failure SHOULD NOT cause refresh to fail. Instead:
//
// 1. Warn instead that during refresh we noticed the resource has become unhealthy.
// 2. Make sure the initialization errors are persisted in the state, so that the next
// `pulumi up` will surface them to the user.
err = nil
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
msg := "Refreshed resource is in an unhealthy state:\n* " + strings.Join(initErrors, "\n* ")
s.Deployment().Diag().Warningf(diag.RawMessage(s.URN(), msg))
}
}
outputs := refreshed.Outputs
// If the provider specified new inputs for this resource, pick them up now. Otherwise, retain the current inputs.
inputs := s.old.Inputs
if refreshed.Inputs != nil {
inputs = refreshed.Inputs
}
if outputs != nil {
// There is a chance that the ID has changed. We want to allow this change to happen
// it will have changed already in the outputs, but we need to persist this change
// at a state level because the Id
if refreshed.ID != "" && refreshed.ID != resourceID {
logging.V(7).Infof("Refreshing ID; oldId=%s, newId=%s", resourceID, refreshed.ID)
resourceID = refreshed.ID
}
s.new = resource.NewState(s.old.Type, s.old.URN, s.old.Custom, s.old.Delete, resourceID, inputs, outputs,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
s.old.Parent, s.old.Protect, s.old.External, s.old.Dependencies, initErrors, s.old.Provider,
s.old.PropertyDependencies, s.old.PendingReplacement, s.old.AdditionalSecretOutputs, s.old.Aliases,
[engine] Add support for source positions These changes add support for passing source position information in gRPC metadata and recording the source position that corresponds to a resource registration in the statefile. Enabling source position information in the resource model can provide substantial benefits, including but not limited to: - Better errors from the Pulumi CLI - Go-to-defintion for resources in state - Editor integration for errors, etc. from `pulumi preview` Source positions are (file, line) or (file, line, column) tuples represented as URIs. The line and column are stored in the fragment portion of the URI as "line(,column)?". The scheme of the URI and the form of its path component depends on the context in which it is generated or used: - During an active update, the URI's scheme is `file` and paths are absolute filesystem paths. This allows consumers to easily access arbitrary files that are available on the host. - In a statefile, the URI's scheme is `project` and paths are relative to the project root. This allows consumers to resolve source positions relative to the project file in different contexts irrespective of the location of the project itself (e.g. given a project-relative path and the URL of the project's root on GitHub, one can build a GitHub URL for the source position). During an update, source position information may be attached to gRPC calls as "source-position" metadata. This allows arbitrary calls to be associated with source positions without changes to their protobuf payloads. Modifying the protobuf payloads is also a viable approach, but is somewhat more invasive than attaching metadata, and requires changes to every call signature. Source positions should reflect the position in user code that initiated a resource model operation (e.g. the source position passed with `RegisterResource` for `pet` in the example above should be the source position in `index.ts`, _not_ the source position in the Pulumi SDK). In general, the Pulumi SDK should be able to infer the source position of the resource registration, as the relationship between a resource registration and its corresponding user code should be static per SDK. Source positions in state files will be stored as a new `registeredAt` property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
&s.old.CustomTimeouts, s.old.ImportID, s.old.RetainOnDelete, s.old.DeletedWith, s.old.Created, s.old.Modified,
s.old.SourcePosition,
)
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
var inputsChange, outputsChange bool
if s.old != nil {
inputsChange = !refreshed.Inputs.DeepEquals(s.old.Inputs)
outputsChange = !refreshed.Outputs.DeepEquals(s.old.Outputs)
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Only update the Modified timestamp if refresh provides new values that differ
// from the old state.
if inputsChange || outputsChange {
// The refresh has identified an incongruence between the provider and state
// updated the Modified timestamp to track this.
now := time.Now().UTC()
s.new.Modified = &now
}
} else {
s.new = nil
}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
return rst, nil, err
}
func (s *RefreshStep) Fail() {
// Nothing to do here.
}
func (s *RefreshStep) Skip() {
// Nothing to do here.
}
type ImportStep struct {
deployment *Deployment // the current deployment.
reg RegisterResourceEvent // the registration intent to convey a URN back to.
original *resource.State // the original resource, if this is an import-replace.
old *resource.State // the state of the resource fetched from the provider.
new *resource.State // the newly computed state of the resource after importing.
replacing bool // true if we are replacing a Pulumi-managed resource.
planned bool // true if this import is from an import deployment.
diffs []resource.PropertyKey // any keys that differed between the user's program and the actual state.
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
ignoreChanges []string // a list of property paths to ignore when updating.
randomSeed []byte // the random seed to use for Check.
provider plugin.Provider // the optional provider to use.
}
func NewImportStep(deployment *Deployment, reg RegisterResourceEvent, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string, randomSeed []byte,
) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: reg,
new: new,
ignoreChanges: ignoreChanges,
randomSeed: randomSeed,
}
}
func NewImportReplacementStep(deployment *Deployment, reg RegisterResourceEvent, original, new *resource.State,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
ignoreChanges []string, randomSeed []byte,
) Step {
contract.Requiref(original != nil, "original", "must not be nil")
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
contract.Requiref(new.ID != "", "new", "must have an ID")
contract.Requiref(new.Custom, "new", "must be a custom resource")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
contract.Requiref(randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: reg,
original: original,
new: new,
replacing: true,
ignoreChanges: ignoreChanges,
randomSeed: randomSeed,
}
}
func newImportDeploymentStep(deployment *Deployment, new *resource.State, randomSeed []byte) Step {
contract.Requiref(new != nil, "new", "must not be nil")
contract.Requiref(new.URN != "", "new", "must have a URN")
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
contract.Requiref(!new.Custom || new.ID != "", "new", "must have an ID")
contract.Requiref(!new.Delete, "new", "must not be marked for deletion")
contract.Requiref(!new.External, "new", "must not be external")
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
contract.Requiref(!new.Custom || randomSeed != nil, "randomSeed", "must not be nil")
return &ImportStep{
deployment: deployment,
reg: noopEvent(0),
new: new,
planned: true,
randomSeed: randomSeed,
}
}
func (s *ImportStep) Op() display.StepOp {
if s.replacing {
return OpImportReplacement
}
return OpImport
}
func (s *ImportStep) Deployment() *Deployment { return s.deployment }
func (s *ImportStep) Type() tokens.Type { return s.new.Type }
func (s *ImportStep) Provider() string { return s.new.Provider }
func (s *ImportStep) URN() resource.URN { return s.new.URN }
func (s *ImportStep) Old() *resource.State { return s.old }
func (s *ImportStep) New() *resource.State { return s.new }
func (s *ImportStep) Res() *resource.State { return s.new }
func (s *ImportStep) Logical() bool { return !s.replacing }
func (s *ImportStep) Diffs() []resource.PropertyKey { return s.diffs }
func (s *ImportStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
func (s *ImportStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
complete := func() {
s.reg.Done(&RegisterResult{State: s.new})
}
// If this is a planned import, ensure that the resource does not exist in the old state file.
if s.planned {
if _, ok := s.deployment.olds[s.new.URN]; ok {
return resource.StatusOK, nil, fmt.Errorf("resource '%v' already exists", s.new.URN)
}
if s.new.Parent.QualifiedType() != resource.RootStackType {
_, ok := s.deployment.news.Load(s.new.Parent)
Allow importing a parent and child resource at the same time (#14461) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Allow `pulumi import` to import one resource and then use that resource as a parent for another imported resource. Currently parents can only refer to resources that already exist in the deployment meaning if you want to do this today you have to do two imports. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-10 13:31:11 +00:00
if !ok {
return resource.StatusOK, nil, fmt.Errorf("unknown parent '%v' for resource '%v'",
s.new.Parent, s.new.URN)
}
}
}
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
// Only need to do anything here for custom resources, components just import as empty
inputs := resource.PropertyMap{}
outputs := resource.PropertyMap{}
var prov plugin.Provider
rst := resource.StatusOK
if s.new.Custom {
// Read the current state of the resource to import. If the provider does not hand us back any inputs for the
// resource, it probably needs to be updated. If the resource does not exist at all, fail the import.
var err error
prov, err = getProvider(s, s.provider)
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
if err != nil {
return resource.StatusOK, nil, err
}
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
var read plugin.ReadResult
read, rst, err = prov.Read(s.new.URN, s.new.ID, nil, nil)
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
if err != nil {
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
s.new.InitErrors = initErr.Reasons
} else {
return rst, nil, err
}
}
if read.Outputs == nil {
return rst, nil, fmt.Errorf("resource '%v' does not exist", s.new.ID)
}
if read.Inputs == nil {
return resource.StatusOK, nil,
fmt.Errorf("provider does not support importing resources; please try updating the '%v' plugin",
s.new.URN.Type().Package())
}
if read.ID != "" {
s.new.ID = read.ID
}
inputs = read.Inputs
outputs = read.Outputs
Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This reverts commit 75340dd94203da02e44ca5f8beb55d9063d302ef. Fixes https://github.com/pulumi/pulumi/issues/16018. This re-enables the locking and race detection. The locking is more finely scoped to not be held over provider methods like Read/Update. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-09 16:15:41 +00:00
} else {
s.new.Lock.Lock()
defer s.new.Lock.Unlock()
}
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
s.new.Outputs = outputs
// Magic up an old state so the frontend can display a proper diff. This state is the output of the just-executed
// `Read` combined with the resource identity and metadata from the desired state. This ensures that the only
// differences between the old and new states are between the inputs and outputs.
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
s.old = resource.NewState(s.new.Type, s.new.URN, s.new.Custom, false, s.new.ID, inputs, outputs,
s.new.Parent, s.new.Protect, false, s.new.Dependencies, s.new.InitErrors, s.new.Provider,
s.new.PropertyDependencies, false, nil, nil, &s.new.CustomTimeouts, s.new.ImportID, s.new.RetainOnDelete,
[engine] Add support for source positions These changes add support for passing source position information in gRPC metadata and recording the source position that corresponds to a resource registration in the statefile. Enabling source position information in the resource model can provide substantial benefits, including but not limited to: - Better errors from the Pulumi CLI - Go-to-defintion for resources in state - Editor integration for errors, etc. from `pulumi preview` Source positions are (file, line) or (file, line, column) tuples represented as URIs. The line and column are stored in the fragment portion of the URI as "line(,column)?". The scheme of the URI and the form of its path component depends on the context in which it is generated or used: - During an active update, the URI's scheme is `file` and paths are absolute filesystem paths. This allows consumers to easily access arbitrary files that are available on the host. - In a statefile, the URI's scheme is `project` and paths are relative to the project root. This allows consumers to resolve source positions relative to the project file in different contexts irrespective of the location of the project itself (e.g. given a project-relative path and the URL of the project's root on GitHub, one can build a GitHub URL for the source position). During an update, source position information may be attached to gRPC calls as "source-position" metadata. This allows arbitrary calls to be associated with source positions without changes to their protobuf payloads. Modifying the protobuf payloads is also a viable approach, but is somewhat more invasive than attaching metadata, and requires changes to every call signature. Source positions should reflect the position in user code that initiated a resource model operation (e.g. the source position passed with `RegisterResource` for `pet` in the example above should be the source position in `index.ts`, _not_ the source position in the Pulumi SDK). In general, the Pulumi SDK should be able to infer the source position of the resource registration, as the relationship between a resource registration and its corresponding user code should be static per SDK. Source positions in state files will be stored as a new `registeredAt` property on each resource. This property is optional.
2023-06-29 18:41:19 +00:00
s.new.DeletedWith, nil, nil, s.new.SourcePosition)
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
// Import takes a resource that Pulumi did not create and imports it into pulumi state.
now := time.Now().UTC()
s.new.Modified = &now
// Set Created to now as the resource has been created in the state.
s.new.Created = &now
Allow `import` to create empty component resources (#14467) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14444. This adds two new flags to the import file to signal that a resource is a component, and also if that is a remote (MLC) component. The import system will create empty resources for those import descriptions, but importantly they can then be used as parents for other resources in the import deployment. We need to know if it's a remote component to know if a provider should be looked up for it. We could probably get away with _just_ knowing if it's a component at import time and not doing any validation that the type token is well formed and the provider exists for MLCs. That makes import a little simpler for users (no need to set if it's remote or not) but pushes error cases downstream to when they actually try to `up` their program. It also means we can't lookup and fill in the default providers for these resources, resulting in a provider diff at `up` time. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-13 17:58:35 +00:00
// If this is a component we don't need to do the rest of the input validation
if !s.new.Custom {
return rst, complete, nil
}
// If this step came from an import deployment, we need to fetch any required inputs from the state.
if s.planned {
contract.Assertf(len(s.new.Inputs) == 0, "import resource cannot have existing inputs")
// Get the import object and see if it had properties set
var inputProperties []string
for _, imp := range s.deployment.imports {
if imp.ID == s.old.ID {
inputProperties = imp.Properties
break
}
}
if len(inputProperties) == 0 {
logging.V(9).Infof("Importing %v with all properties", s.URN())
s.new.Inputs = s.old.Inputs.Copy()
} else {
logging.V(9).Infof("Importing %v with supplied properties: %v", s.URN(), inputProperties)
for _, p := range inputProperties {
k := resource.PropertyKey(p)
if value, has := s.old.Inputs[k]; has {
s.new.Inputs[k] = value
}
}
}
// Check the provider inputs for consistency. If the inputs fail validation, the import will still succeed, but
// we will display the validation failures and a message informing the user that the failures are almost
// definitely a provider bug.
_, failures, err := prov.Check(s.new.URN, s.old.Inputs, s.new.Inputs, preview, s.randomSeed)
if err != nil {
return rst, nil, err
}
// Print this warning before printing all the check failures to give better context.
if len(failures) != 0 {
// Based on if the user passed 'properties' or not we want to change the error message here.
var errorMessage string
if len(inputProperties) == 0 {
ref, err := providers.ParseReference(s.Provider())
contract.AssertNoErrorf(err, "failed to parse provider reference %q", s.Provider())
pkgName := ref.URN().Type().Name()
errorMessage = fmt.Sprintf("This is almost certainly a bug in the `%s` provider.", pkgName)
} else {
errorMessage = "Try specifying a different set of properties to import with in the future."
}
s.deployment.Diag().Warningf(diag.Message(s.new.URN,
"One or more imported inputs failed to validate. %s "+
"The import will still proceed, but you will need to edit the generated code after copying it into your program."),
errorMessage)
}
issueCheckFailures(s.deployment.Diag().Warningf, s.new, s.new.URN, failures)
s.diffs, s.detailedDiff = []resource.PropertyKey{}, map[string]plugin.PropertyDiff{}
Don't edit state in completion functions (#14038) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> The completion functions run _after_ the step executor has called OnResourceStepPost where we trigger the snapshot save. So if the data was saved or not depended on it racing the snapshot goroutine. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-25 18:22:23 +00:00
return rst, complete, nil
}
// Set inputs back to their old values (if any) for any "ignored" properties
processedInputs, err := processIgnoreChanges(s.new.Inputs, s.old.Inputs, s.ignoreChanges)
if err != nil {
return resource.StatusOK, nil, err
}
s.new.Inputs = processedInputs
// Check the inputs using the provider inputs for defaults.
inputs, failures, err := prov.Check(s.new.URN, s.old.Inputs, s.new.Inputs, preview, s.randomSeed)
if err != nil {
return rst, nil, err
}
if issueCheckErrors(s.deployment, s.new, s.new.URN, failures) {
return rst, nil, errors.New("one or more inputs failed to validate")
}
s.new.Inputs = inputs
// Diff the user inputs against the provider inputs. If there are any differences, fail the import unless this step
// is from an import deployment.
diff, err := diffResource(s.new.URN, s.new.ID, s.old.Inputs, s.old.Outputs, s.new.Inputs, prov, preview,
s.ignoreChanges)
if err != nil {
return rst, nil, err
}
s.diffs, s.detailedDiff = diff.ChangedKeys, diff.DetailedDiff
if diff.Changes != plugin.DiffNone {
const message = "inputs to import do not match the existing resource"
if preview {
s.deployment.ctx.Diag.Warningf(diag.StreamMessage(s.new.URN,
message+"; importing this resource will fail", 0))
} else {
err = errors.New(message)
}
}
// If we were asked to replace an existing, non-External resource, pend the deletion here.
if err == nil && s.replacing {
s.original.Delete = true
}
return rst, complete, err
}
func (s *ImportStep) Fail() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateFailed})
}
func (s *ImportStep) Skip() {
s.reg.Done(&RegisterResult{State: s.new, Result: ResultStateSkipped})
}
const (
OpSame display.StepOp = "same" // nothing to do.
OpCreate display.StepOp = "create" // creating a new resource.
OpUpdate display.StepOp = "update" // updating an existing resource.
OpDelete display.StepOp = "delete" // deleting an existing resource.
OpReplace display.StepOp = "replace" // replacing a resource with a new one.
OpCreateReplacement display.StepOp = "create-replacement" // creating a new resource for a replacement.
OpDeleteReplaced display.StepOp = "delete-replaced" // deleting an existing resource after replacement.
OpRead display.StepOp = "read" // reading an existing resource.
OpReadReplacement display.StepOp = "read-replacement" // reading an existing resource for a replacement.
OpRefresh display.StepOp = "refresh" // refreshing an existing resource.
OpReadDiscard display.StepOp = "discard" // removing a resource that was read.
OpDiscardReplaced display.StepOp = "discard-replaced" // discarding a read resource that was replaced.
OpRemovePendingReplace display.StepOp = "remove-pending-replace" // removing a pending replace resource.
OpImport display.StepOp = "import" // import an existing resource.
OpImportReplacement display.StepOp = "import-replacement" // replace an existing resource
// with an imported resource.
)
// StepOps contains the full set of step operation types.
var StepOps = []display.StepOp{
OpSame,
OpCreate,
OpUpdate,
OpDelete,
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
OpReplace,
OpCreateReplacement,
OpDeleteReplaced,
OpRead,
OpReadReplacement,
OpRefresh,
OpReadDiscard,
OpDiscardReplaced,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
OpRemovePendingReplace,
OpImport,
OpImportReplacement,
}
func IsReplacementStep(op display.StepOp) bool {
if op == OpReplace || op == OpCreateReplacement || op == OpDeleteReplaced ||
op == OpReadReplacement || op == OpDiscardReplaced || op == OpRemovePendingReplace ||
op == OpImportReplacement {
return true
}
return false
}
// Color returns a suggested color for lines of this op type.
func Color(op display.StepOp) string {
switch op {
case OpSame:
Switch to parent pointers; display components nicely This change switches from child lists to parent pointers, in the way resource ancestries are represented. This cleans up a fair bit of the old parenting logic, including all notion of ambient parent scopes (and will notably address pulumi/pulumi#435). This lets us show a more parent/child display in the output when doing planning and updating. For instance, here is an update of a lambda's text, which is logically part of a cloud timer: * cloud:timer:Timer: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:timer:Timer::lm-cts-malta-job-CleanSnapshots] * cloud:function:Function: (same) [urn=urn:pulumi:malta::lm-cloud::cloud:function:Function::lm-cts-malta-job-CleanSnapshots] * aws:serverless:Function: (same) [urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots] ~ aws:lambda/function:Function: (modify) [id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741] [urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots] - code : archive(assets:2092f44) { // etc etc etc Note that we still get walls of text, but this will be actually quite nice when combined with pulumi/pulumi#454. I've also suppressed printing properties that didn't change during updates when --detailed was not passed, and also suppressed empty strings and zero-length arrays (since TF uses these as defaults in many places and it just makes creation and deletion quite verbose). Note that this is a far cry from everything we can possibly do here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417). But it's a good start towards taming some of our output spew.
2017-11-17 02:21:41 +00:00
return colors.SpecUnimportant
case OpCreate, OpImport:
return colors.SpecCreate
case OpDelete:
return colors.SpecDelete
case OpUpdate:
return colors.SpecUpdate
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
case OpReplace:
return colors.SpecReplace
case OpCreateReplacement:
return colors.SpecCreateReplacement
case OpDeleteReplaced:
return colors.SpecDeleteReplaced
case OpRead:
return colors.SpecRead
case OpReadReplacement, OpImportReplacement:
return colors.SpecReplace
case OpRefresh:
return colors.SpecUpdate
case OpReadDiscard, OpDiscardReplaced:
return colors.SpecDelete
default:
contract.Failf("Unrecognized resource step op: '%v'", op)
return ""
}
}
// ColorProgress returns a suggested coloring for lines of this of type which
// are progressing.
func ColorProgress(op display.StepOp) string {
return colors.Bold + Color(op)
}
// Prefix returns a suggested prefix for lines of this op type.
func Prefix(op display.StepOp, done bool) string {
var color string
if done {
color = Color(op)
} else {
color = ColorProgress(op)
}
return color + RawPrefix(op)
}
// RawPrefix returns the uncolorized prefix text.
func RawPrefix(op display.StepOp) string {
switch op {
case OpSame:
return " "
case OpCreate:
return "+ "
case OpDelete:
return "- "
case OpUpdate:
return "~ "
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
case OpReplace:
return "+-"
case OpCreateReplacement:
return "++"
case OpDeleteReplaced:
return "--"
case OpRead:
return "> "
case OpReadReplacement:
return ">>"
case OpRefresh:
return "~ "
case OpReadDiscard:
return "< "
case OpDiscardReplaced:
return "<<"
case OpImport:
return "= "
case OpImportReplacement:
return "=>"
default:
contract.Failf("Unrecognized resource step op: %v", op)
return ""
}
}
func PastTense(op display.StepOp) string {
switch op {
case OpSame, OpCreate, OpReplace, OpCreateReplacement, OpUpdate, OpReadReplacement:
return string(op) + "d"
case OpRefresh:
return "refreshed"
case OpRead:
return "read"
case OpReadDiscard, OpDiscardReplaced:
return "discarded"
case OpDelete, OpDeleteReplaced:
return "deleted"
case OpImport, OpImportReplacement:
return "imported"
default:
2017-11-29 18:06:51 +00:00
contract.Failf("Unexpected resource step op: %v", op)
return ""
}
}
// Suffix returns a suggested suffix for lines of this op type.
func Suffix(op display.StepOp) string {
switch op {
case OpCreateReplacement, OpUpdate, OpReplace, OpReadReplacement, OpRefresh, OpImportReplacement:
Implement `get` functions on all resources This change implements the `get` function for resources. Per pulumi/lumi#83, this allows Lumi scripts to actually read from the target environment. For example, we can now look up a SecurityGroup from its ARN: let group = aws.ec2.SecurityGroup.get( "arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79"); The returned object is a fully functional resource object. So, we can then link it up with an EC2 instance, for example, in the usual ways: let instance = new aws.ec2.Instance(..., { securityGroups: [ group ], }); This didn't require any changes to the RPC or provider model, since we already implement the Get function. There are a few loose ends; two are short term: 1) URNs are not rehydrated. 2) Query is not yet implemented. One is mid-term: 3) We probably want a URN-based lookup function. But we will likely wait until we tackle pulumi/lumi#109 before adding this. And one is long term (and subtle): 4) These amount to I/O and are not repeatable! A change in the target environment may cause a script to generate a different plan intermittently. Most likely we want to apply a different kind of deployment "policy" for such scripts. These are inching towards the scripting model of pulumi/lumi#121, which is an entirely different beast than the repeatable immutable infrastructure deployments. Finally, it is worth noting that with this, we have some of the fundamental underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 00:24:00 +00:00
return colors.Reset // updates and replacements colorize individual lines; get has none
}
return ""
}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
// ConstrainedTo returns true if this operation is no more impactful than the constraint.
func ConstrainedTo(op display.StepOp, constraint display.StepOp) bool {
var allowed []display.StepOp
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
switch constraint {
case OpSame, OpDelete, OpRead, OpReadReplacement, OpRefresh, OpReadDiscard, OpDiscardReplaced,
OpRemovePendingReplace, OpImport, OpImportReplacement:
allowed = []display.StepOp{constraint}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpCreate:
allowed = []display.StepOp{OpSame, OpCreate}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpUpdate:
allowed = []display.StepOp{OpSame, OpUpdate}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
case OpReplace, OpCreateReplacement, OpDeleteReplaced:
allowed = []display.StepOp{OpSame, OpUpdate, constraint}
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
for _, candidate := range allowed {
if candidate == op {
return true
}
}
return false
}
// getProvider fetches the provider for the given step.
func getProvider(s Step, override plugin.Provider) (plugin.Provider, error) {
if override != nil {
return override, nil
}
if providers.IsProviderType(s.Type()) {
return s.Deployment().providers, nil
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
ref, err := providers.ParseReference(s.Provider())
if err != nil {
return nil, fmt.Errorf("bad provider reference '%v' for resource %v: %w", s.Provider(), s.URN(), err)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
if providers.IsDenyDefaultsProvider(ref) {
pkg := providers.GetDeniedDefaultProviderPkg(ref)
msg := diag.GetDefaultProviderDenied(s.URN()).Message
return nil, fmt.Errorf(msg, pkg, s.URN())
}
provider, ok := s.Deployment().GetProvider(ref)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
if !ok {
return nil, fmt.Errorf("unknown provider '%v' for resource %v", s.Provider(), s.URN())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 00:50:29 +00:00
}
return provider, nil
}