pulumi/sdk/go/common/resource/plugin/rpc.go

674 lines
24 KiB
Go
Raw Normal View History

// Copyright 2016-2021, Pulumi Corporation.
2018-05-22 19:43:36 +00:00
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package plugin
import (
"fmt"
"reflect"
"sort"
"github.com/pkg/errors"
Remove deprecated Protobufs imports (#15158) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> github.com/golang/protobuf is marked deprecated and I was getting increasingly triggered by the inconsistency of importing the `Empty` type from "github.com/golang/protobuf/ptypes/empty" or "google.golang.org/protobuf/types/known/emptypb" as "pbempty" or "empty" or "emptypb". Similar for the struct type. So this replaces all the Protobufs imports with ones from "google.golang.org/protobuf", normalises the import name to always just be the module name (emptypb), and adds the depguard linter to ensure we don't use the deprecated package anymore. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-17 09:35:20 +00:00
"google.golang.org/protobuf/types/known/structpb"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/archive"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/asset"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
)
// MarshalOptions controls the marshaling of RPC structures.
type MarshalOptions struct {
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
Label string // an optional label for debugging.
SkipNulls bool // true to skip nulls altogether in the resulting map.
KeepUnknowns bool // true if we are keeping unknown values (otherwise we skip them).
RejectUnknowns bool // true if we should return errors on unknown values. Takes precedence over KeepUnknowns.
ElideAssetContents bool // true if we are eliding the contents of assets.
ComputeAssetHashes bool // true if we are computing missing asset hashes on the fly.
KeepSecrets bool // true if we are keeping secrets (otherwise we replace them with their underlying value).
RejectAssets bool // true if we should return errors on Asset and Archive values.
KeepResources bool // true if we are keeping resoures (otherwise we return raw urn).
SkipInternalKeys bool // true to skip internal property keys (keys that start with "__") in the resulting map.
KeepOutputValues bool // true if we are keeping output values.
UpgradeToOutputValues bool // true if secrets and unknowns should be upgraded to output values.
Add asset/archive to conformance tests and fix engine working dir issues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-02 11:32:54 +00:00
WorkingDirectory string // the optional working directory to use when serializing assets & archives.
}
const (
// UnknownBoolValue is a sentinel indicating that a bool property's value is not known, because it depends on
// a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownBoolValue = "1c4a061d-8072-4f0a-a4cb-0ff528b18fe7"
// UnknownNumberValue is a sentinel indicating that a number property's value is not known, because it depends on
// a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownNumberValue = "3eeb2bf0-c639-47a8-9e75-3b44932eb421"
// UnknownStringValue is a sentinel indicating that a string property's value is not known, because it depends on
// a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownStringValue = "04da6b54-80e4-46f7-96ec-b56ff0331ba9"
// UnknownArrayValue is a sentinel indicating that an array property's value is not known, because it depends on
// a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownArrayValue = "6a19a0b0-7e62-4c92-b797-7f8e31da9cc2"
// UnknownAssetValue is a sentinel indicating that an asset property's value is not known, because it depends on
// a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownAssetValue = "030794c1-ac77-496b-92df-f27374a8bd58"
// UnknownArchiveValue is a sentinel indicating that an archive property's value is not known, because it depends
// on a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownArchiveValue = "e48ece36-62e2-4504-bad9-02848725956a"
// UnknownObjectValue is a sentinel indicating that an archive property's value is not known, because it depends
// on a computation with values whose values themselves are not yet known (e.g., dependent upon an output property).
UnknownObjectValue = "dd056dcd-154b-4c76-9bd3-c8f88648b5ff"
)
// MarshalProperties marshals a resource's property map as a "JSON-like" protobuf structure.
func MarshalProperties(props resource.PropertyMap, opts MarshalOptions) (*structpb.Struct, error) {
fields := make(map[string]*structpb.Value)
for _, key := range props.StableKeys() {
v := props[key]
logging.V(9).Infof("Marshaling property for RPC[%s]: %s=%v", opts.Label, key, v)
if opts.SkipNulls && v.IsNull() {
logging.V(9).Infof("Skipping null property for RPC[%s]: %s (as requested)", opts.Label, key)
} else if opts.SkipInternalKeys && resource.IsInternalPropertyKey(key) {
logging.V(9).Infof("Skipping internal property for RPC[%s]: %s (as requested)", opts.Label, key)
} else {
m, err := MarshalPropertyValue(key, v, opts)
if err != nil {
return nil, err
} else if m != nil {
fields[string(key)] = m
}
}
}
return &structpb.Struct{
Fields: fields,
}, nil
Initial support for output properties (1 of 3) This change includes approximately 1/3rd of the change necessary to support output properties, as per pulumi/lumi#90. In short, the runtime now has a new hidden type, Latent<T>, which represents a "speculative" value, whose eventual type will be T, that we can use during evaluation in various ways. Namely, operations against Latent<T>s generally produce new Latent<U>s. During planning, any Latent<T>s that end up in resource properties are transformed into "unknown" property values. An unknown property value is legal only during planning-time activities, such as Check, Name, and InspectChange. As a result, those RPC interfaces have been updated to include lookaside maps indicating which properties have unknown values. My intent is to add some helper functions to make dealing with this circumstance more correct-by-construction. For now, using an unresolved Latent<T> in a conditional will lead to an error. See pulumi/lumi#67. Speculating beyond these -- by supporting iterative planning and application -- is something we want to support eventually, but it makes sense to do that as an additive change beyond this initial support. That is a missing 1/3. Finally, the other missing 1/3rd which will happen much sooner than the rest is restructuing plan application so that it will correctly observe resolution of Latent<T> values. Right now, the evaluation happens in one single pass, prior to the application, and so Latent<T>s never actually get witnessed in a resolved state.
2017-05-24 00:32:59 +00:00
}
// MarshalPropertyValue marshals a single resource property value into its "JSON-like" value representation.
func MarshalPropertyValue(key resource.PropertyKey, v resource.PropertyValue,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
opts MarshalOptions,
) (*structpb.Value, error) {
if v.IsNull() {
return MarshalNull(opts), nil
} else if v.IsBool() {
return &structpb.Value{
Kind: &structpb.Value_BoolValue{
2017-02-28 19:02:33 +00:00
BoolValue: v.BoolValue(),
},
}, nil
} else if v.IsNumber() {
return &structpb.Value{
Kind: &structpb.Value_NumberValue{
2017-02-28 19:02:33 +00:00
NumberValue: v.NumberValue(),
},
}, nil
} else if v.IsString() {
return MarshalString(v.StringValue(), opts), nil
} else if v.IsArray() {
var elems []*structpb.Value
for _, elem := range v.ArrayValue() {
e, err := MarshalPropertyValue(key, elem, opts)
if err != nil {
return nil, err
}
if e != nil {
elems = append(elems, e)
}
}
return &structpb.Value{
Kind: &structpb.Value_ListValue{
2017-02-28 19:02:33 +00:00
ListValue: &structpb.ListValue{Values: elems},
},
}, nil
} else if v.IsAsset() {
if opts.RejectAssets {
return nil, fmt.Errorf("unexpected Asset property value for %q", key)
}
return MarshalAsset(v.AssetValue(), opts)
} else if v.IsArchive() {
if opts.RejectAssets {
return nil, fmt.Errorf("unexpected Asset Archive property value for %q", key)
}
return MarshalArchive(v.ArchiveValue(), opts)
} else if v.IsObject() {
obj, err := MarshalProperties(v.ObjectValue(), opts)
if err != nil {
return nil, err
}
return MarshalStruct(obj, opts), nil
} else if v.IsComputed() {
if opts.RejectUnknowns {
return nil, fmt.Errorf("unexpected unknown property value for %q", key)
} else if opts.KeepUnknowns {
if opts.KeepOutputValues && opts.UpgradeToOutputValues {
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
output := resource.NewObjectProperty(resource.PropertyMap{
resource.SigKey: resource.NewStringProperty(resource.OutputValueSig),
})
return MarshalPropertyValue(key, output, opts)
}
return marshalUnknownProperty(v.Input().Element, opts), nil
}
return nil, nil // return nil and the caller will ignore it.
} else if v.IsOutput() {
if !opts.KeepOutputValues {
result := v.OutputValue().Element
if !v.OutputValue().Known {
// Unknown outputs are marshaled the same as Computed.
Enable output values by default (#8014) * Enable output values by default Enable output values by default in the resource monitor and change the polarity of the envvar from `PULUMI_ENABLE_OUTPUT_VALUES` to `PULUMI_DISABLE_OUTPUT_VALUES`. * Marshal unknown as unknown string when `!KeepOutputValues` Marshal all unknown output values as `resource.MakeComputed(resource.NewStringProperty(""))` when not keeping output values, which is consistent with what the SDKs do. Otherwise, when `v.OutputValue().Element` is nil, `resource.MakeComputed(v.OutputValue().Element)` will be marshaled as a null value rather than as an unknown sentinel. * Add MarshalOptions.DontSkipOutputs and use where needed Before we expanded the meaning of `resource.Output`, `MarshalProperties` always skipped output values: ```go if v.IsOutput() { logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key) } ``` As part of expanding the meaning of `resource.Output`, I'd adjusted `MarshalProperties` to only skip output values when the value was unknown and when not keeping output values: ```go if v.IsOutput() && !v.OutputValue().Known && !opts.KeepOutputValues { logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key) } ``` However, this doesn't work the way we want when marshaling properties that include unknown output values to a provider that does not accept outputs. In that case, `opts.KeepOutputValues` will be `false` because we want the marshaler to fall back to returning non-output-values (e.g. unknown sentinel value for unknown output values), but instead of getting the intended fallback values, the unknown output values are skipped (not what we want). I suspect we may be able to delete the output value skipping in `MarshalProperties` altogether (it's odd that it is skipping `resource.Output` but not `resource.Computed`), but to avoid any unintended side effects of doing that, instead, this commit introduces a new `MarshalOptions.DontSkipOutputs` option that can be set to `true` to opt-in to not skipping output values when marshaling. The check in `MarshalProperties` now looks like this: ```go if !opts.DontSkipOutputs && v.IsOutput() && !v.OutputValue().Known { logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key) } ``` `opts.DontSkipOutputs` is set to `true` when marshaling properties for calls to a provider's `Construct` and `Call`. * [sdk/nodejs] Deserialize output values This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK. * [sdk/python] Deserialize output values This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
2021-09-24 15:57:04 +00:00
result = resource.MakeComputed(resource.NewStringProperty(""))
}
if v.OutputValue().Secret {
result = resource.MakeSecret(result)
}
return MarshalPropertyValue(key, result, opts)
}
obj := resource.PropertyMap{
resource.SigKey: resource.NewStringProperty(resource.OutputValueSig),
}
if v.OutputValue().Known {
obj["value"] = v.OutputValue().Element
}
if v.OutputValue().Secret {
obj["secret"] = resource.NewBoolProperty(v.OutputValue().Secret)
}
if len(v.OutputValue().Dependencies) > 0 {
deps := make([]resource.PropertyValue, len(v.OutputValue().Dependencies))
for i, dep := range v.OutputValue().Dependencies {
deps[i] = resource.NewStringProperty(string(dep))
}
obj["dependencies"] = resource.NewArrayProperty(deps)
}
output := resource.NewObjectProperty(obj)
return MarshalPropertyValue(key, output, opts)
} else if v.IsSecret() {
if !opts.KeepSecrets {
logging.V(5).Infof("marshalling secret value as raw value as opts.KeepSecrets is false")
return MarshalPropertyValue(key, v.SecretValue().Element, opts)
}
if opts.KeepOutputValues && opts.UpgradeToOutputValues {
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
output := resource.NewObjectProperty(resource.PropertyMap{
resource.SigKey: resource.NewStringProperty(resource.OutputValueSig),
"secret": resource.NewBoolProperty(true),
"value": v.SecretValue().Element,
})
return MarshalPropertyValue(key, output, opts)
}
secret := resource.NewObjectProperty(resource.PropertyMap{
resource.SigKey: resource.NewStringProperty(resource.SecretSig),
"value": v.SecretValue().Element,
})
return MarshalPropertyValue(key, secret, opts)
} else if v.IsResourceReference() {
ref := v.ResourceReferenceValue()
if !opts.KeepResources {
val := string(ref.URN)
if !ref.ID.IsNull() {
return MarshalPropertyValue(key, ref.ID, opts)
}
logging.V(5).Infof("marshalling resource value as raw URN or ID as opts.KeepResources is false")
return MarshalString(val, opts), nil
}
m := resource.PropertyMap{
resource.SigKey: resource.NewStringProperty(resource.ResourceReferenceSig),
"urn": resource.NewStringProperty(string(ref.URN)),
}
if id, hasID := ref.IDString(); hasID {
m["id"] = resource.NewStringProperty(id)
}
if ref.PackageVersion != "" {
m["packageVersion"] = resource.NewStringProperty(ref.PackageVersion)
}
return MarshalPropertyValue(key, resource.NewObjectProperty(m), opts)
}
contract.Failf("Unrecognized property value in RPC[%s] for %q: %v (type=%v)",
opts.Label, key, v.V, reflect.TypeOf(v.V))
return nil, nil
}
// marshalUnknownProperty marshals an unknown property in a way that lets us recover its type on the other end.
func marshalUnknownProperty(elem resource.PropertyValue, opts MarshalOptions) *structpb.Value {
// Normal cases, these get sentinels.
if elem.IsBool() {
return MarshalString(UnknownBoolValue, opts)
} else if elem.IsNumber() {
return MarshalString(UnknownNumberValue, opts)
} else if elem.IsString() {
return MarshalString(UnknownStringValue, opts)
} else if elem.IsArray() {
return MarshalString(UnknownArrayValue, opts)
} else if elem.IsAsset() {
return MarshalString(UnknownAssetValue, opts)
} else if elem.IsArchive() {
return MarshalString(UnknownArchiveValue, opts)
} else if elem.IsObject() {
return MarshalString(UnknownObjectValue, opts)
}
// If for some reason we end up with a recursive computed/output, just keep digging.
if elem.IsComputed() {
return marshalUnknownProperty(elem.Input().Element, opts)
} else if elem.IsOutput() {
return marshalUnknownProperty(elem.OutputValue().Element, opts)
}
// Finally, if a null, we can guess its value! (the one and only...)
if elem.IsNull() {
return MarshalNull(opts)
}
contract.Failf("Unexpected output/computed property element in RPC[%s]: %v", opts.Label, elem)
return nil
}
// UnmarshalProperties unmarshals a "JSON-like" protobuf structure into a new resource property map.
func UnmarshalProperties(props *structpb.Struct, opts MarshalOptions) (resource.PropertyMap, error) {
result := make(resource.PropertyMap)
// First sort the keys so we enumerate them in order (in case errors happen, we want determinism).
var keys []string
2017-07-15 01:08:33 +00:00
if props != nil {
for k := range props.Fields {
keys = append(keys, k)
}
sort.Strings(keys)
}
// And now unmarshal every field it into the map.
for _, key := range keys {
pk := resource.PropertyKey(key)
v, err := UnmarshalPropertyValue(pk, props.Fields[key], opts)
if err != nil {
return nil, err
} else if v != nil {
logging.V(9).Infof("Unmarshaling property for RPC[%s]: %s=%v", opts.Label, key, v)
if opts.SkipNulls && v.IsNull() {
logging.V(9).Infof("Skipping unmarshaling for RPC[%s]: %s is null", opts.Label, key)
} else if opts.SkipInternalKeys && resource.IsInternalPropertyKey(pk) {
logging.V(9).Infof("Skipping unmarshaling for RPC[%s]: %s is internal", opts.Label, key)
} else {
result[pk] = *v
}
}
}
return result, nil
}
// UnmarshalPropertyValue unmarshals a single "JSON-like" value into a new property value.
func UnmarshalPropertyValue(key resource.PropertyKey, v *structpb.Value,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
opts MarshalOptions,
) (*resource.PropertyValue, error) {
contract.Assertf(v != nil, "a value is required")
switch v.Kind.(type) {
case *structpb.Value_NullValue:
m := resource.NewNullProperty()
return &m, nil
case *structpb.Value_BoolValue:
m := resource.NewBoolProperty(v.GetBoolValue())
return &m, nil
case *structpb.Value_NumberValue:
m := resource.NewNumberProperty(v.GetNumberValue())
return &m, nil
case *structpb.Value_StringValue:
// If it's a string, it could be an unknown property, or just a regular string.
s := v.GetStringValue()
if unk, isunk := unmarshalUnknownPropertyValue(s, opts); isunk {
if opts.RejectUnknowns {
return nil, fmt.Errorf("unexpected unknown property value for %q", key)
} else if opts.KeepUnknowns {
return &unk, nil
}
return nil, nil
}
m := resource.NewStringProperty(s)
return &m, nil
case *structpb.Value_ListValue:
lst := v.GetListValue()
gRPC bridge: fix unknowns in `Update` previews (#6006) These changes are a combination of three commits, each of which contributes to the testing and/or fixing of a problem with marshaling unknowns in `plugin.provider.Update` when `preview` is true. ## deploytest: add support for gRPC adapters. These changes add support for communicating with providers using the gRPC adapters to the deploytest pacakage. This makes it easier to test the gRPC adapters across typical lifecycle patterns. Supporting these changes are two additions to the `resource/plugin` package: 1. A type that bridges between the `plugin.Provider` interface and the `pulumirpc.ResourceProviderServer` 2. A function to create a `plugin.Provider` given a `pulumirpc.ResourceProviderClient` The deploytest package uses these to wrap an in-process `plugin.Provider` in a gRPC interface and connect to it without using the default plugin host, respectively. ## pulumi_test: test provider preview over gRPC. Add a test that runs the provider preview lifecycle, but using a provider that communicates over gRPC. ## gRPC bridge: fix unknowns in `Update` previews Set the `KeepUnknowns` and `RejectUnknowns` bits in the `MarshalOptions` used when unmarshaling update results to preserve unknowns during a preview and reject them otherwise. These changes also set the `RejectUnknowns` bit in the `MarshalOptions` used by `Create` if `preview` is false, and fix a bug in the array unmarshaler that could cause out-of-bounds accesses. Fixes https://github.com/pulumi/pulumi/issues/6004.
2020-12-23 21:25:48 +00:00
elems := make([]resource.PropertyValue, len(lst.GetValues()))
for i, elem := range lst.GetValues() {
e, err := UnmarshalPropertyValue(key, elem, opts)
if err != nil {
return nil, err
} else if e != nil {
if i == len(elems) {
elems = append(elems, *e)
} else {
elems[i] = *e
}
}
}
m := resource.NewArrayProperty(elems)
return &m, nil
case *structpb.Value_StructValue:
// Start by unmarshaling.
obj, err := UnmarshalProperties(v.GetStructValue(), opts)
if err != nil {
return nil, err
}
// Before returning it as an object, check to see if it's a known recoverable type.
objmap := obj.Mappable()
sig, hasSig := objmap[string(resource.SigKey)]
if !hasSig {
// This is a weakly-typed object map.
m := resource.NewObjectProperty(obj)
return &m, nil
}
switch sig {
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
case asset.AssetSig:
if opts.RejectAssets {
return nil, fmt.Errorf("unexpected Asset property value for %q", key)
}
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
asset, isasset, err := asset.Deserialize(objmap)
if err != nil {
return nil, err
}
// This can only be false with a non-nil error if there is a signature match. We've already verified the
// signature.
contract.Assertf(isasset, "value must be an asset")
if opts.ComputeAssetHashes {
Add asset/archive to conformance tests and fix engine working dir issues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-02 11:32:54 +00:00
if opts.WorkingDirectory == "" {
if err = asset.EnsureHash(); err != nil {
return nil, errors.Wrapf(err, "failed to compute asset hash for %q", key)
}
} else {
if err = asset.EnsureHashWithWD(opts.WorkingDirectory); err != nil {
return nil, errors.Wrapf(err, "failed to compute asset hash for %q", key)
}
}
}
m := resource.NewAssetProperty(asset)
return &m, nil
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
case archive.ArchiveSig:
if opts.RejectAssets {
return nil, fmt.Errorf("unexpected Asset Archive property value for %q", key)
}
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
archive, isarchive, err := archive.Deserialize(objmap)
if err != nil {
return nil, err
}
// This can only be false with a non-nil error if there is a signature match. We've already verified the
// signature.
contract.Assertf(isarchive, "value must be an archive")
if opts.ComputeAssetHashes {
Add asset/archive to conformance tests and fix engine working dir issues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-02 11:32:54 +00:00
if opts.WorkingDirectory == "" {
if err = archive.EnsureHash(); err != nil {
return nil, errors.Wrapf(err, "failed to compute archive hash for %q", key)
}
} else {
if err = archive.EnsureHashWithWD(opts.WorkingDirectory); err != nil {
return nil, errors.Wrapf(err, "failed to compute archive hash for %q", key)
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
}
}
m := resource.NewArchiveProperty(archive)
return &m, nil
case resource.SecretSig:
value, ok := obj["value"]
if !ok {
return nil, fmt.Errorf("malformed RPC secret: missing value for %q", key)
}
return unmarshalSecretPropertyValue(value, opts), nil
case resource.ResourceReferenceSig:
urn, ok := obj["urn"]
if !ok {
return nil, fmt.Errorf("malformed resource reference for %q: missing urn", key)
}
if !urn.IsString() {
return nil, fmt.Errorf("malformed resource reference for %q: urn not a string", key)
}
id, hasID := "", false
if idProp, ok := obj["id"]; ok {
hasID = true
switch {
case idProp.IsString():
id = idProp.StringValue()
case idProp.IsComputed():
// Leave the ID empty to indicate that it is unknown.
case idProp.IsOutput():
if idProp.OutputValue().Known {
if !idProp.OutputValue().Element.IsString() {
return nil, fmt.Errorf("malformed resource reference for %q: id not a string", key)
}
id = idProp.OutputValue().Element.StringValue()
}
default:
return nil, fmt.Errorf("malformed resource reference for %q: id not a string", key)
}
}
var packageVersion string
if packageVersionProp, ok := obj["packageVersion"]; ok {
if !packageVersionProp.IsString() {
return nil, fmt.Errorf("malformed resource reference for %q: packageVersion not a string", key)
}
packageVersion = packageVersionProp.StringValue()
}
if !opts.KeepResources {
value := urn.StringValue()
if hasID {
isIDUnknown := id == ""
if isIDUnknown && opts.KeepUnknowns {
v := structpb.Value{
Kind: &structpb.Value_StringValue{StringValue: UnknownStringValue},
}
return UnmarshalPropertyValue(key, &v, opts)
}
value = id
}
r := resource.NewStringProperty(value)
return &r, nil
}
var ref resource.PropertyValue
if hasID {
ref = resource.MakeCustomResourceReference(resource.URN(urn.StringValue()), resource.ID(id), packageVersion)
} else {
ref = resource.MakeComponentResourceReference(resource.URN(urn.StringValue()), packageVersion)
}
return &ref, nil
case resource.OutputValueSig:
value, known := obj["value"]
var secret bool
if secretProp, ok := obj["secret"]; ok {
if !secretProp.IsBool() {
return nil, fmt.Errorf("malformed output value for %q: secret not a bool", key)
}
secret = secretProp.BoolValue()
}
if !opts.KeepOutputValues {
result := &value
if !known {
result, err = UnmarshalPropertyValue(key, &structpb.Value{
Kind: &structpb.Value_StringValue{StringValue: UnknownStringValue},
}, opts)
if err != nil {
return nil, err
}
}
if secret && result != nil {
result = unmarshalSecretPropertyValue(*result, opts)
}
return result, nil
}
var dependencies []resource.URN
if dependenciesProp, ok := obj["dependencies"]; ok {
if !dependenciesProp.IsArray() {
return nil, fmt.Errorf("malformed output value for %q: dependencies not an array", key)
}
dependencies = make([]resource.URN, len(dependenciesProp.ArrayValue()))
for i, dep := range dependenciesProp.ArrayValue() {
if !dep.IsString() {
return nil, fmt.Errorf(
"malformed output value for %q: element in dependencies not a string", key)
}
dependencies[i] = resource.URN(dep.StringValue())
}
}
output := resource.NewOutputProperty(resource.Output{
Element: value,
Known: known,
Secret: secret,
Dependencies: dependencies,
})
return &output, nil
default:
return nil, fmt.Errorf("unrecognized signature '%v' in property map for %q", sig, key)
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 17:46:30 +00:00
default:
contract.Failf("Unrecognized structpb value kind in RPC[%s] for %q: %v", opts.Label, key, reflect.TypeOf(v.Kind))
return nil, nil
}
}
func unmarshalUnknownPropertyValue(s string, opts MarshalOptions) (resource.PropertyValue, bool) {
var elem resource.PropertyValue
var unknown bool
switch s {
case UnknownBoolValue:
elem, unknown = resource.NewBoolProperty(false), true
case UnknownNumberValue:
elem, unknown = resource.NewNumberProperty(0), true
case UnknownStringValue:
elem, unknown = resource.NewStringProperty(""), true
case UnknownArrayValue:
elem, unknown = resource.NewArrayProperty([]resource.PropertyValue{}), true
case UnknownAssetValue:
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
elem, unknown = resource.NewAssetProperty(&asset.Asset{}), true
case UnknownArchiveValue:
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
elem, unknown = resource.NewArchiveProperty(&archive.Archive{}), true
case UnknownObjectValue:
elem, unknown = resource.NewObjectProperty(make(resource.PropertyMap)), true
}
if unknown {
if opts.KeepOutputValues && opts.UpgradeToOutputValues {
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
return resource.NewOutputProperty(resource.Output{
Element: elem,
}), true
}
comp := resource.Computed{Element: elem}
return resource.NewComputedProperty(comp), true
}
return resource.PropertyValue{}, false
}
func unmarshalSecretPropertyValue(v resource.PropertyValue, opts MarshalOptions) *resource.PropertyValue {
if !opts.KeepSecrets {
logging.V(5).Infof("unmarshalling secret as raw value, as opts.KeepSecrets is false")
return &v
}
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
var s resource.PropertyValue
if opts.KeepOutputValues && opts.UpgradeToOutputValues {
Add UpgradeToOutputValues to MarshalOptions (#15349) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new option `UpgradeToOutputValues` to `MarshalOptions`. When set all `Computed` and `Unknown` values are _upgraded_ to `Output` values. This reduces the number of cases that need to be handled later in the codebase. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-02 11:09:58 +00:00
s = resource.NewOutputProperty(resource.Output{
Element: v,
Secret: true,
Known: true,
})
} else {
s = resource.MakeSecret(v)
}
return &s
}
// MarshalNull marshals a nil to its protobuf form.
func MarshalNull(opts MarshalOptions) *structpb.Value {
return &structpb.Value{
Kind: &structpb.Value_NullValue{
NullValue: structpb.NullValue_NULL_VALUE,
},
}
}
// MarshalString marshals a string to its protobuf form.
func MarshalString(s string, opts MarshalOptions) *structpb.Value {
return &structpb.Value{
Kind: &structpb.Value_StringValue{
StringValue: s,
},
}
}
// MarshalStruct marshals a struct for use in a protobuf field where a value is expected.
func MarshalStruct(obj *structpb.Struct, opts MarshalOptions) *structpb.Value {
return &structpb.Value{
Kind: &structpb.Value_StructValue{
StructValue: obj,
},
}
}
// MarshalAsset marshals an asset into its wire form for resource provider plugins.
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
func MarshalAsset(v *asset.Asset, opts MarshalOptions) (*structpb.Value, error) {
// If we are not providing access to an asset's contents, we simply need to record the fact that this asset existed.
// Serialize the asset with only its hash (if present).
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
if opts.ElideAssetContents {
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
v = &asset.Asset{Hash: v.Hash}
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
} else {
// Ensure a hash is present if needed.
if v.Hash == "" && opts.ComputeAssetHashes {
Add asset/archive to conformance tests and fix engine working dir issues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-02 11:32:54 +00:00
if opts.WorkingDirectory == "" {
if err := v.EnsureHash(); err != nil {
return nil, errors.Wrapf(err, "failed to compute asset hash")
}
} else {
if err := v.EnsureHashWithWD(opts.WorkingDirectory); err != nil {
return nil, errors.Wrapf(err, "failed to compute asset hash")
}
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
}
}
}
// To marshal an asset, we need to first serialize it, and then marshal that.
sera := v.Serialize()
serap := resource.NewPropertyMapFromMap(sera)
pk := resource.PropertyKey(v.URI)
return MarshalPropertyValue(pk, resource.NewObjectProperty(serap), opts)
}
// MarshalArchive marshals an archive into its wire form for resource provider plugins.
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
func MarshalArchive(v *archive.Archive, opts MarshalOptions) (*structpb.Value, error) {
// If we are not providing access to an asset's contents, we simply need to record the fact that this asset existed.
// Serialize the asset with only its hash (if present).
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
if opts.ElideAssetContents {
Move assets and archives to their own package (#15157) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description This PR is motivated by https://github.com/pulumi/pulumi/pull/15145. `resource.*` should be built on top of `property.Value`,[^1] which means that `resource` needs to be able to import `property.Value`, and so `property` cannot import `resource`. Since Assets and Archives are both types of properties, they must be moved out of `resource`. [^1]: For example: https://github.com/pulumi/pulumi/blob/a1d686227cd7e3c70c51bd772450cb0cd57c1479/sdk/go/common/resource/resource_state.go#L35-L36 ## Open Question This PR moves them to their own sub-folders in `resource`. Should `asset` and `archive` live somewhere more high level, like `sdk/go/property/{asset,archive}`? <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> ## Checklist - [ ] I have run `make tidy` to update any new dependencies - [ ] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-25 20:39:31 +00:00
v = &archive.Archive{Hash: v.Hash}
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
} else {
// Ensure a hash is present if needed.
if v.Hash == "" && opts.ComputeAssetHashes {
Add asset/archive to conformance tests and fix engine working dir issues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-05-02 11:32:54 +00:00
if opts.WorkingDirectory == "" {
if err := v.EnsureHash(); err != nil {
return nil, errors.Wrapf(err, "failed to compute archive hash")
}
} else {
if err := v.EnsureHashWithWD(opts.WorkingDirectory); err != nil {
return nil, errors.Wrapf(err, "failed to compute archive hash")
}
Rework asset identity and exposure of old assets. (#548) Note: for the purposes of this discussion, archives will be treated as assets, as their differences are not particularly meaningful. Currently, the identity of an asset is derived from the hash and the location of its contents (i.e. two assets are equal iff their contents have the same hash and the same path/URI/inline value). This means that changing the source of an asset will cause the engine to detect a difference in the asset even if the source's contents are identical. At best, this leads to inefficiencies such as unnecessary updates. This commit changes asset identity so that it is derived solely from an asset's hash. The source of an asset's contents is no longer part of the asset's identity, and need only be provided if the contents themselves may need to be available (e.g. if a hash does not yet exist for the asset or if the asset's contents might be needed for an update). This commit also changes the way old assets are exposed to providers. Currently, an old asset is exposed as both its hash and its contents. This allows providers to take a dependency on the contents of an old asset being available, even though this is not an invariant of the system. These changes remove the contents of old assets from their serialized form when they are passed to providers, eliminating the ability of a provider to take such a dependency. In combination with the changes to asset identity, this allows a provider to detect changes to an asset simply by comparing its old and new hashes. This is half of the fix for [pulumi/pulumi-cloud#158]. The other half involves changes in [pulumi/pulumi-terraform].
2017-11-12 19:45:13 +00:00
}
}
}
// To marshal an archive, we need to first serialize it, and then marshal that.
sera := v.Serialize()
serap := resource.NewPropertyMapFromMap(sera)
pk := resource.PropertyKey(v.URI)
return MarshalPropertyValue(pk, resource.NewObjectProperty(serap), opts)
}