pulumi/pkg/engine/lifecycletest/test_plan.go

506 lines
14 KiB
Go
Raw Permalink Normal View History

2021-09-21 17:00:44 +00:00
//nolint:revive
package lifecycletest
import (
"context"
"errors"
"reflect"
"testing"
"github.com/blang/semver"
"github.com/mitchellh/copystructure"
"github.com/stretchr/testify/assert"
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
"github.com/stretchr/testify/require"
"github.com/pulumi/pulumi/pkg/v3/display"
. "github.com/pulumi/pulumi/pkg/v3/engine"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy"
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/deploytest"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/providers"
"github.com/pulumi/pulumi/pkg/v3/util/cancel"
Add a strict opinionated promise library (#14552) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by having another case in https://github.com/pulumi/pulumi/pull/14548 that looked like "this is just a promise". We already had a mini-promise library for provider config in sdk/go/common/resource/plugin/provider_plugin.go. Plus a couple of places where we did a poor mans Go promise by having a WaitGroup plus a variable to set (see the two changes in /pkg). This adds a little promise library to safely cover all three of these cases plus the case I'll be adding in #14548. It is _minimal_ adding just the features of promises needed for these initial cases. We can add to it as needed. I suspect we've got a number of places in test code that could probably use this as well, but I haven't gone through that. I also suspect that having this type available will result in more places in the future being simpler because "its just a promise" is a fairly common scenario in async systems. In fact Output's internally _are just a promise_ so we could probably rewrite their internals using this, add a `Then()` like method for apply and all the async complexity gets handled in the promise layer while the output layer just cares about unknowns/secrets/etc. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - This is in sdk/go/common and I'm considering it an non-public api for now. <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 14:53:12 +00:00
"github.com/pulumi/pulumi/sdk/v3/go/common/promise"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/config"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
"github.com/pulumi/pulumi/sdk/v3/go/common/slice"
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/result"
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
)
type updateInfo struct {
project workspace.Project
target deploy.Target
}
func (u *updateInfo) GetRoot() string {
// These tests run in-memory, so we don't have a real root. Just pretend we're at the filesystem root.
return "/"
}
func (u *updateInfo) GetProject() *workspace.Project {
return &u.project
}
func (u *updateInfo) GetTarget() *deploy.Target {
return &u.target
}
func ImportOp(imports []deploy.Import) TestOp {
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
return TestOp(func(info UpdateInfo, ctx *Context, opts UpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
dryRun bool,
) (*deploy.Plan, display.ResourceChanges, error) {
return Import(info, ctx, opts, imports, dryRun)
})
}
type TestOp func(UpdateInfo, *Context, UpdateOptions, bool) (*deploy.Plan, display.ResourceChanges, error)
type ValidateFunc func(project workspace.Project, target deploy.Target, entries JournalEntries,
events []Event, err error) error
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
func (op TestOp) Plan(project workspace.Project, target deploy.Target, opts TestUpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Plan, error) {
plan, _, err := op.runWithContext(context.Background(), project, target, opts, true, backendClient, validate)
return plan, err
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
func (op TestOp) Run(project workspace.Project, target deploy.Target, opts TestUpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
dryRun bool, backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Snapshot, error) {
return op.RunWithContext(context.Background(), project, target, opts, dryRun, backendClient, validate)
}
func (op TestOp) RunWithContext(
callerCtx context.Context, project workspace.Project,
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
target deploy.Target, opts TestUpdateOptions, dryRun bool,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Snapshot, error) {
_, snap, err := op.runWithContext(callerCtx, project, target, opts, dryRun, backendClient, validate)
return snap, err
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
func (op TestOp) runWithContext(
callerCtx context.Context, project workspace.Project,
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
target deploy.Target, opts TestUpdateOptions, dryRun bool,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Plan, *deploy.Snapshot, error) {
// Create an appropriate update info and context.
info := &updateInfo{project: project, target: target}
cancelCtx, cancelSrc := cancel.NewContext(context.Background())
done := make(chan bool)
defer close(done)
go func() {
select {
case <-callerCtx.Done():
cancelSrc.Cancel()
case <-done:
}
}()
events := make(chan Event)
journal := NewJournal()
ctx := &Context{
Cancel: cancelCtx,
Events: events,
SnapshotManager: journal,
BackendClient: backendClient,
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
updateOpts := opts.Options()
defer func() {
if updateOpts.Host != nil {
contract.IgnoreClose(updateOpts.Host)
}
}()
// Begin draining events.
Add a strict opinionated promise library (#14552) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by having another case in https://github.com/pulumi/pulumi/pull/14548 that looked like "this is just a promise". We already had a mini-promise library for provider config in sdk/go/common/resource/plugin/provider_plugin.go. Plus a couple of places where we did a poor mans Go promise by having a WaitGroup plus a variable to set (see the two changes in /pkg). This adds a little promise library to safely cover all three of these cases plus the case I'll be adding in #14548. It is _minimal_ adding just the features of promises needed for these initial cases. We can add to it as needed. I suspect we've got a number of places in test code that could probably use this as well, but I haven't gone through that. I also suspect that having this type available will result in more places in the future being simpler because "its just a promise" is a fairly common scenario in async systems. In fact Output's internally _are just a promise_ so we could probably rewrite their internals using this, add a `Then()` like method for apply and all the async complexity gets handled in the promise layer while the output layer just cares about unknowns/secrets/etc. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - This is in sdk/go/common and I'm considering it an non-public api for now. <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 14:53:12 +00:00
firedEventsPromise := promise.Run(func() ([]Event, error) {
var firedEvents []Event
for e := range events {
firedEvents = append(firedEvents, e)
}
Add a strict opinionated promise library (#14552) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by having another case in https://github.com/pulumi/pulumi/pull/14548 that looked like "this is just a promise". We already had a mini-promise library for provider config in sdk/go/common/resource/plugin/provider_plugin.go. Plus a couple of places where we did a poor mans Go promise by having a WaitGroup plus a variable to set (see the two changes in /pkg). This adds a little promise library to safely cover all three of these cases plus the case I'll be adding in #14548. It is _minimal_ adding just the features of promises needed for these initial cases. We can add to it as needed. I suspect we've got a number of places in test code that could probably use this as well, but I haven't gone through that. I also suspect that having this type available will result in more places in the future being simpler because "its just a promise" is a fairly common scenario in async systems. In fact Output's internally _are just a promise_ so we could probably rewrite their internals using this, add a `Then()` like method for apply and all the async complexity gets handled in the promise layer while the output layer just cares about unknowns/secrets/etc. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - This is in sdk/go/common and I'm considering it an non-public api for now. <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 14:53:12 +00:00
return firedEvents, nil
})
// Run the step and its validator.
plan, _, opErr := op(info, ctx, updateOpts, dryRun)
close(events)
closeErr := journal.Close()
// Wait for the events to finish. You'd think this would cancel with the callerCtx but tests explicitly use that for
// the deployment context, not expecting it to have any effect on the test code here. See
// https://github.com/pulumi/pulumi/issues/14588 for what happens if you try to use callerCtx here.
firedEvents, err := firedEventsPromise.Result(context.Background())
Add a strict opinionated promise library (#14552) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by having another case in https://github.com/pulumi/pulumi/pull/14548 that looked like "this is just a promise". We already had a mini-promise library for provider config in sdk/go/common/resource/plugin/provider_plugin.go. Plus a couple of places where we did a poor mans Go promise by having a WaitGroup plus a variable to set (see the two changes in /pkg). This adds a little promise library to safely cover all three of these cases plus the case I'll be adding in #14548. It is _minimal_ adding just the features of promises needed for these initial cases. We can add to it as needed. I suspect we've got a number of places in test code that could probably use this as well, but I haven't gone through that. I also suspect that having this type available will result in more places in the future being simpler because "its just a promise" is a fairly common scenario in async systems. In fact Output's internally _are just a promise_ so we could probably rewrite their internals using this, add a `Then()` like method for apply and all the async complexity gets handled in the promise layer while the output layer just cares about unknowns/secrets/etc. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - This is in sdk/go/common and I'm considering it an non-public api for now. <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 14:53:12 +00:00
if err != nil {
return nil, nil, err
}
if validate != nil {
opErr = validate(project, target, journal.Entries(), firedEvents, opErr)
}
errs := []error{opErr, closeErr}
if dryRun {
return plan, nil, errors.Join(errs...)
}
Verify partial journals in engine tests (#15018) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This updates test_plan.go to verify that every partial snapshot is also a valid snapshot. This tests that the journal always writes out _valid_ snapshots even for scenarios where an update only partially ran. While doing this it flagged up that the validation code was wrong for delete-before-create resources. When we do a delete-before-create replacement we need to leave the "being replaced" resource in the snapshot. This is because while that resource is being replaced it might have dependents that are not being replaced, and so there is a period of time where _it_ is deleted but dependents are still referring to it. If we removed it from the snapshot at this point those dependent resources would not validate. But in the case of two resources both being delete-before-create replaced you can have a scenario where both get deleted (but are still in the snapshot file) and the first then gets created thus removing it's old entry from the snapshot file and adding it's new state. If the first resource was a provider that will change it's ID, thus making the provider reference in the second resource now look invalid.
2024-01-05 23:16:40 +00:00
entries := journal.Entries()
// Check that each possible snapshot we could have created is valid
var snap *deploy.Snapshot
for i := 0; i <= len(entries); i++ {
var err error
snap, err = entries[0:i].Snap(target.Snapshot)
if err != nil {
// if any snapshot fails to create just return this error, don't keep going
errs = append(errs, err)
snap = nil
break
}
err = snap.VerifyIntegrity()
if err != nil {
Verify partial journals in engine tests (#15018) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This updates test_plan.go to verify that every partial snapshot is also a valid snapshot. This tests that the journal always writes out _valid_ snapshots even for scenarios where an update only partially ran. While doing this it flagged up that the validation code was wrong for delete-before-create resources. When we do a delete-before-create replacement we need to leave the "being replaced" resource in the snapshot. This is because while that resource is being replaced it might have dependents that are not being replaced, and so there is a period of time where _it_ is deleted but dependents are still referring to it. If we removed it from the snapshot at this point those dependent resources would not validate. But in the case of two resources both being delete-before-create replaced you can have a scenario where both get deleted (but are still in the snapshot file) and the first then gets created thus removing it's old entry from the snapshot file and adding it's new state. If the first resource was a provider that will change it's ID, thus making the provider reference in the second resource now look invalid.
2024-01-05 23:16:40 +00:00
// Likewise as soon as one snapshot fails to validate stop checking
errs = append(errs, err)
Verify partial journals in engine tests (#15018) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This updates test_plan.go to verify that every partial snapshot is also a valid snapshot. This tests that the journal always writes out _valid_ snapshots even for scenarios where an update only partially ran. While doing this it flagged up that the validation code was wrong for delete-before-create resources. When we do a delete-before-create replacement we need to leave the "being replaced" resource in the snapshot. This is because while that resource is being replaced it might have dependents that are not being replaced, and so there is a period of time where _it_ is deleted but dependents are still referring to it. If we removed it from the snapshot at this point those dependent resources would not validate. But in the case of two resources both being delete-before-create replaced you can have a scenario where both get deleted (but are still in the snapshot file) and the first then gets created thus removing it's old entry from the snapshot file and adding it's new state. If the first resource was a provider that will change it's ID, thus making the provider reference in the second resource now look invalid.
2024-01-05 23:16:40 +00:00
snap = nil
break
}
}
return nil, snap, errors.Join(errs...)
}
type TestStep struct {
Op TestOp
ExpectFailure bool
SkipPreview bool
Validate ValidateFunc
}
func (t *TestStep) ValidateAnd(f ValidateFunc) {
o := t.Validate
t.Validate = func(project workspace.Project, target deploy.Target, entries JournalEntries,
events []Event, err error,
) error {
r := o(project, target, entries, events, err)
if r != nil {
return r
}
return f(project, target, entries, events, err)
}
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
// TestUpdateOptions is UpdateOptions for a TestPlan.
type TestUpdateOptions struct {
UpdateOptions
// a factory to produce a plugin host for an update operation.
HostF deploytest.PluginHostFactory
}
// Options produces UpdateOptions for an update operation.
func (o TestUpdateOptions) Options() UpdateOptions {
opts := o.UpdateOptions
if o.HostF != nil {
opts.Host = o.HostF()
}
return opts
}
type TestPlan struct {
Project string
Stack string
Runtime string
RuntimeOptions map[string]interface{}
Config config.Map
Decrypter config.Decrypter
BackendClient deploy.BackendClient
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
Options TestUpdateOptions
Steps []TestStep
}
Add tokens.StackName (#14487) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new type `tokens.StackName` which is a relatively strongly typed container for a stack name. The only weakly typed aspect of it is Go will always allow the "zero" value to be created for a struct, which for a stack name is the empty string which is invalid. To prevent introducing unexpected empty strings when working with stack names the `String()` method will panic for zero initialized stack names. Apart from the zero value, all other instances of `StackName` are via `ParseStackName` which returns a descriptive error if the string is not valid. This PR only updates "pkg/" to use this type. There are a number of places in "sdk/" which could do with this type as well, but there's no harm in doing a staggered roll out, and some parts of "sdk/" are user facing and will probably have to stay on the current `tokens.Name` and `tokens.QName` types. There are two places in the system where we panic on invalid stack names, both in the http backend. This _should_ be fine as we've had long standing validation that stacks created in the service are valid stack names. Just in case people have managed to introduce invalid stack names, there is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn off the validation _and_ panicing for stack names. Users can use that to temporarily disable the validation and continue working, but it should only be seen as a temporary measure. If they have invalid names they should rename them, or if they think they should be valid raise an issue with us to change the validation code. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 07:44:54 +00:00
func (p *TestPlan) getNames() (stack tokens.StackName, project tokens.PackageName, runtime string) {
project = tokens.PackageName(p.Project)
if project == "" {
project = "test"
}
runtime = p.Runtime
if runtime == "" {
runtime = "test"
}
Add tokens.StackName (#14487) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new type `tokens.StackName` which is a relatively strongly typed container for a stack name. The only weakly typed aspect of it is Go will always allow the "zero" value to be created for a struct, which for a stack name is the empty string which is invalid. To prevent introducing unexpected empty strings when working with stack names the `String()` method will panic for zero initialized stack names. Apart from the zero value, all other instances of `StackName` are via `ParseStackName` which returns a descriptive error if the string is not valid. This PR only updates "pkg/" to use this type. There are a number of places in "sdk/" which could do with this type as well, but there's no harm in doing a staggered roll out, and some parts of "sdk/" are user facing and will probably have to stay on the current `tokens.Name` and `tokens.QName` types. There are two places in the system where we panic on invalid stack names, both in the http backend. This _should_ be fine as we've had long standing validation that stacks created in the service are valid stack names. Just in case people have managed to introduce invalid stack names, there is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn off the validation _and_ panicing for stack names. Users can use that to temporarily disable the validation and continue working, but it should only be seen as a temporary measure. If they have invalid names they should rename them, or if they think they should be valid raise an issue with us to change the validation code. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-15 07:44:54 +00:00
stack = tokens.MustParseStackName("test")
if p.Stack != "" {
stack = tokens.MustParseStackName(p.Stack)
}
return stack, project, runtime
}
func (p *TestPlan) NewURN(typ tokens.Type, name string, parent resource.URN) resource.URN {
stack, project, _ := p.getNames()
var pt tokens.Type
if parent != "" {
When changing parents also fix URNs (#13935) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/13903 When we change parents in a statefile we also need to fix up the URN. This is needed so aliases can track over partial updates. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-14 19:52:27 +00:00
pt = parent.QualifiedType()
}
Allow anything in resource names (#14107) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/13968. Fixes https://github.com/pulumi/pulumi/issues/8949. This requires changing the parsing of URN's slightly, it is _very_ likely that providers will need to update to handle URNs like this correctly. This changes resource names to be `string` not `QName`. We never validated this before and it turns out that users have put all manner of text for resource names so we just updating the system to correctly reflect that. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-11-20 08:59:00 +00:00
return resource.NewURN(stack.Q(), project, pt, typ, name)
}
func (p *TestPlan) NewProviderURN(pkg tokens.Package, name string, parent resource.URN) resource.URN {
return p.NewURN(providers.MakeProviderType(pkg), name, parent)
}
func (p *TestPlan) GetProject() workspace.Project {
_, projectName, runtime := p.getNames()
return workspace.Project{
Name: projectName,
Runtime: workspace.NewProjectRuntimeInfo(runtime, p.RuntimeOptions),
}
}
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func (p *TestPlan) GetTarget(t testing.TB, snapshot *deploy.Snapshot) deploy.Target {
stack, _, _ := p.getNames()
cfg := p.Config
if cfg == nil {
cfg = config.Map{}
}
return deploy.Target{
Name: stack,
Config: cfg,
Decrypter: p.Decrypter,
// note: it's really important that the preview and update operate on different snapshots. the engine can and
// does mutate the snapshot in-place, even in previews, and sharing a snapshot between preview and update can
// cause state changes from the preview to persist even when doing an update.
Snapshot: CloneSnapshot(t, snapshot),
}
}
// CloneSnapshot makes a deep copy of the given snapshot and returns a pointer to the clone.
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func CloneSnapshot(t testing.TB, snap *deploy.Snapshot) *deploy.Snapshot {
t.Helper()
if snap != nil {
copiedSnap := copystructure.Must(copystructure.Copy(*snap)).(deploy.Snapshot)
assert.True(t, reflect.DeepEqual(*snap, copiedSnap))
return &copiedSnap
}
return snap
}
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func (p *TestPlan) Run(t testing.TB, snapshot *deploy.Snapshot) *deploy.Snapshot {
project := p.GetProject()
snap := snapshot
for _, step := range p.Steps {
// note: it's really important that the preview and update operate on different snapshots. the engine can and
// does mutate the snapshot in-place, even in previews, and sharing a snapshot between preview and update can
// cause state changes from the preview to persist even when doing an update.
// GetTarget ALWAYS clones the snapshot, so the previewTarget.Snapshot != target.Snapshot
if !step.SkipPreview {
previewTarget := p.GetTarget(t, snap)
// Don't run validate on the preview step
_, err := step.Op.Run(project, previewTarget, p.Options, true, p.BackendClient, nil)
if step.ExpectFailure {
assert.Error(t, err)
continue
}
assert.NoError(t, err)
}
var err error
target := p.GetTarget(t, snap)
snap, err = step.Op.Run(project, target, p.Options, false, p.BackendClient, step.Validate)
if step.ExpectFailure {
assert.Error(t, err)
continue
}
if err != nil {
if result.IsBail(err) {
t.Logf("Got unexpected bail result: %v", err)
t.FailNow()
} else {
t.Logf("Got unexpected error result: %v", err)
t.FailNow()
}
}
assert.NoError(t, err)
}
return snap
}
// resCount is the expected number of resources registered during this test.
func MakeBasicLifecycleSteps(t *testing.T, resCount int) []TestStep {
return []TestStep{
// Initial update
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only creates or reads.
for _, entry := range entries {
op := entry.Step.Op()
assert.True(t, op == deploy.OpCreate || op == deploy.OpRead)
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only refresh-sames.
for _, entry := range entries {
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op update
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only sames.
for _, entry := range entries {
op := entry.Step.Op()
assert.True(t, op == deploy.OpSame || op == deploy.OpRead)
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only refresh-sames.
for _, entry := range entries {
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// Destroy
{
Op: Destroy,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only deletes.
for _, entry := range entries {
switch entry.Step.Op() {
case deploy.OpDelete, deploy.OpReadDiscard:
// ok
default:
assert.Fail(t, "expected OpDelete or OpReadDiscard")
}
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, 0)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
assert.Len(t, entries, 0)
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, 0)
return err
},
},
}
}
type testBuilder struct {
t *testing.T
loaders []*deploytest.ProviderLoader
snap *deploy.Snapshot
}
func newTestBuilder(t *testing.T, snap *deploy.Snapshot) *testBuilder {
return &testBuilder{
t: t,
snap: snap,
loaders: slice.Prealloc[*deploytest.ProviderLoader](1),
}
}
func (b *testBuilder) WithProvider(name string, version string, prov *deploytest.Provider) *testBuilder {
loader := deploytest.NewProviderLoader(
tokens.Package(name), semver.MustParse(version), func() (plugin.Provider, error) {
return prov, nil
})
b.loaders = append(b.loaders, loader)
return b
}
type Result struct {
snap *deploy.Snapshot
err error
}
func (b *testBuilder) RunUpdate(program func(info plugin.RunInfo, monitor *deploytest.ResourceMonitor) error) *Result {
programF := deploytest.NewLanguageRuntimeF(program)
hostF := deploytest.NewPluginHostF(nil, nil, programF, b.loaders...)
p := &TestPlan{
Options: TestUpdateOptions{HostF: hostF},
}
// Run an update for initial state.
var err error
snap, err := TestOp(Update).Run(
p.GetProject(), p.GetTarget(b.t, b.snap), p.Options, false, p.BackendClient, nil)
return &Result{
snap: snap,
err: err,
}
}
// Then() is used to convey dependence between program runs via program structure.
func (res *Result) Then(do func(snap *deploy.Snapshot, err error)) {
do(res.snap, res.err)
}