pulumi/pkg/engine/lifecycletest/test_plan.go

429 lines
12 KiB
Go
Raw Permalink Normal View History

2021-09-21 17:00:44 +00:00
//nolint:revive
package lifecycletest
import (
"context"
"reflect"
"sync"
"testing"
"github.com/mitchellh/copystructure"
"github.com/stretchr/testify/assert"
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
"github.com/stretchr/testify/require"
"github.com/pulumi/pulumi/pkg/v3/display"
. "github.com/pulumi/pulumi/pkg/v3/engine"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy"
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/deploytest"
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/providers"
"github.com/pulumi/pulumi/pkg/v3/util/cancel"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/config"
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/result"
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
)
type updateInfo struct {
project workspace.Project
target deploy.Target
}
func (u *updateInfo) GetRoot() string {
return ""
}
func (u *updateInfo) GetProject() *workspace.Project {
return &u.project
}
func (u *updateInfo) GetTarget() *deploy.Target {
return &u.target
}
func ImportOp(imports []deploy.Import) TestOp {
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
return TestOp(func(info UpdateInfo, ctx *Context, opts UpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
dryRun bool,
) (*deploy.Plan, display.ResourceChanges, error) {
return Import(info, ctx, opts, imports, dryRun)
})
}
type TestOp func(UpdateInfo, *Context, UpdateOptions, bool) (*deploy.Plan, display.ResourceChanges, error)
type ValidateFunc func(project workspace.Project, target deploy.Target, entries JournalEntries,
events []Event, err error) error
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
func (op TestOp) Plan(project workspace.Project, target deploy.Target, opts TestUpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Plan, error) {
plan, _, err := op.runWithContext(context.Background(), project, target, opts, true, backendClient, validate)
return plan, err
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
func (op TestOp) Run(project workspace.Project, target deploy.Target, opts TestUpdateOptions,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
dryRun bool, backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Snapshot, error) {
return op.RunWithContext(context.Background(), project, target, opts, dryRun, backendClient, validate)
}
func (op TestOp) RunWithContext(
callerCtx context.Context, project workspace.Project,
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
target deploy.Target, opts TestUpdateOptions, dryRun bool,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Snapshot, error) {
_, snap, err := op.runWithContext(callerCtx, project, target, opts, dryRun, backendClient, validate)
return snap, err
Preview of update plans (#8448) * Implement resource plans in the engine * Plumb plans through the CLI. * Update wording * plan renderer * constraints * Renames * Update message * fixes for rebase breaks and diffs * WIP: outputs in plans * fix diff * fixup * Liniting and test fixing * Test and fix PropertyPath.String() * Fix colors * Fix cmdutil.PrintTable to handle non-simple strings * More tests * Readd test_plan.go * lint * Test expected deletes * Test expected delete * Test missing create * Fix test for missing creates * rm Paths() * property set shrink test * notes * More tests * Pop op before constraint check * Delete plan cmd, rename arguments to preview and up * Hide behind envvars * typo * Better constraint diffs * Adds/Deletes/Updates * Fix aliased * Check more constraints * fix test * revert stack changes * Resource sames test * Fix same resource test * Fix more tests * linting * Update pkg/cmd/pulumi/up.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Update pkg/cmd/pulumi/preview.go Co-authored-by: Alex Mullans <a.mullans@pulumi.com> * Auto refresh if using plans * Fix TestGetRefreshOption * Fix TestExplicitDeleteBeforeReplace * lint * More copying in tests because I do not trust myself to get mutation correct * Small preview plan test * Add TestPlannedUpdateChangedStack * Revert auto-refresh changes * Validate outputs don't change * omitempty * Add manifest to plan * Add proper Plan type * wip config work * Config and manifest serder * linting * Asset NoError * Actually check error * Fix clone * Test diag message * Start on more tests * Add String and GoString to Result I got fed up assert errors in tests that looked like: ``` Expected nil, but got: &result.simpleResult{err:(*errors.fundamental)(0xc0002fa5d0)} ``` It was very hard to work out at a glance what had gone wrong and I kept having to hook a debugger just to look at what the error was. With GoString these now print something like: ``` Expected nil, but got: &simpleResult{err: Unexpected diag message: <{%reset%}>resource violates plan: properties changed: -zed, -baz, -foo<{%reset%}> } ``` Which is much more ussful. * Add test error text * Fix reporting of unseen op errors * Fix unneeded deletes * Fix unexpected deletes * Fix up tests * Fix merge conflict * lint * Fix nil map error * Fix serialisation typo * Diff against old inputs * Diff against checked goal * Diff against empty for creates * Fix test * inputs not outputs * Seperate PlanDiff type * Add properties * Fix input diffs * Handle creates * lint * Add plan message * Clone plan for update preview * Save and serialise env vars in plans * lint * pretty print json * input output difference test * test alias * fix typo in for loop * Handle resource plans with nil goal * go mod tidy * typo * Auto use plans from up previews in experimental mode * Don't preview if we have plan * Don't run previews with plans now * fixing tests * Handle diffs and goals * Update copystructure * tests/go.sum * Revert mod changes * Add copystructure to tests/go.sum * includeUnknowns * go mod tidy * Make plans for imports * Remove unused function * Move code more locally * Handle nil in serialize * Handle empty output diffs * Add test for dropping computed values * Allow computed properties to become deletes * if out the generation of plans unless experimental mode is opt'd into * lint * typo * Revert back to plans not skipping previews, this is orthognal to --skip-preview * Trying to work out non-determinism * Remove notes.txt * Hacking with check idea * Pass checked inputs back to Check from plan file * Include resource urn in constraint error * Give much more informative errors when plans fail * lint * Update expected diag strings in tests * Remove unused code * Duplicate Diff and DeepEquals methods for plans * Add comment about check ops with failures * Fix CheckedInputs comment * OutputDiff doesn't need to be a pointer * Fix checks against computed * diffStringSets * lint * lint pkg * Use 4 space indent * Don't wrap Buffer in Writer * Mark flags hidden rather than disabled * Remove envvars from plans * Assert MarkHidden error * Add to changelog * Note plan/save-plan is experimental Co-authored-by: Pat Gavlin <pat@pulumi.com> Co-authored-by: Alex Mullans <a.mullans@pulumi.com>
2022-01-31 10:31:51 +00:00
}
func (op TestOp) runWithContext(
callerCtx context.Context, project workspace.Project,
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
target deploy.Target, opts TestUpdateOptions, dryRun bool,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
backendClient deploy.BackendClient, validate ValidateFunc,
) (*deploy.Plan, *deploy.Snapshot, error) {
// Create an appropriate update info and context.
info := &updateInfo{project: project, target: target}
cancelCtx, cancelSrc := cancel.NewContext(context.Background())
done := make(chan bool)
defer close(done)
go func() {
select {
case <-callerCtx.Done():
cancelSrc.Cancel()
case <-done:
}
}()
events := make(chan Event)
journal := NewJournal()
ctx := &Context{
Cancel: cancelCtx,
Events: events,
SnapshotManager: journal,
BackendClient: backendClient,
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
updateOpts := opts.Options()
defer func() {
if updateOpts.Host != nil {
contract.IgnoreClose(updateOpts.Host)
}
}()
// Begin draining events.
var wg sync.WaitGroup
var firedEvents []Event
wg.Add(1)
go func() {
for e := range events {
firedEvents = append(firedEvents, e)
}
wg.Done()
}()
// Run the step and its validator.
plan, _, opErr := op(info, ctx, updateOpts, dryRun)
close(events)
wg.Wait()
contract.IgnoreClose(journal)
if validate != nil {
opErr = validate(project, target, journal.Entries(), firedEvents, opErr)
}
if dryRun {
return plan, nil, opErr
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := journal.Snap(target.Snapshot)
if opErr == nil && err != nil {
opErr = err
} else if opErr == nil && snap != nil {
opErr = snap.VerifyIntegrity()
}
return nil, snap, opErr
}
type TestStep struct {
Op TestOp
ExpectFailure bool
SkipPreview bool
Validate ValidateFunc
}
func (t *TestStep) ValidateAnd(f ValidateFunc) {
o := t.Validate
t.Validate = func(project workspace.Project, target deploy.Target, entries JournalEntries,
events []Event, err error,
) error {
r := o(project, target, entries, events, err)
if r != nil {
return r
}
return f(project, target, entries, events, err)
}
}
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
// TestUpdateOptions is UpdateOptions for a TestPlan.
type TestUpdateOptions struct {
UpdateOptions
// a factory to produce a plugin host for an update operation.
HostF deploytest.PluginHostFactory
}
// Options produces UpdateOptions for an update operation.
func (o TestUpdateOptions) Options() UpdateOptions {
opts := o.UpdateOptions
if o.HostF != nil {
opts.Host = o.HostF()
}
return opts
}
type TestPlan struct {
Project string
Stack string
Runtime string
RuntimeOptions map[string]interface{}
Config config.Map
Decrypter config.Decrypter
BackendClient deploy.BackendClient
Lifecycle tests shouldn't use a closed host (#14063) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This PR fixes the inadvertent use of a closed plugin host in the lifecycle tests. The tests override the host that is provided to the engine, for good reasons, but that same host is re-used across multiple engine operations. Since the engine closes the supplied host at the end of each operation, subsequent operations are handed a closed host. In order to detect engine bugs related to the use of a closed host (see https://github.com/pulumi/pulumi/pull/14057), the fake host should return an error if it is used after being closed (as does the real host). This PR addresses this. The detailed change is to shift to using a host factory that produces a host in `TestOp.Run`. The `TestPlan` now takes a `TestUpdateOptions` with `HostF` and an embedded `UpdateOptions`. Note that two tests fail due to https://github.com/pulumi/pulumi/pull/14057 which was being masked by the problem that is fixed here. This PR disables those tests and the other PR will re-enable them. - `TestCanceledRefresh` - `TestProviderCancellation` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-28 21:50:18 +00:00
Options TestUpdateOptions
Steps []TestStep
}
func (p *TestPlan) getNames() (stack tokens.Name, project tokens.PackageName, runtime string) {
project = tokens.PackageName(p.Project)
if project == "" {
project = "test"
}
runtime = p.Runtime
if runtime == "" {
runtime = "test"
}
stack = tokens.Name(p.Stack)
if stack == "" {
stack = "test"
}
return stack, project, runtime
}
func (p *TestPlan) NewURN(typ tokens.Type, name string, parent resource.URN) resource.URN {
stack, project, _ := p.getNames()
var pt tokens.Type
if parent != "" {
When changing parents also fix URNs (#13935) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/13903 When we change parents in a statefile we also need to fix up the URN. This is needed so aliases can track over partial updates. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-09-14 19:52:27 +00:00
pt = parent.QualifiedType()
}
return resource.NewURN(stack.Q(), project, pt, typ, tokens.QName(name))
}
func (p *TestPlan) NewProviderURN(pkg tokens.Package, name string, parent resource.URN) resource.URN {
return p.NewURN(providers.MakeProviderType(pkg), name, parent)
}
func (p *TestPlan) GetProject() workspace.Project {
_, projectName, runtime := p.getNames()
return workspace.Project{
Name: projectName,
Runtime: workspace.NewProjectRuntimeInfo(runtime, p.RuntimeOptions),
}
}
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func (p *TestPlan) GetTarget(t testing.TB, snapshot *deploy.Snapshot) deploy.Target {
stack, _, _ := p.getNames()
cfg := p.Config
if cfg == nil {
cfg = config.Map{}
}
return deploy.Target{
Name: stack,
Config: cfg,
Decrypter: p.Decrypter,
// note: it's really important that the preview and update operate on different snapshots. the engine can and
// does mutate the snapshot in-place, even in previews, and sharing a snapshot between preview and update can
// cause state changes from the preview to persist even when doing an update.
Snapshot: CloneSnapshot(t, snapshot),
}
}
// CloneSnapshot makes a deep copy of the given snapshot and returns a pointer to the clone.
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func CloneSnapshot(t testing.TB, snap *deploy.Snapshot) *deploy.Snapshot {
t.Helper()
if snap != nil {
copiedSnap := copystructure.Must(copystructure.Copy(*snap)).(deploy.Snapshot)
assert.True(t, reflect.DeepEqual(*snap, copiedSnap))
return &copiedSnap
}
return snap
}
[snapshot] Add diff benchmarks and more tests - Add a utility for generating snapshots for diff testing - Add a framework for testing and benchmarking the deployment differ Baseline benchmark results: httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee base.txt ``` │ base.txt │ │ sec/op │ DiffStack/1_x_2_B-10 46.30µ ± 1% DiffStack/2_x_2_B-10 82.68µ ± 1% DiffStack/4_x_2_B-10 191.0µ ± 1% DiffStack/8_x_2_B-10 499.7µ ± 0% DiffStack/16_x_2_B-10 1.489m ± 0% DiffStack/32_x_2_B-10 4.895m ± 1% DiffStack/48_x_2_B-10 9.651m ± 1% DiffStack/64_x_2_B-10 15.11m ± 0% DiffStack/1_x_8.2_kB-10 103.3µ ± 1% DiffStack/2_x_8.2_kB-10 308.8µ ± 0% DiffStack/4_x_8.2_kB-10 899.2µ ± 0% DiffStack/8_x_8.2_kB-10 2.989m ± 1% DiffStack/16_x_8.2_kB-10 10.53m ± 0% DiffStack/32_x_8.2_kB-10 36.27m ± 1% DiffStack/48_x_8.2_kB-10 72.53m ± 0% DiffStack/64_x_8.2_kB-10 125.5m ± 0% DiffStack/1_x_33_kB-10 317.4µ ± 1% DiffStack/2_x_33_kB-10 756.8µ ± 1% DiffStack/4_x_33_kB-10 2.605m ± 0% DiffStack/8_x_33_kB-10 9.241m ± 2% DiffStack/16_x_33_kB-10 32.86m ± 1% DiffStack/32_x_33_kB-10 110.2m ± 2% DiffStack/48_x_33_kB-10 231.3m ± 0% DiffStack/64_x_33_kB-10 383.8m ± 1% DiffStack/2_x_131_kB-10 2.748m ± 1% DiffStack/4_x_131_kB-10 8.609m ± 0% DiffStack/8_x_131_kB-10 28.44m ± 0% DiffStack/16_x_131_kB-10 96.98m ± 1% DiffStack/32_x_131_kB-10 349.4m ± 1% DiffStack/48_x_131_kB-10 761.7m ± 0% DiffStack/64_x_131_kB-10 1.312 ± 1% DiffStack/1_x_524_kB-10 3.238m ± 1% DiffStack/2_x_524_kB-10 9.867m ± 0% DiffStack/4_x_524_kB-10 29.65m ± 0% DiffStack/8_x_524_kB-10 98.91m ± 1% DiffStack/16_x_524_kB-10 348.1m ± 1% DiffStackRecorded/two-large-checkpoints.json-10 46.62m ± 0% DiffStackRecorded/checkpoints.json-10 835.8m ± 0% geomean 11.15m │ base.txt │ │ ratio │ DiffStack/1_x_2_B-10 789.5m ± 0% DiffStack/2_x_2_B-10 1.034 ± 0% DiffStack/4_x_2_B-10 1.676 ± 0% DiffStack/8_x_2_B-10 2.883 ± 0% DiffStack/16_x_2_B-10 5.270 ± 0% DiffStack/32_x_2_B-10 10.20 ± 0% DiffStack/48_x_2_B-10 14.24 ± 0% DiffStack/64_x_2_B-10 17.51 ± 0% DiffStack/1_x_8.2_kB-10 950.9m ± 0% DiffStack/2_x_8.2_kB-10 1.716 ± 0% DiffStack/4_x_8.2_kB-10 2.792 ± 0% DiffStack/8_x_8.2_kB-10 5.350 ± 0% DiffStack/16_x_8.2_kB-10 10.62 ± 0% DiffStack/32_x_8.2_kB-10 20.01 ± 0% DiffStack/48_x_8.2_kB-10 27.48 ± 0% DiffStack/64_x_8.2_kB-10 36.97 ± 0% DiffStack/1_x_33_kB-10 1.467 ± 0% DiffStack/2_x_33_kB-10 1.581 ± 0% DiffStack/4_x_33_kB-10 2.967 ± 0% DiffStack/8_x_33_kB-10 5.766 ± 0% DiffStack/16_x_33_kB-10 10.87 ± 0% DiffStack/32_x_33_kB-10 19.00 ± 0% DiffStack/48_x_33_kB-10 27.14 ± 0% DiffStack/64_x_33_kB-10 34.39 ± 0% DiffStack/2_x_131_kB-10 1.993 ± 0% DiffStack/4_x_131_kB-10 3.173 ± 0% DiffStack/8_x_131_kB-10 5.334 ± 0% DiffStack/16_x_131_kB-10 9.296 ± 0% DiffStack/32_x_131_kB-10 17.10 ± 0% DiffStack/48_x_131_kB-10 25.25 ± 0% DiffStack/64_x_131_kB-10 33.13 ± 0% DiffStack/1_x_524_kB-10 1.498 ± 0% DiffStack/2_x_524_kB-10 1.998 ± 0% DiffStack/4_x_524_kB-10 2.998 ± 0% DiffStack/8_x_524_kB-10 5.084 ± 0% DiffStack/16_x_524_kB-10 9.079 ± 0% DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% DiffStackRecorded/checkpoints.json-10 36.60 ± 0% geomean 5.961 │ base.txt │ │ B/op │ DiffStack/1_x_2_B-10 35.17Ki ± 0% DiffStack/2_x_2_B-10 70.05Ki ± 0% DiffStack/4_x_2_B-10 186.8Ki ± 0% DiffStack/8_x_2_B-10 529.0Ki ± 0% DiffStack/16_x_2_B-10 1.768Mi ± 0% DiffStack/32_x_2_B-10 6.513Mi ± 0% DiffStack/48_x_2_B-10 14.05Mi ± 0% DiffStack/64_x_2_B-10 22.39Mi ± 0% DiffStack/1_x_8.2_kB-10 154.7Ki ± 0% DiffStack/2_x_8.2_kB-10 571.9Ki ± 0% DiffStack/4_x_8.2_kB-10 1.846Mi ± 0% DiffStack/8_x_8.2_kB-10 6.920Mi ± 0% DiffStack/16_x_8.2_kB-10 26.50Mi ± 0% DiffStack/32_x_8.2_kB-10 98.33Mi ± 0% DiffStack/48_x_8.2_kB-10 209.4Mi ± 0% DiffStack/64_x_8.2_kB-10 365.6Mi ± 0% DiffStack/1_x_33_kB-10 808.7Ki ± 0% DiffStack/2_x_33_kB-10 2.013Mi ± 0% DiffStack/4_x_33_kB-10 7.551Mi ± 0% DiffStack/8_x_33_kB-10 28.58Mi ± 0% DiffStack/16_x_33_kB-10 104.4Mi ± 0% DiffStack/32_x_33_kB-10 365.5Mi ± 0% DiffStack/48_x_33_kB-10 801.1Mi ± 0% DiffStack/64_x_33_kB-10 1.283Gi ± 0% DiffStack/2_x_131_kB-10 9.179Mi ± 0% DiffStack/4_x_131_kB-10 29.14Mi ± 0% DiffStack/8_x_131_kB-10 98.77Mi ± 0% DiffStack/16_x_131_kB-10 337.1Mi ± 0% DiffStack/32_x_131_kB-10 1.206Gi ± 0% DiffStack/48_x_131_kB-10 2.771Gi ± 0% DiffStack/64_x_131_kB-10 4.606Gi ± 0% DiffStack/1_x_524_kB-10 12.41Mi ± 1% DiffStack/2_x_524_kB-10 37.01Mi ± 1% DiffStack/4_x_524_kB-10 110.6Mi ± 0% DiffStack/8_x_524_kB-10 367.3Mi ± 1% DiffStack/16_x_524_kB-10 1.261Gi ± 0% DiffStackRecorded/two-large-checkpoints.json-10 69.25Mi ± 0% DiffStackRecorded/checkpoints.json-10 1.198Gi ± 0% geomean 26.14Mi │ base.txt │ │ allocs/op │ DiffStack/1_x_2_B-10 361.0 ± 0% DiffStack/2_x_2_B-10 649.0 ± 0% DiffStack/4_x_2_B-10 1.512k ± 0% DiffStack/8_x_2_B-10 4.141k ± 0% DiffStack/16_x_2_B-10 13.09k ± 0% DiffStack/32_x_2_B-10 46.38k ± 0% DiffStack/48_x_2_B-10 94.38k ± 0% DiffStack/64_x_2_B-10 152.7k ± 0% DiffStack/1_x_8.2_kB-10 323.0 ± 0% DiffStack/2_x_8.2_kB-10 630.0 ± 0% DiffStack/4_x_8.2_kB-10 1.354k ± 0% DiffStack/8_x_8.2_kB-10 3.606k ± 0% DiffStack/16_x_8.2_kB-10 11.16k ± 0% DiffStack/32_x_8.2_kB-10 36.45k ± 0% DiffStack/48_x_8.2_kB-10 71.55k ± 0% DiffStack/64_x_8.2_kB-10 123.9k ± 0% DiffStack/1_x_33_kB-10 346.0 ± 0% DiffStack/2_x_33_kB-10 607.0 ± 0% DiffStack/4_x_33_kB-10 1.396k ± 0% DiffStack/8_x_33_kB-10 3.764k ± 0% DiffStack/16_x_33_kB-10 11.26k ± 0% DiffStack/32_x_33_kB-10 34.55k ± 0% DiffStack/48_x_33_kB-10 69.72k ± 0% DiffStack/64_x_33_kB-10 114.1k ± 0% DiffStack/2_x_131_kB-10 678.0 ± 0% DiffStack/4_x_131_kB-10 1.494k ± 0% DiffStack/8_x_131_kB-10 3.737k ± 0% DiffStack/16_x_131_kB-10 10.40k ± 0% DiffStack/32_x_131_kB-10 32.24k ± 0% DiffStack/48_x_131_kB-10 66.12k ± 0% DiffStack/64_x_131_kB-10 110.9k ± 0% DiffStack/1_x_524_kB-10 374.0 ± 1% DiffStack/2_x_524_kB-10 707.0 ± 0% DiffStack/4_x_524_kB-10 1.528k ± 0% DiffStack/8_x_524_kB-10 3.797k ± 1% DiffStack/16_x_524_kB-10 10.56k ± 1% DiffStackRecorded/two-large-checkpoints.json-10 512.1k ± 0% DiffStackRecorded/checkpoints.json-10 7.240M ± 0% geomean 8.309k ```
2023-05-04 23:28:49 +00:00
func (p *TestPlan) Run(t testing.TB, snapshot *deploy.Snapshot) *deploy.Snapshot {
project := p.GetProject()
snap := snapshot
for _, step := range p.Steps {
// note: it's really important that the preview and update operate on different snapshots. the engine can and
// does mutate the snapshot in-place, even in previews, and sharing a snapshot between preview and update can
// cause state changes from the preview to persist even when doing an update.
// GetTarget ALWAYS clones the snapshot, so the previewTarget.Snapshot != target.Snapshot
if !step.SkipPreview {
previewTarget := p.GetTarget(t, snap)
// Don't run validate on the preview step
_, err := step.Op.Run(project, previewTarget, p.Options, true, p.BackendClient, nil)
if step.ExpectFailure {
assert.Error(t, err)
continue
}
assert.NoError(t, err)
}
var err error
target := p.GetTarget(t, snap)
snap, err = step.Op.Run(project, target, p.Options, false, p.BackendClient, step.Validate)
if step.ExpectFailure {
assert.Error(t, err)
continue
}
if err != nil {
if result.IsBail(err) {
t.Logf("Got unexpected bail result: %v", err)
t.FailNow()
} else {
t.Logf("Got unexpected error result: %v", err)
t.FailNow()
}
}
assert.NoError(t, err)
}
return snap
}
// resCount is the expected number of resources registered during this test.
func MakeBasicLifecycleSteps(t *testing.T, resCount int) []TestStep {
return []TestStep{
// Initial update
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only creates or reads.
for _, entry := range entries {
op := entry.Step.Op()
assert.True(t, op == deploy.OpCreate || op == deploy.OpRead)
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only refresh-sames.
for _, entry := range entries {
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op update
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only sames.
for _, entry := range entries {
op := entry.Step.Op()
assert.True(t, op == deploy.OpSame || op == deploy.OpRead)
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only refresh-sames.
for _, entry := range entries {
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, resCount)
return err
},
},
// Destroy
{
Op: Destroy,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
// Should see only deletes.
for _, entry := range entries {
switch entry.Step.Op() {
case deploy.OpDelete, deploy.OpReadDiscard:
// ok
default:
assert.Fail(t, "expected OpDelete or OpReadDiscard")
}
}
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, 0)
return err
},
},
// No-op refresh
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, entries JournalEntries,
_ []Event, err error,
) error {
require.NoError(t, err)
assert.Len(t, entries, 0)
[engine] Only record a resource's chosen alias. (#9288) As we discovered when removing aliases from the state entirely, the snapshotter needs to be alias-aware so that it can fix up references to resources that were aliased. After a resource operation finishes, the snapshotter needs to write out a new copy of the snapshot. However, at the time we write the snapshot, there may be resources that have not yet been registered that refer to the just-registered resources by a different URN due to aliasing. Those references need to be fixed up prior to writing the snapshot in order to preserve the snapshot's integrity (in particular, the property that all URNs refer to resources that exist in the snapshot). For example, consider the following simple dependency graph: A <-- B. When that graph is serialized, B will contain a reference to A in its dependency list. Let the next run of the program produces the graph A' <-- B where A' is aliased to A. After A' is registered, the snapshotter needs to write a snapshot that contains its state, but B must also be updated so it references A' instead of A, which will no longer be in the snapshot. These changes take advantage of the fact that although a resource can provide multiple aliases, it can only ever resolve those aliases to a single resource in the existing state. Therefore, at the time the statefile is fixed up, each resource in the statefile could only have been aliased to a single old resource, and it is sufficient to store only the URN of the chosen resource rather than all possible aliases. In addition to preserving the ability to fix up references to aliased resources, retaining the chosen alias allows the history of a logical resource to be followed across aliases.
2022-03-28 15:36:08 +00:00
snap, err := entries.Snap(target.Snapshot)
require.NoError(t, err)
assert.Len(t, snap.Resources, 0)
return err
},
},
}
}