Add tokens.StackName (#14487)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.
This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.
There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.
Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 07:44:54 +00:00
|
|
|
// Copyright 2016-2023, Pulumi Corporation.
|
2022-09-19 16:44:37 +00:00
|
|
|
//
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
// you may not use this file except in compliance with the License.
|
|
|
|
// You may obtain a copy of the License at
|
|
|
|
//
|
|
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
//
|
|
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
// See the License for the specific language governing permissions and
|
|
|
|
// limitations under the License.
|
|
|
|
|
|
|
|
package httpstate
|
|
|
|
|
|
|
|
import (
|
|
|
|
"bytes"
|
|
|
|
"compress/gzip"
|
|
|
|
"context"
|
|
|
|
"crypto/sha256"
|
2023-05-04 23:28:49 +00:00
|
|
|
"encoding/base64"
|
2022-09-19 16:44:37 +00:00
|
|
|
"encoding/json"
|
|
|
|
"fmt"
|
|
|
|
"io"
|
2023-05-04 23:28:49 +00:00
|
|
|
"math/rand"
|
2022-09-19 16:44:37 +00:00
|
|
|
"net/http"
|
|
|
|
"net/http/httptest"
|
2023-03-03 16:36:39 +00:00
|
|
|
"os"
|
2023-05-04 23:28:49 +00:00
|
|
|
"path/filepath"
|
|
|
|
"strings"
|
2023-03-03 16:36:39 +00:00
|
|
|
"testing"
|
2022-09-19 16:44:37 +00:00
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
"github.com/dustin/go-humanize"
|
2022-09-19 16:44:37 +00:00
|
|
|
"github.com/hexops/gotextdiff"
|
|
|
|
"github.com/stretchr/testify/assert"
|
2022-12-06 21:19:23 +00:00
|
|
|
"github.com/stretchr/testify/require"
|
2022-09-19 16:44:37 +00:00
|
|
|
|
|
|
|
"github.com/pulumi/pulumi/pkg/v3/backend/httpstate/client"
|
2023-05-04 23:28:49 +00:00
|
|
|
"github.com/pulumi/pulumi/pkg/v3/engine"
|
|
|
|
"github.com/pulumi/pulumi/pkg/v3/engine/lifecycletest"
|
2022-09-19 16:44:37 +00:00
|
|
|
"github.com/pulumi/pulumi/pkg/v3/resource/deploy"
|
2023-05-04 23:28:49 +00:00
|
|
|
"github.com/pulumi/pulumi/pkg/v3/resource/deploy/deploytest"
|
|
|
|
"github.com/pulumi/pulumi/pkg/v3/resource/stack"
|
2022-12-06 21:19:23 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/apitype"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
|
2023-05-04 23:28:49 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
|
Add tokens.StackName (#14487)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.
This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.
There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.
Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 07:44:54 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
|
2022-12-06 21:19:23 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil"
|
2023-05-04 23:28:49 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
|
2022-09-19 16:44:37 +00:00
|
|
|
)
|
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
func applyEdits(before, deltas json.RawMessage) (json.RawMessage, error) {
|
|
|
|
var edits []gotextdiff.TextEdit
|
|
|
|
if err := json.Unmarshal(deltas, &edits); err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return json.RawMessage(gotextdiff.ApplyEdits(string(before), edits)), nil
|
|
|
|
}
|
|
|
|
|
2022-09-19 16:44:37 +00:00
|
|
|
// Check that cloudSnapshotPersister can talk the diff-based
|
|
|
|
// "checkpointverbatim" and "checkpointdelta" protocol when saving
|
|
|
|
// snapshots.
|
|
|
|
func TestCloudSnapshotPersisterUseOfDiffProtocol(t *testing.T) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
2022-09-19 16:44:37 +00:00
|
|
|
ctx := context.Background()
|
|
|
|
|
2022-12-06 21:19:23 +00:00
|
|
|
expectationsFile := "testdata/snapshot_test.json"
|
|
|
|
expectations := map[string]string{}
|
|
|
|
accept := cmdutil.IsTruthy(os.Getenv("PULUMI_ACCEPT"))
|
|
|
|
if accept {
|
|
|
|
t.Cleanup(func() {
|
|
|
|
bytes, err := json.MarshalIndent(expectations, "", " ")
|
|
|
|
require.NoError(t, err)
|
2023-03-03 16:36:39 +00:00
|
|
|
err = os.WriteFile(expectationsFile, bytes, 0o600)
|
2022-12-06 21:19:23 +00:00
|
|
|
require.NoError(t, err)
|
|
|
|
})
|
|
|
|
} else {
|
|
|
|
data, err := os.ReadFile(expectationsFile)
|
|
|
|
require.NoError(t, err)
|
|
|
|
err = json.Unmarshal(data, &expectations)
|
|
|
|
require.NoError(t, err)
|
|
|
|
}
|
|
|
|
|
|
|
|
assertEquals := func(expectedKey string, actual string) {
|
|
|
|
if accept {
|
|
|
|
expectations[expectedKey] = actual
|
|
|
|
return
|
|
|
|
}
|
|
|
|
expected, ok := expectations[expectedKey]
|
|
|
|
assert.True(t, ok)
|
|
|
|
assert.Equal(t, expected, actual, expectedKey)
|
|
|
|
}
|
|
|
|
|
|
|
|
assertEqual := func(expectedKey string, actual json.RawMessage) {
|
|
|
|
assertEquals(expectedKey, string(actual))
|
|
|
|
}
|
|
|
|
|
2022-09-19 16:44:37 +00:00
|
|
|
stackID := client.StackIdentifier{
|
|
|
|
Owner: "owner",
|
|
|
|
Project: "project",
|
Add tokens.StackName (#14487)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.
This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.
There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.
Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 07:44:54 +00:00
|
|
|
Stack: tokens.MustParseStackName("stack"),
|
2022-09-19 16:44:37 +00:00
|
|
|
}
|
|
|
|
updateID := "update-id"
|
|
|
|
|
|
|
|
var persistedState json.RawMessage
|
|
|
|
|
|
|
|
var lastRequest *http.Request
|
|
|
|
|
|
|
|
lastRequestAsVerbatim := func() (ret apitype.PatchUpdateVerbatimCheckpointRequest) {
|
|
|
|
err := json.NewDecoder(lastRequest.Body).Decode(&ret)
|
|
|
|
assert.Equal(t, "/api/stacks/owner/project/stack/update/update-id/checkpointverbatim", lastRequest.URL.Path)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
lastRequestAsDelta := func() (ret apitype.PatchUpdateCheckpointDeltaRequest) {
|
|
|
|
err := json.NewDecoder(lastRequest.Body).Decode(&ret)
|
|
|
|
assert.Equal(t, "/api/stacks/owner/project/stack/update/update-id/checkpointdelta", lastRequest.URL.Path)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
handleVerbatim := func(req apitype.PatchUpdateVerbatimCheckpointRequest) {
|
|
|
|
persistedState = req.UntypedDeployment
|
|
|
|
}
|
|
|
|
|
|
|
|
handleDelta := func(req apitype.PatchUpdateCheckpointDeltaRequest) {
|
|
|
|
edits := []gotextdiff.TextEdit{}
|
|
|
|
if err := json.Unmarshal(req.DeploymentDelta, &edits); err != nil {
|
|
|
|
assert.NoError(t, err)
|
|
|
|
}
|
|
|
|
persistedState = json.RawMessage([]byte(gotextdiff.ApplyEdits(string(persistedState), edits)))
|
|
|
|
assert.Equal(t, req.CheckpointHash, fmt.Sprintf("%x", sha256.Sum256(persistedState)))
|
|
|
|
}
|
|
|
|
|
|
|
|
typedPersistedState := func() apitype.DeploymentV3 {
|
|
|
|
var ud apitype.UntypedDeployment
|
|
|
|
err := json.Unmarshal(persistedState, &ud)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
var d3 apitype.DeploymentV3
|
|
|
|
err = json.Unmarshal(ud.Deployment, &d3)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
return d3
|
|
|
|
}
|
|
|
|
|
|
|
|
newMockServer := func() *httptest.Server {
|
2022-11-22 17:33:51 +00:00
|
|
|
return httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
|
|
|
|
switch req.URL.Path {
|
|
|
|
case "/api/capabilities":
|
|
|
|
resp := apitype.CapabilitiesResponse{Capabilities: []apitype.APICapabilityConfig{{
|
|
|
|
Capability: apitype.DeltaCheckpointUploads,
|
|
|
|
Configuration: json.RawMessage(`{"checkpointCutoffSizeBytes":1}`),
|
|
|
|
}}}
|
|
|
|
err := json.NewEncoder(rw).Encode(resp)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
return
|
|
|
|
case "/api/stacks/owner/project/stack/update/update-id/checkpointverbatim",
|
|
|
|
"/api/stacks/owner/project/stack/update/update-id/checkpointdelta":
|
2022-09-19 16:44:37 +00:00
|
|
|
lastRequest = req
|
|
|
|
rw.WriteHeader(200)
|
|
|
|
message := `{}`
|
|
|
|
reader, err := gzip.NewReader(req.Body)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
defer reader.Close()
|
2023-01-06 22:39:16 +00:00
|
|
|
rbytes, err := io.ReadAll(reader)
|
2022-09-19 16:44:37 +00:00
|
|
|
assert.NoError(t, err)
|
|
|
|
_, err = rw.Write([]byte(message))
|
|
|
|
assert.NoError(t, err)
|
|
|
|
req.Body = io.NopCloser(bytes.NewBuffer(rbytes))
|
2022-11-22 17:33:51 +00:00
|
|
|
default:
|
|
|
|
panic(fmt.Sprintf("Path not supported: %v", req.URL.Path))
|
|
|
|
}
|
|
|
|
}))
|
2022-09-19 16:44:37 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
newMockTokenSource := func() tokenSourceCapability {
|
|
|
|
return tokenSourceFn(func() (string, error) {
|
|
|
|
return "token", nil
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
initPersister := func() *cloudSnapshotPersister {
|
|
|
|
server := newMockServer()
|
2023-03-03 20:32:42 +00:00
|
|
|
backendGeneric, err := New(nil, server.URL, nil, false)
|
2022-09-19 16:44:37 +00:00
|
|
|
assert.NoError(t, err)
|
|
|
|
backend := backendGeneric.(*cloudBackend)
|
|
|
|
persister := backend.newSnapshotPersister(ctx, client.UpdateIdentifier{
|
|
|
|
StackIdentifier: stackID,
|
|
|
|
UpdateKind: apitype.UpdateUpdate,
|
|
|
|
UpdateID: updateID,
|
2023-09-19 18:31:56 +00:00
|
|
|
}, newMockTokenSource())
|
2022-09-19 16:44:37 +00:00
|
|
|
return persister
|
|
|
|
}
|
|
|
|
|
|
|
|
persister := initPersister()
|
|
|
|
|
|
|
|
// Req 1: the first request sends indented data verbatim to establish a good baseline state for further diffs.
|
|
|
|
|
|
|
|
err := persister.Save(&deploy.Snapshot{
|
|
|
|
Resources: []*resource.State{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
},
|
|
|
|
})
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
req1 := lastRequestAsVerbatim()
|
|
|
|
assert.Equal(t, 1, req1.SequenceNumber)
|
|
|
|
assert.Equal(t, 3, req1.Version)
|
2022-12-06 21:19:23 +00:00
|
|
|
assertEqual("req1", req1.UntypedDeployment)
|
2022-09-19 16:44:37 +00:00
|
|
|
|
|
|
|
handleVerbatim(req1)
|
|
|
|
assert.Equal(t, []apitype.ResourceV3{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
}, typedPersistedState().Resources)
|
|
|
|
|
|
|
|
// Req 2: then it switches to sending deltas as text diffs together with SHA-256 checksum of the expected
|
|
|
|
// resulting text representation of state.
|
|
|
|
|
|
|
|
err = persister.Save(&deploy.Snapshot{
|
|
|
|
Resources: []*resource.State{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
{URN: resource.URN("urn-2")},
|
|
|
|
},
|
|
|
|
})
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
req2 := lastRequestAsDelta()
|
|
|
|
assert.Equal(t, 2, req2.SequenceNumber)
|
2022-12-06 21:19:23 +00:00
|
|
|
assertEqual("req2", req2.DeploymentDelta)
|
|
|
|
assertEquals("req2.hash", req2.CheckpointHash)
|
2022-09-19 16:44:37 +00:00
|
|
|
|
|
|
|
handleDelta(req2)
|
|
|
|
assert.Equal(t, []apitype.ResourceV3{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
{URN: resource.URN("urn-2")},
|
|
|
|
}, typedPersistedState().Resources)
|
|
|
|
|
|
|
|
// Req 3: and continues using the diff protocol.
|
|
|
|
|
|
|
|
err = persister.Save(&deploy.Snapshot{
|
|
|
|
Resources: []*resource.State{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
},
|
|
|
|
})
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
req3 := lastRequestAsDelta()
|
|
|
|
assert.Equal(t, 3, req3.SequenceNumber)
|
2022-12-06 21:19:23 +00:00
|
|
|
assertEqual("req3", req3.DeploymentDelta)
|
|
|
|
assertEquals("req3.hash", req3.CheckpointHash)
|
2022-09-19 16:44:37 +00:00
|
|
|
|
|
|
|
handleDelta(req3)
|
|
|
|
assert.Equal(t, []apitype.ResourceV3{
|
|
|
|
{URN: resource.URN("urn-1")},
|
|
|
|
}, typedPersistedState().Resources)
|
|
|
|
}
|
|
|
|
|
|
|
|
type tokenSourceFn func() (string, error)
|
|
|
|
|
|
|
|
var _ tokenSourceCapability = tokenSourceFn(nil)
|
|
|
|
|
2023-03-01 22:23:20 +00:00
|
|
|
func (tsf tokenSourceFn) GetToken(_ context.Context) (string, error) {
|
2022-09-19 16:44:37 +00:00
|
|
|
return tsf()
|
|
|
|
}
|
2023-05-04 23:28:49 +00:00
|
|
|
|
|
|
|
func generateSnapshots(t testing.TB, r *rand.Rand, resourceCount, resourcePayloadBytes int) []*apitype.DeploymentV3 {
|
2023-09-28 21:50:18 +00:00
|
|
|
programF := deploytest.NewLanguageRuntimeF(func(info plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
|
2023-05-04 23:28:49 +00:00
|
|
|
ctx, err := pulumi.NewContext(context.Background(), pulumi.RunInfo{
|
|
|
|
Project: info.Project,
|
|
|
|
Stack: info.Stack,
|
|
|
|
Parallel: info.Parallel,
|
|
|
|
DryRun: info.DryRun,
|
|
|
|
MonitorAddr: info.MonitorAddress,
|
|
|
|
})
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
return pulumi.RunWithContext(ctx, func(ctx *pulumi.Context) error {
|
|
|
|
type Dummy struct {
|
|
|
|
pulumi.ResourceState
|
|
|
|
}
|
|
|
|
|
|
|
|
for i := 0; i < resourceCount; i++ {
|
|
|
|
var dummy Dummy
|
|
|
|
err := ctx.RegisterComponentResource("examples:dummy:Dummy", fmt.Sprintf("dummy-%d", i), &dummy)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
err = ctx.RegisterResourceOutputs(&dummy, pulumi.Map{
|
|
|
|
"deadweight": pulumi.String(pseudoRandomString(r, resourcePayloadBytes)),
|
|
|
|
})
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return nil
|
|
|
|
})
|
|
|
|
})
|
2023-09-28 21:50:18 +00:00
|
|
|
hostF := deploytest.NewPluginHostF(nil, nil, programF)
|
2023-05-04 23:28:49 +00:00
|
|
|
|
|
|
|
var journalEntries engine.JournalEntries
|
|
|
|
p := &lifecycletest.TestPlan{
|
Add display to the engine tests (#16050)
We want to add more test coverage to the display code. The best way to
do that is to add it to the engine tests, that already cover most of the
pulumi functionality.
It's probably not really possible to review all of the output, but at
least it gives us a baseline, which we can work with.
There's a couple of tests that are flaky for reasons I don't quite
understand yet. I marked them as to skip and we can look at them later.
I'd rather get in the baseline tests sooner, rather than spending a
bunch of time looking at that. The output differences also seem very
minor, so not super concerning.
The biggest remaining issue is that this doesn't interact well with the
Chdir we're doing in the engine. We could either pass the CWD through,
or just try to get rid of that Chdir. So this should only be merged
after https://github.com/pulumi/pulumi/pull/15607.
I've tried to split this into a few commits, separating out adding the
testdata, so it's hopefully a little easier to review, even though the
PR is still quite large.
One other thing to note is that we're comparing that the output has all
the same lines, and not that it is exactly the same. Because of how the
engine is implemented, there's a bunch of race conditions otherwise,
that would make us have to skip a bunch of tests, just because e.g.
resource A is sometimes deleted before resource B and sometimes it's the
other way around.
The biggest downside of that is that running with `PULUMI_ACCEPT` will
produce a diff even when there are no changes. Hopefully we won't have
to run that way too often though, so it might not be a huge issue?
---------
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-05-13 07:18:25 +00:00
|
|
|
// This test generates big amounts of data so the event streams that would need to be
|
|
|
|
// checked in get too big. Skip them instead.
|
|
|
|
Options: lifecycletest.TestUpdateOptions{T: t, HostF: hostF, SkipDisplayTests: true},
|
2023-05-04 23:28:49 +00:00
|
|
|
Steps: []lifecycletest.TestStep{
|
|
|
|
{
|
|
|
|
Op: engine.Update,
|
|
|
|
SkipPreview: true,
|
|
|
|
Validate: func(
|
|
|
|
_ workspace.Project,
|
|
|
|
_ deploy.Target,
|
|
|
|
entries engine.JournalEntries,
|
|
|
|
_ []engine.Event,
|
2023-10-11 14:44:09 +00:00
|
|
|
_ error,
|
|
|
|
) error {
|
2023-05-04 23:28:49 +00:00
|
|
|
journalEntries = entries
|
|
|
|
return nil
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
}
|
|
|
|
p.Run(t, nil)
|
|
|
|
|
|
|
|
snaps := make([]*apitype.DeploymentV3, len(journalEntries))
|
|
|
|
for i := range journalEntries {
|
|
|
|
snap, err := journalEntries[:i].Snap(nil)
|
|
|
|
require.NoError(t, err)
|
Lift context parameter to SerializeDeployment/Resource/Operations/Properties (#15929)
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
SerializePropertyValue needed a `context.Context` object to pass to the
`config.Encrypter`. It was using `context.TODO()`, this change instead
accepts a context on the parameters and lifts that up to
SerializeProperties, SerializeResource, SerializeOperation, and
SerializeDeployment.
There were a few call sites for those methods that already had a context
on hand, and they now pass that context. The other calls sites now use
`context.TODO()`, we should continue to iterate in this area to ensure
everywhere that needs a context has one passed in.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-15 07:45:46 +00:00
|
|
|
deployment, err := stack.SerializeDeployment(context.Background(), snap, true)
|
2023-05-04 23:28:49 +00:00
|
|
|
require.NoError(t, err)
|
|
|
|
snaps[i] = deployment
|
|
|
|
}
|
|
|
|
return snaps
|
|
|
|
}
|
|
|
|
|
[snapshot] Use a newer diff package
The version of gopls that gotextdiff is derived from is over three years
old. In the intervening time, the differ used by gopls has improved
tremendously, and the language has grown new features like generics.
These changes use a different diff package that is derived from an
up-to-date version of gopls and exposes an API that is parameterized
over the storage of the text. Taken together, this allows us to use a
much faster and more memory-efficient algorithm for diffing while
safely avoiding string copies when invoking the differ.
The more efficient algorithm operates on arbitrary slices whose elements
are comparable. Naive use of this algorithm--passing byte slices
in--can cause increased time spent diffing, as the sequences being
compared are longer. In order to avoid this, these changes record
spans of marshaled deployments as the deployments are marshaled. These
spans are strings that cover the diffable elements of the deployment:
its header, its resources, and its pending operations.
The output of the newer algorithm is converible to the format we
currently send to the service, so these changes should not require
service-side changes.
In order to avoid bumping the minimum Go version required to build the
`github.com/pulumi/pulumi/pkg/v3" and therefore break backwards
compatibility with consumers using older toolchains (e.g. providers),
these changes use the existing differ when built with a pre-1.20
toolchain. Official builds of the CLI are already using Go 1.20+ and
should use the new differ without additional changes.
These changes improve CPU time, allocation volume, and the compression
ratio by 57%, 87%, and 52%, respectively.
Benchmark results and analysis:
httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee spanned.txt
httpstate ❯ benchstat buffer-pool.txt spanned.txt >stat.txt
```
│ buffer-pool.txt │ spanned.txt │
│ sec/op │ sec/op vs base │
DiffStack/1_x_2_B-10 45.09µ ± 1% 22.81µ ± 1% -49.41% (p=0.000 n=10)
DiffStack/2_x_2_B-10 79.49µ ± 1% 40.60µ ± 1% -48.93% (p=0.000 n=10)
DiffStack/4_x_2_B-10 172.54µ ± 2% 81.57µ ± 0% -52.72% (p=0.000 n=10)
DiffStack/8_x_2_B-10 455.4µ ± 1% 200.4µ ± 0% -56.00% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1388.2µ ± 1% 575.1µ ± 1% -58.57% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4.409m ± 1% 1.866m ± 0% -57.67% (p=0.000 n=10)
DiffStack/48_x_2_B-10 9.275m ± 0% 3.484m ± 0% -62.44% (p=0.000 n=10)
DiffStack/64_x_2_B-10 14.564m ± 0% 5.945m ± 1% -59.18% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 124.81µ ± 1% 40.14µ ± 1% -67.84% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 299.8µ ± 0% 121.8µ ± 1% -59.38% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 846.2µ ± 1% 355.8µ ± 0% -57.96% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 2.743m ± 1% 1.150m ± 0% -58.06% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 9.271m ± 1% 4.041m ± 0% -56.41% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 31.78m ± 0% 14.97m ± 0% -52.88% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 66.14m ± 1% 30.66m ± 0% -53.63% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 107.50m ± 0% 53.44m ± 0% -50.29% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 240.51µ ± 1% 76.95µ ± 0% -68.01% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 798.5µ ± 0% 310.6µ ± 0% -61.10% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 2451.9µ ± 1% 978.9µ ± 3% -60.08% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 8.258m ± 0% 3.731m ± 0% -54.82% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 28.29m ± 1% 12.77m ± 0% -54.85% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 104.11m ± 1% 44.84m ± 1% -56.93% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 199.73m ± 1% 95.87m ± 1% -52.00% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 339.6m ± 2% 159.2m ± 2% -53.12% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 2721.7µ ± 0% 984.3µ ± 0% -63.84% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 8.334m ± 1% 3.654m ± 2% -56.15% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 25.89m ± 0% 12.23m ± 0% -52.77% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 87.49m ± 1% 40.26m ± 0% -53.98% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 317.0m ± 5% 150.4m ± 0% -52.57% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 681.8m ± 1% 332.1m ± 0% -51.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 1190.4m ± 1% 583.1m ± 0% -51.01% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 3.215m ± 0% 1.086m ± 0% -66.21% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 9.676m ± 1% 3.994m ± 0% -58.72% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 29.84m ± 0% 12.88m ± 1% -56.84% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 92.11m ± 1% 42.35m ± 0% -54.02% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 322.1m ± 0% 154.0m ± 0% -52.19% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 54.73m ± 1% 22.63m ± 0% -58.65% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 815.1m ± 0% 395.5m ± 2% -51.48% (p=0.000 n=10)
geomean 10.45m 4.530m -56.65%
│ buffer-pool.txt │ spanned.txt │
│ checkpoint_bytes │ checkpoint_bytes vs base │
DiffStack/1_x_2_B-10 2.539k ± 0% 1.950k ± 0% -23.20% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.201k ± 0% 4.548k ± 0% -12.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 13.63k ± 0% 12.30k ± 0% -9.77% (p=0.000 n=10)
DiffStack/8_x_2_B-10 45.09k ± 0% 42.86k ± 0% -4.95% (p=0.000 n=10)
DiffStack/16_x_2_B-10 161.9k ± 0% 156.4k ± 0% -3.42% (p=0.000 n=10)
DiffStack/32_x_2_B-10 579.8k ± 0% 604.2k ± 0% +4.21% (p=0.000 n=10)
DiffStack/48_x_2_B-10 1.306M ± 0% 1.180M ± 0% -9.68% (p=0.000 n=10)
DiffStack/64_x_2_B-10 2.164M ± 0% 2.134M ± 0% -1.36% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 27.109k ± 0% 9.856k ± 0% -63.64% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 78.91k ± 0% 61.59k ± 0% -21.94% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 284.2k ± 0% 258.3k ± 0% -9.13% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 1.119M ± 0% 1.041M ± 0% -6.89% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 4.389M ± 0% 4.201M ± 0% -4.29% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 16.74M ± 0% 17.06M ± 0% +1.95% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 36.30M ± 0% 35.93M ± 0% -1.00% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 60.98M ± 0% 64.06M ± 0% +5.05% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 67.79k ± 0% 34.43k ± 0% -49.20% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 300.1k ± 0% 233.6k ± 0% -22.15% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1128.3k ± 0% 929.5k ± 0% -17.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 4.437M ± 0% 4.238M ± 0% -4.49% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 16.57M ± 0% 15.78M ± 0% -4.79% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 64.35M ± 0% 58.66M ± 0% -8.84% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 127.0M ± 0% 128.0M ± 0% +0.80% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 217.7M ± 0% 216.5M ± 0% -0.54% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1316.2k ± 0% 921.8k ± 0% -29.97% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 4.733M ± 0% 4.207M ± 0% -11.11% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 16.03M ± 0% 15.77M ± 0% -1.64% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 57.42M ± 0% 54.66M ± 0% -4.81% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 214.8M ± 0% 210.7M ± 0% -1.90% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 472.9M ± 0% 466.9M ± 0% -1.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 832.3M ± 0% 824.7M ± 0% -0.92% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.575M ± 0% 1.051M ± 0% -33.32% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 5.248M ± 0% 4.199M ± 0% -20.00% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 18.36M ± 0% 15.74M ± 0% -14.28% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 60.86M ± 0% 56.66M ± 0% -6.90% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 223.5M ± 0% 214.0M ± 0% -4.23% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 13.21M ± 0% 13.20M ± 0% -0.01% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 155.9M ± 0% 154.9M ± 0% -0.60% (p=0.000 n=10)
geomean 3.606M 3.173M -12.01%
│ buffer-pool.txt │ spanned.txt │
│ ratio │ ratio vs base │
DiffStack/1_x_2_B-10 789.5m ± 0% 1641.0m ± 0% +107.85% (p=0.000 n=10)
DiffStack/2_x_2_B-10 1.034 ± 0% 2.353 ± 0% +127.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1.575 ± 0% 3.592 ± 0% +128.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 2.829 ± 0% 6.684 ± 0% +136.27% (p=0.000 n=10)
DiffStack/16_x_2_B-10 5.296 ± 0% 12.610 ± 0% +138.10% (p=0.000 n=10)
DiffStack/32_x_2_B-10 9.681 ± 0% 24.790 ± 0% +156.07% (p=0.000 n=10)
DiffStack/48_x_2_B-10 14.65 ± 0% 32.44 ± 0% +121.43% (p=0.000 n=10)
DiffStack/64_x_2_B-10 18.26 ± 0% 44.11 ± 0% +141.57% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 1.383 ± 0% 1.051 ± 0% -24.01% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 1.716 ± 0% 2.324 ± 0% +35.43% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 2.878 ± 0% 4.251 ± 0% +47.71% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 5.475 ± 0% 8.056 ± 0% +47.14% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.56 ± 0% 15.77 ± 0% +49.34% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 19.97 ± 0% 31.57 ± 0% +58.09% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 28.80 ± 0% 44.11 ± 0% +53.16% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 36.23 ± 0% 58.84 ± 0% +62.41% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 986.0m ± 0% 1014.0m ± 0% +2.84% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 1.777 ± 0% 2.331 ± 0% +31.18% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 3.057 ± 0% 3.993 ± 0% +30.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 5.766 ± 0% 8.511 ± 0% +47.61% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.55 ± 0% 15.35 ± 0% +45.50% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 20.28 ± 0% 28.08 ± 0% +38.46% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 26.59 ± 0% 40.64 ± 0% +52.84% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 34.14 ± 0% 51.44 ± 0% +50.67% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1.993 ± 0% 2.333 ± 0% +17.06% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 3.263 ± 0% 4.569 ± 0% +40.02% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 5.291 ± 0% 7.995 ± 0% +51.11% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.275 ± 0% 13.410 ± 0% +44.58% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 17.17 ± 0% 25.44 ± 0% +48.17% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 25.11 ± 0% 37.39 ± 0% +48.90% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 33.09 ± 0% 49.40 ± 0% +49.29% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.498 ± 0% 1.999 ± 0% +33.44% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 1.998 ± 0% 2.666 ± 0% +33.43% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 3.180 ± 0% 4.285 ± 0% +34.75% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 5.040 ± 0% 7.199 ± 0% +42.84% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.058 ± 0% 13.160 ± 0% +45.29% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% 1.999 ± 0% +0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 36.60 ± 0% 40.42 ± 0% +10.44% (p=0.000 n=10)
geomean 5.993 9.122 +52.20%
│ buffer-pool.txt │ spanned.txt │
│ wire_bytes │ wire_bytes vs base │
DiffStack/1_x_2_B-10 3.216k ± 0% 1.188k ± 0% -63.06% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.031k ± 0% 1.933k ± 0% -61.58% (p=0.000 n=10)
DiffStack/4_x_2_B-10 8.655k ± 0% 3.424k ± 0% -60.44% (p=0.000 n=10)
DiffStack/8_x_2_B-10 15.938k ± 0% 6.412k ± 0% -59.77% (p=0.000 n=10)
DiffStack/16_x_2_B-10 30.58k ± 0% 12.40k ± 0% -59.45% (p=0.000 n=10)
DiffStack/32_x_2_B-10 59.89k ± 0% 24.37k ± 0% -59.31% (p=0.000 n=10)
DiffStack/48_x_2_B-10 89.17k ± 0% 36.38k ± 0% -59.21% (p=0.000 n=10)
DiffStack/64_x_2_B-10 118.48k ± 0% 48.39k ± 0% -59.16% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 19.596k ± 0% 9.378k ± 0% -52.14% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 45.98k ± 0% 26.50k ± 0% -42.36% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 98.75k ± 0% 60.76k ± 0% -38.47% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 204.3k ± 0% 129.3k ± 0% -36.72% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 415.5k ± 0% 266.3k ± 0% -35.91% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 837.9k ± 0% 540.4k ± 0% -35.50% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 1260.4k ± 0% 814.6k ± 0% -35.37% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 1.683M ± 0% 1.089M ± 0% -35.31% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 68.75k ± 0% 33.95k ± 0% -50.61% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 168.9k ± 0% 100.2k ± 0% -40.64% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 369.1k ± 0% 232.8k ± 0% -36.93% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 769.6k ± 0% 497.9k ± 0% -35.30% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 1.571M ± 0% 1.028M ± 0% -34.53% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 3.173M ± 0% 2.089M ± 0% -34.16% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 4.775M ± 0% 3.149M ± 0% -34.04% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 6.377M ± 0% 4.210M ± 0% -33.98% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 660.4k ± 0% 395.1k ± 0% -40.16% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1450.4k ± 0% 920.9k ± 0% -36.51% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.031M ± 0% 1.972M ± 0% -34.91% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 6.191M ± 0% 4.076M ± 0% -34.17% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 12.512M ± 0% 8.282M ± 0% -33.81% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 18.83M ± 0% 12.49M ± 0% -33.69% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 25.15M ± 0% 16.69M ± 0% -33.63% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1051.8k ± 0% 525.5k ± 0% -50.04% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 2.626M ± 0% 1.575M ± 0% -40.04% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 5.776M ± 0% 3.673M ± 0% -36.40% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 12.075M ± 0% 7.871M ± 0% -34.82% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 24.67M ± 0% 16.27M ± 0% -34.07% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 6.612M ± 0% 6.606M ± 0% -0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 4.259M ± 0% 3.833M ± 0% -10.00% (p=0.000 n=10)
geomean 601.7k 347.8k -42.19%
│ buffer-pool.txt │ spanned.txt │
│ B/op │ B/op vs base │
DiffStack/1_x_2_B-10 31.85Ki ± 0% 15.79Ki ± 0% -50.42% (p=0.000 n=10)
DiffStack/2_x_2_B-10 61.81Ki ± 0% 28.01Ki ± 0% -54.68% (p=0.000 n=10)
DiffStack/4_x_2_B-10 147.58Ki ± 0% 55.60Ki ± 0% -62.33% (p=0.000 n=10)
DiffStack/8_x_2_B-10 425.9Ki ± 0% 116.9Ki ± 0% -72.55% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1429.9Ki ± 0% 307.8Ki ± 0% -78.47% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4955.7Ki ± 0% 934.1Ki ± 0% -81.15% (p=0.000 n=10)
DiffStack/48_x_2_B-10 10.746Mi ± 0% 1.682Mi ± 0% -84.34% (p=0.000 n=10)
DiffStack/64_x_2_B-10 17.664Mi ± 0% 2.822Mi ± 0% -84.03% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 202.50Ki ± 0% 41.18Ki ± 0% -79.66% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 511.5Ki ± 0% 134.1Ki ± 0% -73.78% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1513.5Ki ± 0% 309.4Ki ± 0% -79.56% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 4886.0Ki ± 0% 674.2Ki ± 0% -86.20% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 16.287Mi ± 0% 1.427Mi ± 0% -91.24% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 57.541Mi ± 0% 3.198Mi ± 0% -94.44% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 121.703Mi ± 0% 5.810Mi ± 0% -95.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 200.588Mi ± 0% 7.495Mi ± 0% -96.26% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 562.5Ki ± 0% 136.3Ki ± 0% -75.78% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 2041.0Ki ± 0% 505.5Ki ± 0% -75.23% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 5.874Mi ± 0% 1.129Mi ± 0% -80.78% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 18.837Mi ± 0% 2.410Mi ± 0% -87.21% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 61.898Mi ± 0% 5.015Mi ± 0% -91.90% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 221.69Mi ± 0% 10.36Mi ± 0% -95.32% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 430.13Mi ± 0% 18.50Mi ± 0% -95.70% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 722.07Mi ± 0% 21.63Mi ± 0% -97.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 8.098Mi ± 0% 1.656Mi ± 0% -79.55% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 22.662Mi ± 0% 3.934Mi ± 0% -82.64% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 65.217Mi ± 0% 8.425Mi ± 0% -87.08% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 204.90Mi ± 0% 17.22Mi ± 1% -91.60% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 703.12Mi ± 0% 34.89Mi ± 0% -95.04% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 1505.36Mi ± 0% 61.24Mi ± 0% -95.93% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 2586.04Mi ± 0% 70.48Mi ± 1% -97.27% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 11.854Mi ± 1% 2.127Mi ± 0% -82.06% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 32.932Mi ± 1% 6.558Mi ± 1% -80.09% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 90.04Mi ± 0% 15.15Mi ± 0% -83.17% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 252.94Mi ± 0% 32.27Mi ± 1% -87.24% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 801.77Mi ± 0% 66.46Mi ± 1% -91.71% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 103.67Mi ± 0% 35.52Mi ± 0% -65.74% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 913.90Mi ± 0% 53.53Mi ± 1% -94.14% (p=0.000 n=10)
geomean 19.16Mi 2.409Mi -87.43%
│ buffer-pool.txt │ spanned.txt │
│ allocs/op │ allocs/op vs base │
DiffStack/1_x_2_B-10 344.0 ± 0% 184.0 ± 0% -46.51% (p=0.000 n=10)
DiffStack/2_x_2_B-10 616.0 ± 0% 295.0 ± 0% -52.11% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1381.0 ± 0% 524.0 ± 0% -62.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 3.922k ± 0% 1.032k ± 0% -73.69% (p=0.000 n=10)
DiffStack/16_x_2_B-10 12.791k ± 0% 2.233k ± 0% -82.54% (p=0.000 n=10)
DiffStack/32_x_2_B-10 43.416k ± 0% 5.410k ± 0% -87.54% (p=0.000 n=10)
DiffStack/48_x_2_B-10 95.803k ± 0% 9.036k ± 0% -90.57% (p=0.000 n=10)
DiffStack/64_x_2_B-10 156.99k ± 0% 13.99k ± 0% -91.09% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 335.0 ± 0% 182.0 ± 0% -45.67% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 597.5 ± 0% 295.0 ± 0% -50.63% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1304.0 ± 0% 527.0 ± 0% -59.59% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 3.475k ± 0% 1.036k ± 0% -70.19% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.677k ± 0% 2.239k ± 0% -79.03% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 35.357k ± 0% 5.414k ± 0% -84.69% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 72.782k ± 0% 9.296k ± 0% -87.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 119.24k ± 0% 14.26k ± 0% -88.04% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 309.0 ± 0% 182.0 ± 0% -41.10% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 594.0 ± 0% 296.0 ± 0% -50.17% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1324.5 ± 0% 528.0 ± 0% -60.14% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 3.554k ± 0% 1.047k ± 0% -70.54% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.511k ± 0% 2.232k ± 0% -78.77% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 35.147k ± 0% 5.194k ± 0% -85.22% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 66.519k ± 0% 8.958k ± 0% -86.53% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 110.71k ± 0% 13.28k ± 0% -88.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 644.0 ± 0% 296.0 ± 0% -54.04% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1418.5 ± 0% 535.0 ± 0% -62.28% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.497k ± 0% 1.047k ± 0% -70.06% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.805k ± 0% 2.182k ± 0% -77.75% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 30.996k ± 0% 5.040k ± 0% -83.74% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 63.751k ± 0% 8.656k ± 0% -86.42% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 107.97k ± 0% 13.03k ± 0% -87.93% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 356.0 ± 0% 186.0 ± 0% -47.75% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 671.0 ± 0% 300.0 ± 0% -55.29% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 1467.5 ± 0% 536.0 ± 0% -63.48% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 3.553k ± 0% 1.038k ± 0% -70.80% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.971k ± 0% 2.182k ± 0% -78.12% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 489.16k ± 0% 73.65k ± 0% -84.94% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 7236.9k ± 0% 857.8k ± 0% -88.15% (p=0.000 n=10)
geomean 7.989k 1.934k -75.79%
```
2023-05-05 02:15:09 +00:00
|
|
|
func testMarshalDeployment(t *testing.T, snaps []*apitype.DeploymentV3) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
[snapshot] Use a newer diff package
The version of gopls that gotextdiff is derived from is over three years
old. In the intervening time, the differ used by gopls has improved
tremendously, and the language has grown new features like generics.
These changes use a different diff package that is derived from an
up-to-date version of gopls and exposes an API that is parameterized
over the storage of the text. Taken together, this allows us to use a
much faster and more memory-efficient algorithm for diffing while
safely avoiding string copies when invoking the differ.
The more efficient algorithm operates on arbitrary slices whose elements
are comparable. Naive use of this algorithm--passing byte slices
in--can cause increased time spent diffing, as the sequences being
compared are longer. In order to avoid this, these changes record
spans of marshaled deployments as the deployments are marshaled. These
spans are strings that cover the diffable elements of the deployment:
its header, its resources, and its pending operations.
The output of the newer algorithm is converible to the format we
currently send to the service, so these changes should not require
service-side changes.
In order to avoid bumping the minimum Go version required to build the
`github.com/pulumi/pulumi/pkg/v3" and therefore break backwards
compatibility with consumers using older toolchains (e.g. providers),
these changes use the existing differ when built with a pre-1.20
toolchain. Official builds of the CLI are already using Go 1.20+ and
should use the new differ without additional changes.
These changes improve CPU time, allocation volume, and the compression
ratio by 57%, 87%, and 52%, respectively.
Benchmark results and analysis:
httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee spanned.txt
httpstate ❯ benchstat buffer-pool.txt spanned.txt >stat.txt
```
│ buffer-pool.txt │ spanned.txt │
│ sec/op │ sec/op vs base │
DiffStack/1_x_2_B-10 45.09µ ± 1% 22.81µ ± 1% -49.41% (p=0.000 n=10)
DiffStack/2_x_2_B-10 79.49µ ± 1% 40.60µ ± 1% -48.93% (p=0.000 n=10)
DiffStack/4_x_2_B-10 172.54µ ± 2% 81.57µ ± 0% -52.72% (p=0.000 n=10)
DiffStack/8_x_2_B-10 455.4µ ± 1% 200.4µ ± 0% -56.00% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1388.2µ ± 1% 575.1µ ± 1% -58.57% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4.409m ± 1% 1.866m ± 0% -57.67% (p=0.000 n=10)
DiffStack/48_x_2_B-10 9.275m ± 0% 3.484m ± 0% -62.44% (p=0.000 n=10)
DiffStack/64_x_2_B-10 14.564m ± 0% 5.945m ± 1% -59.18% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 124.81µ ± 1% 40.14µ ± 1% -67.84% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 299.8µ ± 0% 121.8µ ± 1% -59.38% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 846.2µ ± 1% 355.8µ ± 0% -57.96% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 2.743m ± 1% 1.150m ± 0% -58.06% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 9.271m ± 1% 4.041m ± 0% -56.41% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 31.78m ± 0% 14.97m ± 0% -52.88% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 66.14m ± 1% 30.66m ± 0% -53.63% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 107.50m ± 0% 53.44m ± 0% -50.29% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 240.51µ ± 1% 76.95µ ± 0% -68.01% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 798.5µ ± 0% 310.6µ ± 0% -61.10% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 2451.9µ ± 1% 978.9µ ± 3% -60.08% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 8.258m ± 0% 3.731m ± 0% -54.82% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 28.29m ± 1% 12.77m ± 0% -54.85% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 104.11m ± 1% 44.84m ± 1% -56.93% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 199.73m ± 1% 95.87m ± 1% -52.00% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 339.6m ± 2% 159.2m ± 2% -53.12% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 2721.7µ ± 0% 984.3µ ± 0% -63.84% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 8.334m ± 1% 3.654m ± 2% -56.15% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 25.89m ± 0% 12.23m ± 0% -52.77% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 87.49m ± 1% 40.26m ± 0% -53.98% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 317.0m ± 5% 150.4m ± 0% -52.57% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 681.8m ± 1% 332.1m ± 0% -51.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 1190.4m ± 1% 583.1m ± 0% -51.01% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 3.215m ± 0% 1.086m ± 0% -66.21% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 9.676m ± 1% 3.994m ± 0% -58.72% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 29.84m ± 0% 12.88m ± 1% -56.84% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 92.11m ± 1% 42.35m ± 0% -54.02% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 322.1m ± 0% 154.0m ± 0% -52.19% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 54.73m ± 1% 22.63m ± 0% -58.65% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 815.1m ± 0% 395.5m ± 2% -51.48% (p=0.000 n=10)
geomean 10.45m 4.530m -56.65%
│ buffer-pool.txt │ spanned.txt │
│ checkpoint_bytes │ checkpoint_bytes vs base │
DiffStack/1_x_2_B-10 2.539k ± 0% 1.950k ± 0% -23.20% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.201k ± 0% 4.548k ± 0% -12.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 13.63k ± 0% 12.30k ± 0% -9.77% (p=0.000 n=10)
DiffStack/8_x_2_B-10 45.09k ± 0% 42.86k ± 0% -4.95% (p=0.000 n=10)
DiffStack/16_x_2_B-10 161.9k ± 0% 156.4k ± 0% -3.42% (p=0.000 n=10)
DiffStack/32_x_2_B-10 579.8k ± 0% 604.2k ± 0% +4.21% (p=0.000 n=10)
DiffStack/48_x_2_B-10 1.306M ± 0% 1.180M ± 0% -9.68% (p=0.000 n=10)
DiffStack/64_x_2_B-10 2.164M ± 0% 2.134M ± 0% -1.36% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 27.109k ± 0% 9.856k ± 0% -63.64% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 78.91k ± 0% 61.59k ± 0% -21.94% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 284.2k ± 0% 258.3k ± 0% -9.13% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 1.119M ± 0% 1.041M ± 0% -6.89% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 4.389M ± 0% 4.201M ± 0% -4.29% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 16.74M ± 0% 17.06M ± 0% +1.95% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 36.30M ± 0% 35.93M ± 0% -1.00% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 60.98M ± 0% 64.06M ± 0% +5.05% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 67.79k ± 0% 34.43k ± 0% -49.20% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 300.1k ± 0% 233.6k ± 0% -22.15% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1128.3k ± 0% 929.5k ± 0% -17.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 4.437M ± 0% 4.238M ± 0% -4.49% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 16.57M ± 0% 15.78M ± 0% -4.79% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 64.35M ± 0% 58.66M ± 0% -8.84% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 127.0M ± 0% 128.0M ± 0% +0.80% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 217.7M ± 0% 216.5M ± 0% -0.54% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1316.2k ± 0% 921.8k ± 0% -29.97% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 4.733M ± 0% 4.207M ± 0% -11.11% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 16.03M ± 0% 15.77M ± 0% -1.64% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 57.42M ± 0% 54.66M ± 0% -4.81% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 214.8M ± 0% 210.7M ± 0% -1.90% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 472.9M ± 0% 466.9M ± 0% -1.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 832.3M ± 0% 824.7M ± 0% -0.92% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.575M ± 0% 1.051M ± 0% -33.32% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 5.248M ± 0% 4.199M ± 0% -20.00% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 18.36M ± 0% 15.74M ± 0% -14.28% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 60.86M ± 0% 56.66M ± 0% -6.90% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 223.5M ± 0% 214.0M ± 0% -4.23% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 13.21M ± 0% 13.20M ± 0% -0.01% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 155.9M ± 0% 154.9M ± 0% -0.60% (p=0.000 n=10)
geomean 3.606M 3.173M -12.01%
│ buffer-pool.txt │ spanned.txt │
│ ratio │ ratio vs base │
DiffStack/1_x_2_B-10 789.5m ± 0% 1641.0m ± 0% +107.85% (p=0.000 n=10)
DiffStack/2_x_2_B-10 1.034 ± 0% 2.353 ± 0% +127.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1.575 ± 0% 3.592 ± 0% +128.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 2.829 ± 0% 6.684 ± 0% +136.27% (p=0.000 n=10)
DiffStack/16_x_2_B-10 5.296 ± 0% 12.610 ± 0% +138.10% (p=0.000 n=10)
DiffStack/32_x_2_B-10 9.681 ± 0% 24.790 ± 0% +156.07% (p=0.000 n=10)
DiffStack/48_x_2_B-10 14.65 ± 0% 32.44 ± 0% +121.43% (p=0.000 n=10)
DiffStack/64_x_2_B-10 18.26 ± 0% 44.11 ± 0% +141.57% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 1.383 ± 0% 1.051 ± 0% -24.01% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 1.716 ± 0% 2.324 ± 0% +35.43% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 2.878 ± 0% 4.251 ± 0% +47.71% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 5.475 ± 0% 8.056 ± 0% +47.14% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.56 ± 0% 15.77 ± 0% +49.34% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 19.97 ± 0% 31.57 ± 0% +58.09% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 28.80 ± 0% 44.11 ± 0% +53.16% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 36.23 ± 0% 58.84 ± 0% +62.41% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 986.0m ± 0% 1014.0m ± 0% +2.84% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 1.777 ± 0% 2.331 ± 0% +31.18% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 3.057 ± 0% 3.993 ± 0% +30.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 5.766 ± 0% 8.511 ± 0% +47.61% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.55 ± 0% 15.35 ± 0% +45.50% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 20.28 ± 0% 28.08 ± 0% +38.46% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 26.59 ± 0% 40.64 ± 0% +52.84% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 34.14 ± 0% 51.44 ± 0% +50.67% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1.993 ± 0% 2.333 ± 0% +17.06% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 3.263 ± 0% 4.569 ± 0% +40.02% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 5.291 ± 0% 7.995 ± 0% +51.11% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.275 ± 0% 13.410 ± 0% +44.58% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 17.17 ± 0% 25.44 ± 0% +48.17% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 25.11 ± 0% 37.39 ± 0% +48.90% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 33.09 ± 0% 49.40 ± 0% +49.29% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.498 ± 0% 1.999 ± 0% +33.44% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 1.998 ± 0% 2.666 ± 0% +33.43% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 3.180 ± 0% 4.285 ± 0% +34.75% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 5.040 ± 0% 7.199 ± 0% +42.84% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.058 ± 0% 13.160 ± 0% +45.29% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% 1.999 ± 0% +0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 36.60 ± 0% 40.42 ± 0% +10.44% (p=0.000 n=10)
geomean 5.993 9.122 +52.20%
│ buffer-pool.txt │ spanned.txt │
│ wire_bytes │ wire_bytes vs base │
DiffStack/1_x_2_B-10 3.216k ± 0% 1.188k ± 0% -63.06% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.031k ± 0% 1.933k ± 0% -61.58% (p=0.000 n=10)
DiffStack/4_x_2_B-10 8.655k ± 0% 3.424k ± 0% -60.44% (p=0.000 n=10)
DiffStack/8_x_2_B-10 15.938k ± 0% 6.412k ± 0% -59.77% (p=0.000 n=10)
DiffStack/16_x_2_B-10 30.58k ± 0% 12.40k ± 0% -59.45% (p=0.000 n=10)
DiffStack/32_x_2_B-10 59.89k ± 0% 24.37k ± 0% -59.31% (p=0.000 n=10)
DiffStack/48_x_2_B-10 89.17k ± 0% 36.38k ± 0% -59.21% (p=0.000 n=10)
DiffStack/64_x_2_B-10 118.48k ± 0% 48.39k ± 0% -59.16% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 19.596k ± 0% 9.378k ± 0% -52.14% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 45.98k ± 0% 26.50k ± 0% -42.36% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 98.75k ± 0% 60.76k ± 0% -38.47% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 204.3k ± 0% 129.3k ± 0% -36.72% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 415.5k ± 0% 266.3k ± 0% -35.91% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 837.9k ± 0% 540.4k ± 0% -35.50% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 1260.4k ± 0% 814.6k ± 0% -35.37% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 1.683M ± 0% 1.089M ± 0% -35.31% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 68.75k ± 0% 33.95k ± 0% -50.61% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 168.9k ± 0% 100.2k ± 0% -40.64% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 369.1k ± 0% 232.8k ± 0% -36.93% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 769.6k ± 0% 497.9k ± 0% -35.30% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 1.571M ± 0% 1.028M ± 0% -34.53% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 3.173M ± 0% 2.089M ± 0% -34.16% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 4.775M ± 0% 3.149M ± 0% -34.04% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 6.377M ± 0% 4.210M ± 0% -33.98% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 660.4k ± 0% 395.1k ± 0% -40.16% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1450.4k ± 0% 920.9k ± 0% -36.51% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.031M ± 0% 1.972M ± 0% -34.91% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 6.191M ± 0% 4.076M ± 0% -34.17% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 12.512M ± 0% 8.282M ± 0% -33.81% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 18.83M ± 0% 12.49M ± 0% -33.69% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 25.15M ± 0% 16.69M ± 0% -33.63% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1051.8k ± 0% 525.5k ± 0% -50.04% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 2.626M ± 0% 1.575M ± 0% -40.04% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 5.776M ± 0% 3.673M ± 0% -36.40% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 12.075M ± 0% 7.871M ± 0% -34.82% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 24.67M ± 0% 16.27M ± 0% -34.07% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 6.612M ± 0% 6.606M ± 0% -0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 4.259M ± 0% 3.833M ± 0% -10.00% (p=0.000 n=10)
geomean 601.7k 347.8k -42.19%
│ buffer-pool.txt │ spanned.txt │
│ B/op │ B/op vs base │
DiffStack/1_x_2_B-10 31.85Ki ± 0% 15.79Ki ± 0% -50.42% (p=0.000 n=10)
DiffStack/2_x_2_B-10 61.81Ki ± 0% 28.01Ki ± 0% -54.68% (p=0.000 n=10)
DiffStack/4_x_2_B-10 147.58Ki ± 0% 55.60Ki ± 0% -62.33% (p=0.000 n=10)
DiffStack/8_x_2_B-10 425.9Ki ± 0% 116.9Ki ± 0% -72.55% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1429.9Ki ± 0% 307.8Ki ± 0% -78.47% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4955.7Ki ± 0% 934.1Ki ± 0% -81.15% (p=0.000 n=10)
DiffStack/48_x_2_B-10 10.746Mi ± 0% 1.682Mi ± 0% -84.34% (p=0.000 n=10)
DiffStack/64_x_2_B-10 17.664Mi ± 0% 2.822Mi ± 0% -84.03% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 202.50Ki ± 0% 41.18Ki ± 0% -79.66% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 511.5Ki ± 0% 134.1Ki ± 0% -73.78% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1513.5Ki ± 0% 309.4Ki ± 0% -79.56% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 4886.0Ki ± 0% 674.2Ki ± 0% -86.20% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 16.287Mi ± 0% 1.427Mi ± 0% -91.24% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 57.541Mi ± 0% 3.198Mi ± 0% -94.44% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 121.703Mi ± 0% 5.810Mi ± 0% -95.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 200.588Mi ± 0% 7.495Mi ± 0% -96.26% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 562.5Ki ± 0% 136.3Ki ± 0% -75.78% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 2041.0Ki ± 0% 505.5Ki ± 0% -75.23% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 5.874Mi ± 0% 1.129Mi ± 0% -80.78% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 18.837Mi ± 0% 2.410Mi ± 0% -87.21% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 61.898Mi ± 0% 5.015Mi ± 0% -91.90% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 221.69Mi ± 0% 10.36Mi ± 0% -95.32% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 430.13Mi ± 0% 18.50Mi ± 0% -95.70% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 722.07Mi ± 0% 21.63Mi ± 0% -97.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 8.098Mi ± 0% 1.656Mi ± 0% -79.55% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 22.662Mi ± 0% 3.934Mi ± 0% -82.64% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 65.217Mi ± 0% 8.425Mi ± 0% -87.08% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 204.90Mi ± 0% 17.22Mi ± 1% -91.60% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 703.12Mi ± 0% 34.89Mi ± 0% -95.04% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 1505.36Mi ± 0% 61.24Mi ± 0% -95.93% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 2586.04Mi ± 0% 70.48Mi ± 1% -97.27% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 11.854Mi ± 1% 2.127Mi ± 0% -82.06% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 32.932Mi ± 1% 6.558Mi ± 1% -80.09% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 90.04Mi ± 0% 15.15Mi ± 0% -83.17% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 252.94Mi ± 0% 32.27Mi ± 1% -87.24% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 801.77Mi ± 0% 66.46Mi ± 1% -91.71% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 103.67Mi ± 0% 35.52Mi ± 0% -65.74% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 913.90Mi ± 0% 53.53Mi ± 1% -94.14% (p=0.000 n=10)
geomean 19.16Mi 2.409Mi -87.43%
│ buffer-pool.txt │ spanned.txt │
│ allocs/op │ allocs/op vs base │
DiffStack/1_x_2_B-10 344.0 ± 0% 184.0 ± 0% -46.51% (p=0.000 n=10)
DiffStack/2_x_2_B-10 616.0 ± 0% 295.0 ± 0% -52.11% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1381.0 ± 0% 524.0 ± 0% -62.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 3.922k ± 0% 1.032k ± 0% -73.69% (p=0.000 n=10)
DiffStack/16_x_2_B-10 12.791k ± 0% 2.233k ± 0% -82.54% (p=0.000 n=10)
DiffStack/32_x_2_B-10 43.416k ± 0% 5.410k ± 0% -87.54% (p=0.000 n=10)
DiffStack/48_x_2_B-10 95.803k ± 0% 9.036k ± 0% -90.57% (p=0.000 n=10)
DiffStack/64_x_2_B-10 156.99k ± 0% 13.99k ± 0% -91.09% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 335.0 ± 0% 182.0 ± 0% -45.67% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 597.5 ± 0% 295.0 ± 0% -50.63% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1304.0 ± 0% 527.0 ± 0% -59.59% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 3.475k ± 0% 1.036k ± 0% -70.19% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.677k ± 0% 2.239k ± 0% -79.03% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 35.357k ± 0% 5.414k ± 0% -84.69% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 72.782k ± 0% 9.296k ± 0% -87.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 119.24k ± 0% 14.26k ± 0% -88.04% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 309.0 ± 0% 182.0 ± 0% -41.10% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 594.0 ± 0% 296.0 ± 0% -50.17% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1324.5 ± 0% 528.0 ± 0% -60.14% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 3.554k ± 0% 1.047k ± 0% -70.54% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.511k ± 0% 2.232k ± 0% -78.77% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 35.147k ± 0% 5.194k ± 0% -85.22% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 66.519k ± 0% 8.958k ± 0% -86.53% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 110.71k ± 0% 13.28k ± 0% -88.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 644.0 ± 0% 296.0 ± 0% -54.04% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1418.5 ± 0% 535.0 ± 0% -62.28% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.497k ± 0% 1.047k ± 0% -70.06% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.805k ± 0% 2.182k ± 0% -77.75% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 30.996k ± 0% 5.040k ± 0% -83.74% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 63.751k ± 0% 8.656k ± 0% -86.42% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 107.97k ± 0% 13.03k ± 0% -87.93% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 356.0 ± 0% 186.0 ± 0% -47.75% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 671.0 ± 0% 300.0 ± 0% -55.29% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 1467.5 ± 0% 536.0 ± 0% -63.48% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 3.553k ± 0% 1.038k ± 0% -70.80% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.971k ± 0% 2.182k ± 0% -78.12% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 489.16k ± 0% 73.65k ± 0% -84.94% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 7236.9k ± 0% 857.8k ± 0% -88.15% (p=0.000 n=10)
geomean 7.989k 1.934k -75.79%
```
2023-05-05 02:15:09 +00:00
|
|
|
dds := newDeploymentDiffState(0)
|
|
|
|
for _, s := range snaps {
|
|
|
|
expected, err := dds.MarshalDeployment(s)
|
|
|
|
require.NoError(t, err)
|
|
|
|
|
|
|
|
marshaled, err := json.Marshal(apitype.PatchUpdateVerbatimCheckpointRequest{
|
|
|
|
Version: 3,
|
|
|
|
UntypedDeployment: expected.raw,
|
|
|
|
})
|
|
|
|
require.NoError(t, err)
|
|
|
|
|
|
|
|
var req apitype.PatchUpdateVerbatimCheckpointRequest
|
|
|
|
err = json.Unmarshal(marshaled, &req)
|
|
|
|
require.NoError(t, err)
|
|
|
|
|
|
|
|
assert.Equal(t, expected.raw, req.UntypedDeployment)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
func testDiffStack(t *testing.T, snaps []*apitype.DeploymentV3) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
ctx := context.Background()
|
|
|
|
|
|
|
|
dds := newDeploymentDiffState(0)
|
|
|
|
for _, s := range snaps {
|
2023-05-05 00:12:34 +00:00
|
|
|
json, err := dds.MarshalDeployment(s)
|
2023-05-04 23:28:49 +00:00
|
|
|
require.NoError(t, err)
|
|
|
|
if dds.ShouldDiff(json) {
|
|
|
|
d, err := dds.Diff(ctx, json)
|
|
|
|
require.NoError(t, err)
|
2023-05-05 00:12:34 +00:00
|
|
|
actual, err := applyEdits(dds.lastSavedDeployment.raw, d.deploymentDelta)
|
2023-05-04 23:28:49 +00:00
|
|
|
require.NoError(t, err)
|
2023-05-05 00:12:34 +00:00
|
|
|
assert.Equal(t, json.raw, actual)
|
2023-05-04 23:28:49 +00:00
|
|
|
}
|
|
|
|
err = dds.Saved(ctx, json)
|
|
|
|
require.NoError(t, err)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func benchmarkDiffStack(b *testing.B, snaps []*apitype.DeploymentV3) {
|
|
|
|
ctx := context.Background()
|
|
|
|
for i := 0; i < b.N; i++ {
|
|
|
|
wireSize, verbatimSize, diffs, verbatims := 0, 0, 0, 0
|
|
|
|
dds := newDeploymentDiffState(0)
|
|
|
|
|
|
|
|
for _, s := range snaps {
|
2023-05-05 00:12:34 +00:00
|
|
|
json, err := dds.MarshalDeployment(s)
|
2023-05-04 23:28:49 +00:00
|
|
|
require.NoError(b, err)
|
2023-05-05 00:12:34 +00:00
|
|
|
verbatimSize += len(json.raw)
|
2023-05-04 23:28:49 +00:00
|
|
|
if dds.ShouldDiff(json) {
|
|
|
|
diffs++
|
|
|
|
d, err := dds.Diff(ctx, json)
|
|
|
|
require.NoError(b, err)
|
|
|
|
wireSize += len(d.deploymentDelta)
|
|
|
|
} else {
|
|
|
|
verbatims++
|
2023-05-05 00:12:34 +00:00
|
|
|
wireSize += len(json.raw)
|
2023-05-04 23:28:49 +00:00
|
|
|
}
|
|
|
|
err = dds.Saved(ctx, json)
|
|
|
|
require.NoError(b, err)
|
|
|
|
}
|
|
|
|
b.ReportMetric(float64(diffs), "diffs")
|
|
|
|
b.ReportMetric(float64(verbatims), "verbatims")
|
|
|
|
b.ReportMetric(float64(wireSize), "wire_bytes")
|
|
|
|
b.ReportMetric(float64(verbatimSize), "checkpoint_bytes")
|
|
|
|
b.ReportMetric(float64(verbatimSize)/float64(wireSize), "ratio")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func pseudoRandomString(r *rand.Rand, desiredLength int) string {
|
|
|
|
buf := make([]byte, desiredLength)
|
|
|
|
r.Read(buf)
|
|
|
|
text := base64.StdEncoding.EncodeToString(buf)
|
|
|
|
return text[0:desiredLength]
|
|
|
|
}
|
|
|
|
|
|
|
|
type testingTB[TB any] interface {
|
|
|
|
testing.TB
|
|
|
|
|
|
|
|
Run(name string, inner func(tb TB)) bool
|
|
|
|
}
|
|
|
|
|
|
|
|
type diffStackTestFunc[TB testingTB[TB]] func(tb TB, snaps []*apitype.DeploymentV3)
|
|
|
|
|
|
|
|
type diffStackCase interface {
|
|
|
|
getName() string
|
|
|
|
getSnaps(t testing.TB) []*apitype.DeploymentV3
|
|
|
|
}
|
|
|
|
|
|
|
|
func testOrBenchmarkDiffStack[TB testingTB[TB]](
|
|
|
|
tb TB,
|
|
|
|
inner diffStackTestFunc[TB],
|
|
|
|
cases []diffStackCase,
|
|
|
|
) {
|
|
|
|
for _, c := range cases {
|
|
|
|
name, snaps := c.getName(), c.getSnaps(tb)
|
|
|
|
tb.Run(name, func(tb TB) {
|
|
|
|
inner(tb, snaps)
|
|
|
|
})
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
type dynamicStackCase struct {
|
|
|
|
seed int
|
|
|
|
resourceCount int
|
|
|
|
resourcePayloadBytes int
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c dynamicStackCase) getName() string {
|
|
|
|
return fmt.Sprintf("%v_x_%v", c.resourceCount, humanize.Bytes(uint64(c.resourcePayloadBytes)))
|
|
|
|
}
|
|
|
|
|
|
|
|
//nolint:gosec
|
|
|
|
func (c dynamicStackCase) getSnaps(tb testing.TB) []*apitype.DeploymentV3 {
|
|
|
|
r := rand.New(rand.NewSource(int64(c.seed)))
|
|
|
|
return generateSnapshots(tb, r, c.resourceCount, c.resourcePayloadBytes)
|
|
|
|
}
|
|
|
|
|
|
|
|
var dynamicCases = []diffStackCase{
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 1, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 2, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 4, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 8, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 16, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 32, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 48, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 64, resourcePayloadBytes: 2},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 1, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 2, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 4, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 8, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 16, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 32, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 48, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 64, resourcePayloadBytes: 8192},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 1, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 2, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 4, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 8, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 16, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 32, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 48, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 64, resourcePayloadBytes: 32768},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 2, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 4, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 8, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 16, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 32, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 48, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 64, resourcePayloadBytes: 131072},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 1, resourcePayloadBytes: 524288},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 2, resourcePayloadBytes: 524288},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 4, resourcePayloadBytes: 524288},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 8, resourcePayloadBytes: 524288},
|
|
|
|
dynamicStackCase{seed: 0, resourceCount: 16, resourcePayloadBytes: 524288},
|
|
|
|
}
|
|
|
|
|
|
|
|
func BenchmarkDiffStack(b *testing.B) {
|
|
|
|
testOrBenchmarkDiffStack(b, benchmarkDiffStack, dynamicCases)
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestDiffStack(t *testing.T) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
testOrBenchmarkDiffStack(t, testDiffStack, dynamicCases)
|
|
|
|
}
|
|
|
|
|
|
|
|
type recordedStackCase string
|
|
|
|
|
|
|
|
func (c recordedStackCase) getName() string {
|
|
|
|
return string(c)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c recordedStackCase) getSnaps(tb testing.TB) []*apitype.DeploymentV3 {
|
|
|
|
f, err := os.Open(filepath.Join("testdata", string(c)))
|
|
|
|
require.NoError(tb, err)
|
|
|
|
defer contract.IgnoreClose(f)
|
|
|
|
|
|
|
|
var deployments []*apitype.DeploymentV3
|
|
|
|
dec := json.NewDecoder(f)
|
|
|
|
for {
|
|
|
|
var d struct {
|
|
|
|
Version int
|
|
|
|
Deployment *apitype.DeploymentV3
|
|
|
|
}
|
|
|
|
err := dec.Decode(&d)
|
|
|
|
if err == io.EOF {
|
|
|
|
break
|
|
|
|
}
|
|
|
|
require.NoError(tb, err)
|
|
|
|
deployments = append(deployments, d.Deployment)
|
|
|
|
}
|
|
|
|
return deployments
|
|
|
|
}
|
|
|
|
|
|
|
|
var recordedCases = []diffStackCase{
|
|
|
|
recordedStackCase("two-large-checkpoints.json"),
|
|
|
|
}
|
|
|
|
|
|
|
|
func init() {
|
|
|
|
for _, c := range strings.Split(os.Getenv("PULUMI_TEST_CHECKPOINT_DIFFS"), ",") {
|
|
|
|
if c != "" {
|
|
|
|
recordedCases = append(recordedCases, recordedStackCase(c))
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func BenchmarkDiffStackRecorded(b *testing.B) {
|
|
|
|
testOrBenchmarkDiffStack(b, benchmarkDiffStack, recordedCases)
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestDiffStackRecorded(t *testing.T) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
2023-05-04 23:28:49 +00:00
|
|
|
testOrBenchmarkDiffStack(t, testDiffStack, recordedCases)
|
|
|
|
}
|
[snapshot] Use a newer diff package
The version of gopls that gotextdiff is derived from is over three years
old. In the intervening time, the differ used by gopls has improved
tremendously, and the language has grown new features like generics.
These changes use a different diff package that is derived from an
up-to-date version of gopls and exposes an API that is parameterized
over the storage of the text. Taken together, this allows us to use a
much faster and more memory-efficient algorithm for diffing while
safely avoiding string copies when invoking the differ.
The more efficient algorithm operates on arbitrary slices whose elements
are comparable. Naive use of this algorithm--passing byte slices
in--can cause increased time spent diffing, as the sequences being
compared are longer. In order to avoid this, these changes record
spans of marshaled deployments as the deployments are marshaled. These
spans are strings that cover the diffable elements of the deployment:
its header, its resources, and its pending operations.
The output of the newer algorithm is converible to the format we
currently send to the service, so these changes should not require
service-side changes.
In order to avoid bumping the minimum Go version required to build the
`github.com/pulumi/pulumi/pkg/v3" and therefore break backwards
compatibility with consumers using older toolchains (e.g. providers),
these changes use the existing differ when built with a pre-1.20
toolchain. Official builds of the CLI are already using Go 1.20+ and
should use the new differ without additional changes.
These changes improve CPU time, allocation volume, and the compression
ratio by 57%, 87%, and 52%, respectively.
Benchmark results and analysis:
httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee spanned.txt
httpstate ❯ benchstat buffer-pool.txt spanned.txt >stat.txt
```
│ buffer-pool.txt │ spanned.txt │
│ sec/op │ sec/op vs base │
DiffStack/1_x_2_B-10 45.09µ ± 1% 22.81µ ± 1% -49.41% (p=0.000 n=10)
DiffStack/2_x_2_B-10 79.49µ ± 1% 40.60µ ± 1% -48.93% (p=0.000 n=10)
DiffStack/4_x_2_B-10 172.54µ ± 2% 81.57µ ± 0% -52.72% (p=0.000 n=10)
DiffStack/8_x_2_B-10 455.4µ ± 1% 200.4µ ± 0% -56.00% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1388.2µ ± 1% 575.1µ ± 1% -58.57% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4.409m ± 1% 1.866m ± 0% -57.67% (p=0.000 n=10)
DiffStack/48_x_2_B-10 9.275m ± 0% 3.484m ± 0% -62.44% (p=0.000 n=10)
DiffStack/64_x_2_B-10 14.564m ± 0% 5.945m ± 1% -59.18% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 124.81µ ± 1% 40.14µ ± 1% -67.84% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 299.8µ ± 0% 121.8µ ± 1% -59.38% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 846.2µ ± 1% 355.8µ ± 0% -57.96% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 2.743m ± 1% 1.150m ± 0% -58.06% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 9.271m ± 1% 4.041m ± 0% -56.41% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 31.78m ± 0% 14.97m ± 0% -52.88% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 66.14m ± 1% 30.66m ± 0% -53.63% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 107.50m ± 0% 53.44m ± 0% -50.29% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 240.51µ ± 1% 76.95µ ± 0% -68.01% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 798.5µ ± 0% 310.6µ ± 0% -61.10% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 2451.9µ ± 1% 978.9µ ± 3% -60.08% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 8.258m ± 0% 3.731m ± 0% -54.82% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 28.29m ± 1% 12.77m ± 0% -54.85% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 104.11m ± 1% 44.84m ± 1% -56.93% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 199.73m ± 1% 95.87m ± 1% -52.00% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 339.6m ± 2% 159.2m ± 2% -53.12% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 2721.7µ ± 0% 984.3µ ± 0% -63.84% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 8.334m ± 1% 3.654m ± 2% -56.15% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 25.89m ± 0% 12.23m ± 0% -52.77% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 87.49m ± 1% 40.26m ± 0% -53.98% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 317.0m ± 5% 150.4m ± 0% -52.57% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 681.8m ± 1% 332.1m ± 0% -51.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 1190.4m ± 1% 583.1m ± 0% -51.01% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 3.215m ± 0% 1.086m ± 0% -66.21% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 9.676m ± 1% 3.994m ± 0% -58.72% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 29.84m ± 0% 12.88m ± 1% -56.84% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 92.11m ± 1% 42.35m ± 0% -54.02% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 322.1m ± 0% 154.0m ± 0% -52.19% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 54.73m ± 1% 22.63m ± 0% -58.65% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 815.1m ± 0% 395.5m ± 2% -51.48% (p=0.000 n=10)
geomean 10.45m 4.530m -56.65%
│ buffer-pool.txt │ spanned.txt │
│ checkpoint_bytes │ checkpoint_bytes vs base │
DiffStack/1_x_2_B-10 2.539k ± 0% 1.950k ± 0% -23.20% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.201k ± 0% 4.548k ± 0% -12.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 13.63k ± 0% 12.30k ± 0% -9.77% (p=0.000 n=10)
DiffStack/8_x_2_B-10 45.09k ± 0% 42.86k ± 0% -4.95% (p=0.000 n=10)
DiffStack/16_x_2_B-10 161.9k ± 0% 156.4k ± 0% -3.42% (p=0.000 n=10)
DiffStack/32_x_2_B-10 579.8k ± 0% 604.2k ± 0% +4.21% (p=0.000 n=10)
DiffStack/48_x_2_B-10 1.306M ± 0% 1.180M ± 0% -9.68% (p=0.000 n=10)
DiffStack/64_x_2_B-10 2.164M ± 0% 2.134M ± 0% -1.36% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 27.109k ± 0% 9.856k ± 0% -63.64% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 78.91k ± 0% 61.59k ± 0% -21.94% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 284.2k ± 0% 258.3k ± 0% -9.13% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 1.119M ± 0% 1.041M ± 0% -6.89% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 4.389M ± 0% 4.201M ± 0% -4.29% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 16.74M ± 0% 17.06M ± 0% +1.95% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 36.30M ± 0% 35.93M ± 0% -1.00% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 60.98M ± 0% 64.06M ± 0% +5.05% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 67.79k ± 0% 34.43k ± 0% -49.20% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 300.1k ± 0% 233.6k ± 0% -22.15% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1128.3k ± 0% 929.5k ± 0% -17.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 4.437M ± 0% 4.238M ± 0% -4.49% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 16.57M ± 0% 15.78M ± 0% -4.79% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 64.35M ± 0% 58.66M ± 0% -8.84% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 127.0M ± 0% 128.0M ± 0% +0.80% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 217.7M ± 0% 216.5M ± 0% -0.54% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1316.2k ± 0% 921.8k ± 0% -29.97% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 4.733M ± 0% 4.207M ± 0% -11.11% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 16.03M ± 0% 15.77M ± 0% -1.64% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 57.42M ± 0% 54.66M ± 0% -4.81% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 214.8M ± 0% 210.7M ± 0% -1.90% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 472.9M ± 0% 466.9M ± 0% -1.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 832.3M ± 0% 824.7M ± 0% -0.92% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.575M ± 0% 1.051M ± 0% -33.32% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 5.248M ± 0% 4.199M ± 0% -20.00% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 18.36M ± 0% 15.74M ± 0% -14.28% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 60.86M ± 0% 56.66M ± 0% -6.90% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 223.5M ± 0% 214.0M ± 0% -4.23% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 13.21M ± 0% 13.20M ± 0% -0.01% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 155.9M ± 0% 154.9M ± 0% -0.60% (p=0.000 n=10)
geomean 3.606M 3.173M -12.01%
│ buffer-pool.txt │ spanned.txt │
│ ratio │ ratio vs base │
DiffStack/1_x_2_B-10 789.5m ± 0% 1641.0m ± 0% +107.85% (p=0.000 n=10)
DiffStack/2_x_2_B-10 1.034 ± 0% 2.353 ± 0% +127.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1.575 ± 0% 3.592 ± 0% +128.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 2.829 ± 0% 6.684 ± 0% +136.27% (p=0.000 n=10)
DiffStack/16_x_2_B-10 5.296 ± 0% 12.610 ± 0% +138.10% (p=0.000 n=10)
DiffStack/32_x_2_B-10 9.681 ± 0% 24.790 ± 0% +156.07% (p=0.000 n=10)
DiffStack/48_x_2_B-10 14.65 ± 0% 32.44 ± 0% +121.43% (p=0.000 n=10)
DiffStack/64_x_2_B-10 18.26 ± 0% 44.11 ± 0% +141.57% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 1.383 ± 0% 1.051 ± 0% -24.01% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 1.716 ± 0% 2.324 ± 0% +35.43% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 2.878 ± 0% 4.251 ± 0% +47.71% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 5.475 ± 0% 8.056 ± 0% +47.14% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.56 ± 0% 15.77 ± 0% +49.34% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 19.97 ± 0% 31.57 ± 0% +58.09% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 28.80 ± 0% 44.11 ± 0% +53.16% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 36.23 ± 0% 58.84 ± 0% +62.41% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 986.0m ± 0% 1014.0m ± 0% +2.84% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 1.777 ± 0% 2.331 ± 0% +31.18% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 3.057 ± 0% 3.993 ± 0% +30.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 5.766 ± 0% 8.511 ± 0% +47.61% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.55 ± 0% 15.35 ± 0% +45.50% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 20.28 ± 0% 28.08 ± 0% +38.46% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 26.59 ± 0% 40.64 ± 0% +52.84% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 34.14 ± 0% 51.44 ± 0% +50.67% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1.993 ± 0% 2.333 ± 0% +17.06% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 3.263 ± 0% 4.569 ± 0% +40.02% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 5.291 ± 0% 7.995 ± 0% +51.11% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.275 ± 0% 13.410 ± 0% +44.58% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 17.17 ± 0% 25.44 ± 0% +48.17% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 25.11 ± 0% 37.39 ± 0% +48.90% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 33.09 ± 0% 49.40 ± 0% +49.29% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.498 ± 0% 1.999 ± 0% +33.44% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 1.998 ± 0% 2.666 ± 0% +33.43% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 3.180 ± 0% 4.285 ± 0% +34.75% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 5.040 ± 0% 7.199 ± 0% +42.84% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.058 ± 0% 13.160 ± 0% +45.29% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% 1.999 ± 0% +0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 36.60 ± 0% 40.42 ± 0% +10.44% (p=0.000 n=10)
geomean 5.993 9.122 +52.20%
│ buffer-pool.txt │ spanned.txt │
│ wire_bytes │ wire_bytes vs base │
DiffStack/1_x_2_B-10 3.216k ± 0% 1.188k ± 0% -63.06% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.031k ± 0% 1.933k ± 0% -61.58% (p=0.000 n=10)
DiffStack/4_x_2_B-10 8.655k ± 0% 3.424k ± 0% -60.44% (p=0.000 n=10)
DiffStack/8_x_2_B-10 15.938k ± 0% 6.412k ± 0% -59.77% (p=0.000 n=10)
DiffStack/16_x_2_B-10 30.58k ± 0% 12.40k ± 0% -59.45% (p=0.000 n=10)
DiffStack/32_x_2_B-10 59.89k ± 0% 24.37k ± 0% -59.31% (p=0.000 n=10)
DiffStack/48_x_2_B-10 89.17k ± 0% 36.38k ± 0% -59.21% (p=0.000 n=10)
DiffStack/64_x_2_B-10 118.48k ± 0% 48.39k ± 0% -59.16% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 19.596k ± 0% 9.378k ± 0% -52.14% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 45.98k ± 0% 26.50k ± 0% -42.36% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 98.75k ± 0% 60.76k ± 0% -38.47% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 204.3k ± 0% 129.3k ± 0% -36.72% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 415.5k ± 0% 266.3k ± 0% -35.91% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 837.9k ± 0% 540.4k ± 0% -35.50% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 1260.4k ± 0% 814.6k ± 0% -35.37% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 1.683M ± 0% 1.089M ± 0% -35.31% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 68.75k ± 0% 33.95k ± 0% -50.61% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 168.9k ± 0% 100.2k ± 0% -40.64% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 369.1k ± 0% 232.8k ± 0% -36.93% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 769.6k ± 0% 497.9k ± 0% -35.30% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 1.571M ± 0% 1.028M ± 0% -34.53% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 3.173M ± 0% 2.089M ± 0% -34.16% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 4.775M ± 0% 3.149M ± 0% -34.04% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 6.377M ± 0% 4.210M ± 0% -33.98% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 660.4k ± 0% 395.1k ± 0% -40.16% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1450.4k ± 0% 920.9k ± 0% -36.51% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.031M ± 0% 1.972M ± 0% -34.91% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 6.191M ± 0% 4.076M ± 0% -34.17% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 12.512M ± 0% 8.282M ± 0% -33.81% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 18.83M ± 0% 12.49M ± 0% -33.69% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 25.15M ± 0% 16.69M ± 0% -33.63% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1051.8k ± 0% 525.5k ± 0% -50.04% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 2.626M ± 0% 1.575M ± 0% -40.04% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 5.776M ± 0% 3.673M ± 0% -36.40% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 12.075M ± 0% 7.871M ± 0% -34.82% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 24.67M ± 0% 16.27M ± 0% -34.07% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 6.612M ± 0% 6.606M ± 0% -0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 4.259M ± 0% 3.833M ± 0% -10.00% (p=0.000 n=10)
geomean 601.7k 347.8k -42.19%
│ buffer-pool.txt │ spanned.txt │
│ B/op │ B/op vs base │
DiffStack/1_x_2_B-10 31.85Ki ± 0% 15.79Ki ± 0% -50.42% (p=0.000 n=10)
DiffStack/2_x_2_B-10 61.81Ki ± 0% 28.01Ki ± 0% -54.68% (p=0.000 n=10)
DiffStack/4_x_2_B-10 147.58Ki ± 0% 55.60Ki ± 0% -62.33% (p=0.000 n=10)
DiffStack/8_x_2_B-10 425.9Ki ± 0% 116.9Ki ± 0% -72.55% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1429.9Ki ± 0% 307.8Ki ± 0% -78.47% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4955.7Ki ± 0% 934.1Ki ± 0% -81.15% (p=0.000 n=10)
DiffStack/48_x_2_B-10 10.746Mi ± 0% 1.682Mi ± 0% -84.34% (p=0.000 n=10)
DiffStack/64_x_2_B-10 17.664Mi ± 0% 2.822Mi ± 0% -84.03% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 202.50Ki ± 0% 41.18Ki ± 0% -79.66% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 511.5Ki ± 0% 134.1Ki ± 0% -73.78% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1513.5Ki ± 0% 309.4Ki ± 0% -79.56% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 4886.0Ki ± 0% 674.2Ki ± 0% -86.20% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 16.287Mi ± 0% 1.427Mi ± 0% -91.24% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 57.541Mi ± 0% 3.198Mi ± 0% -94.44% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 121.703Mi ± 0% 5.810Mi ± 0% -95.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 200.588Mi ± 0% 7.495Mi ± 0% -96.26% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 562.5Ki ± 0% 136.3Ki ± 0% -75.78% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 2041.0Ki ± 0% 505.5Ki ± 0% -75.23% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 5.874Mi ± 0% 1.129Mi ± 0% -80.78% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 18.837Mi ± 0% 2.410Mi ± 0% -87.21% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 61.898Mi ± 0% 5.015Mi ± 0% -91.90% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 221.69Mi ± 0% 10.36Mi ± 0% -95.32% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 430.13Mi ± 0% 18.50Mi ± 0% -95.70% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 722.07Mi ± 0% 21.63Mi ± 0% -97.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 8.098Mi ± 0% 1.656Mi ± 0% -79.55% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 22.662Mi ± 0% 3.934Mi ± 0% -82.64% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 65.217Mi ± 0% 8.425Mi ± 0% -87.08% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 204.90Mi ± 0% 17.22Mi ± 1% -91.60% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 703.12Mi ± 0% 34.89Mi ± 0% -95.04% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 1505.36Mi ± 0% 61.24Mi ± 0% -95.93% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 2586.04Mi ± 0% 70.48Mi ± 1% -97.27% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 11.854Mi ± 1% 2.127Mi ± 0% -82.06% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 32.932Mi ± 1% 6.558Mi ± 1% -80.09% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 90.04Mi ± 0% 15.15Mi ± 0% -83.17% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 252.94Mi ± 0% 32.27Mi ± 1% -87.24% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 801.77Mi ± 0% 66.46Mi ± 1% -91.71% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 103.67Mi ± 0% 35.52Mi ± 0% -65.74% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 913.90Mi ± 0% 53.53Mi ± 1% -94.14% (p=0.000 n=10)
geomean 19.16Mi 2.409Mi -87.43%
│ buffer-pool.txt │ spanned.txt │
│ allocs/op │ allocs/op vs base │
DiffStack/1_x_2_B-10 344.0 ± 0% 184.0 ± 0% -46.51% (p=0.000 n=10)
DiffStack/2_x_2_B-10 616.0 ± 0% 295.0 ± 0% -52.11% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1381.0 ± 0% 524.0 ± 0% -62.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 3.922k ± 0% 1.032k ± 0% -73.69% (p=0.000 n=10)
DiffStack/16_x_2_B-10 12.791k ± 0% 2.233k ± 0% -82.54% (p=0.000 n=10)
DiffStack/32_x_2_B-10 43.416k ± 0% 5.410k ± 0% -87.54% (p=0.000 n=10)
DiffStack/48_x_2_B-10 95.803k ± 0% 9.036k ± 0% -90.57% (p=0.000 n=10)
DiffStack/64_x_2_B-10 156.99k ± 0% 13.99k ± 0% -91.09% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 335.0 ± 0% 182.0 ± 0% -45.67% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 597.5 ± 0% 295.0 ± 0% -50.63% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1304.0 ± 0% 527.0 ± 0% -59.59% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 3.475k ± 0% 1.036k ± 0% -70.19% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.677k ± 0% 2.239k ± 0% -79.03% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 35.357k ± 0% 5.414k ± 0% -84.69% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 72.782k ± 0% 9.296k ± 0% -87.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 119.24k ± 0% 14.26k ± 0% -88.04% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 309.0 ± 0% 182.0 ± 0% -41.10% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 594.0 ± 0% 296.0 ± 0% -50.17% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1324.5 ± 0% 528.0 ± 0% -60.14% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 3.554k ± 0% 1.047k ± 0% -70.54% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.511k ± 0% 2.232k ± 0% -78.77% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 35.147k ± 0% 5.194k ± 0% -85.22% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 66.519k ± 0% 8.958k ± 0% -86.53% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 110.71k ± 0% 13.28k ± 0% -88.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 644.0 ± 0% 296.0 ± 0% -54.04% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1418.5 ± 0% 535.0 ± 0% -62.28% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.497k ± 0% 1.047k ± 0% -70.06% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.805k ± 0% 2.182k ± 0% -77.75% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 30.996k ± 0% 5.040k ± 0% -83.74% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 63.751k ± 0% 8.656k ± 0% -86.42% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 107.97k ± 0% 13.03k ± 0% -87.93% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 356.0 ± 0% 186.0 ± 0% -47.75% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 671.0 ± 0% 300.0 ± 0% -55.29% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 1467.5 ± 0% 536.0 ± 0% -63.48% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 3.553k ± 0% 1.038k ± 0% -70.80% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.971k ± 0% 2.182k ± 0% -78.12% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 489.16k ± 0% 73.65k ± 0% -84.94% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 7236.9k ± 0% 857.8k ± 0% -88.15% (p=0.000 n=10)
geomean 7.989k 1.934k -75.79%
```
2023-05-05 02:15:09 +00:00
|
|
|
|
|
|
|
func TestMarshalDeployment(t *testing.T) {
|
2024-05-03 11:49:14 +00:00
|
|
|
t.Parallel()
|
|
|
|
|
[snapshot] Use a newer diff package
The version of gopls that gotextdiff is derived from is over three years
old. In the intervening time, the differ used by gopls has improved
tremendously, and the language has grown new features like generics.
These changes use a different diff package that is derived from an
up-to-date version of gopls and exposes an API that is parameterized
over the storage of the text. Taken together, this allows us to use a
much faster and more memory-efficient algorithm for diffing while
safely avoiding string copies when invoking the differ.
The more efficient algorithm operates on arbitrary slices whose elements
are comparable. Naive use of this algorithm--passing byte slices
in--can cause increased time spent diffing, as the sequences being
compared are longer. In order to avoid this, these changes record
spans of marshaled deployments as the deployments are marshaled. These
spans are strings that cover the diffable elements of the deployment:
its header, its resources, and its pending operations.
The output of the newer algorithm is converible to the format we
currently send to the service, so these changes should not require
service-side changes.
In order to avoid bumping the minimum Go version required to build the
`github.com/pulumi/pulumi/pkg/v3" and therefore break backwards
compatibility with consumers using older toolchains (e.g. providers),
these changes use the existing differ when built with a pre-1.20
toolchain. Official builds of the CLI are already using Go 1.20+ and
should use the new differ without additional changes.
These changes improve CPU time, allocation volume, and the compression
ratio by 57%, 87%, and 52%, respectively.
Benchmark results and analysis:
httpstate ❯ go test -count=10 -run none -benchmem -bench . | tee spanned.txt
httpstate ❯ benchstat buffer-pool.txt spanned.txt >stat.txt
```
│ buffer-pool.txt │ spanned.txt │
│ sec/op │ sec/op vs base │
DiffStack/1_x_2_B-10 45.09µ ± 1% 22.81µ ± 1% -49.41% (p=0.000 n=10)
DiffStack/2_x_2_B-10 79.49µ ± 1% 40.60µ ± 1% -48.93% (p=0.000 n=10)
DiffStack/4_x_2_B-10 172.54µ ± 2% 81.57µ ± 0% -52.72% (p=0.000 n=10)
DiffStack/8_x_2_B-10 455.4µ ± 1% 200.4µ ± 0% -56.00% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1388.2µ ± 1% 575.1µ ± 1% -58.57% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4.409m ± 1% 1.866m ± 0% -57.67% (p=0.000 n=10)
DiffStack/48_x_2_B-10 9.275m ± 0% 3.484m ± 0% -62.44% (p=0.000 n=10)
DiffStack/64_x_2_B-10 14.564m ± 0% 5.945m ± 1% -59.18% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 124.81µ ± 1% 40.14µ ± 1% -67.84% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 299.8µ ± 0% 121.8µ ± 1% -59.38% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 846.2µ ± 1% 355.8µ ± 0% -57.96% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 2.743m ± 1% 1.150m ± 0% -58.06% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 9.271m ± 1% 4.041m ± 0% -56.41% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 31.78m ± 0% 14.97m ± 0% -52.88% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 66.14m ± 1% 30.66m ± 0% -53.63% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 107.50m ± 0% 53.44m ± 0% -50.29% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 240.51µ ± 1% 76.95µ ± 0% -68.01% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 798.5µ ± 0% 310.6µ ± 0% -61.10% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 2451.9µ ± 1% 978.9µ ± 3% -60.08% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 8.258m ± 0% 3.731m ± 0% -54.82% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 28.29m ± 1% 12.77m ± 0% -54.85% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 104.11m ± 1% 44.84m ± 1% -56.93% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 199.73m ± 1% 95.87m ± 1% -52.00% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 339.6m ± 2% 159.2m ± 2% -53.12% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 2721.7µ ± 0% 984.3µ ± 0% -63.84% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 8.334m ± 1% 3.654m ± 2% -56.15% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 25.89m ± 0% 12.23m ± 0% -52.77% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 87.49m ± 1% 40.26m ± 0% -53.98% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 317.0m ± 5% 150.4m ± 0% -52.57% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 681.8m ± 1% 332.1m ± 0% -51.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 1190.4m ± 1% 583.1m ± 0% -51.01% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 3.215m ± 0% 1.086m ± 0% -66.21% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 9.676m ± 1% 3.994m ± 0% -58.72% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 29.84m ± 0% 12.88m ± 1% -56.84% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 92.11m ± 1% 42.35m ± 0% -54.02% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 322.1m ± 0% 154.0m ± 0% -52.19% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 54.73m ± 1% 22.63m ± 0% -58.65% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 815.1m ± 0% 395.5m ± 2% -51.48% (p=0.000 n=10)
geomean 10.45m 4.530m -56.65%
│ buffer-pool.txt │ spanned.txt │
│ checkpoint_bytes │ checkpoint_bytes vs base │
DiffStack/1_x_2_B-10 2.539k ± 0% 1.950k ± 0% -23.20% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.201k ± 0% 4.548k ± 0% -12.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 13.63k ± 0% 12.30k ± 0% -9.77% (p=0.000 n=10)
DiffStack/8_x_2_B-10 45.09k ± 0% 42.86k ± 0% -4.95% (p=0.000 n=10)
DiffStack/16_x_2_B-10 161.9k ± 0% 156.4k ± 0% -3.42% (p=0.000 n=10)
DiffStack/32_x_2_B-10 579.8k ± 0% 604.2k ± 0% +4.21% (p=0.000 n=10)
DiffStack/48_x_2_B-10 1.306M ± 0% 1.180M ± 0% -9.68% (p=0.000 n=10)
DiffStack/64_x_2_B-10 2.164M ± 0% 2.134M ± 0% -1.36% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 27.109k ± 0% 9.856k ± 0% -63.64% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 78.91k ± 0% 61.59k ± 0% -21.94% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 284.2k ± 0% 258.3k ± 0% -9.13% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 1.119M ± 0% 1.041M ± 0% -6.89% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 4.389M ± 0% 4.201M ± 0% -4.29% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 16.74M ± 0% 17.06M ± 0% +1.95% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 36.30M ± 0% 35.93M ± 0% -1.00% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 60.98M ± 0% 64.06M ± 0% +5.05% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 67.79k ± 0% 34.43k ± 0% -49.20% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 300.1k ± 0% 233.6k ± 0% -22.15% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1128.3k ± 0% 929.5k ± 0% -17.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 4.437M ± 0% 4.238M ± 0% -4.49% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 16.57M ± 0% 15.78M ± 0% -4.79% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 64.35M ± 0% 58.66M ± 0% -8.84% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 127.0M ± 0% 128.0M ± 0% +0.80% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 217.7M ± 0% 216.5M ± 0% -0.54% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1316.2k ± 0% 921.8k ± 0% -29.97% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 4.733M ± 0% 4.207M ± 0% -11.11% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 16.03M ± 0% 15.77M ± 0% -1.64% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 57.42M ± 0% 54.66M ± 0% -4.81% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 214.8M ± 0% 210.7M ± 0% -1.90% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 472.9M ± 0% 466.9M ± 0% -1.28% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 832.3M ± 0% 824.7M ± 0% -0.92% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.575M ± 0% 1.051M ± 0% -33.32% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 5.248M ± 0% 4.199M ± 0% -20.00% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 18.36M ± 0% 15.74M ± 0% -14.28% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 60.86M ± 0% 56.66M ± 0% -6.90% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 223.5M ± 0% 214.0M ± 0% -4.23% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 13.21M ± 0% 13.20M ± 0% -0.01% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 155.9M ± 0% 154.9M ± 0% -0.60% (p=0.000 n=10)
geomean 3.606M 3.173M -12.01%
│ buffer-pool.txt │ spanned.txt │
│ ratio │ ratio vs base │
DiffStack/1_x_2_B-10 789.5m ± 0% 1641.0m ± 0% +107.85% (p=0.000 n=10)
DiffStack/2_x_2_B-10 1.034 ± 0% 2.353 ± 0% +127.56% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1.575 ± 0% 3.592 ± 0% +128.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 2.829 ± 0% 6.684 ± 0% +136.27% (p=0.000 n=10)
DiffStack/16_x_2_B-10 5.296 ± 0% 12.610 ± 0% +138.10% (p=0.000 n=10)
DiffStack/32_x_2_B-10 9.681 ± 0% 24.790 ± 0% +156.07% (p=0.000 n=10)
DiffStack/48_x_2_B-10 14.65 ± 0% 32.44 ± 0% +121.43% (p=0.000 n=10)
DiffStack/64_x_2_B-10 18.26 ± 0% 44.11 ± 0% +141.57% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 1.383 ± 0% 1.051 ± 0% -24.01% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 1.716 ± 0% 2.324 ± 0% +35.43% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 2.878 ± 0% 4.251 ± 0% +47.71% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 5.475 ± 0% 8.056 ± 0% +47.14% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.56 ± 0% 15.77 ± 0% +49.34% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 19.97 ± 0% 31.57 ± 0% +58.09% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 28.80 ± 0% 44.11 ± 0% +53.16% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 36.23 ± 0% 58.84 ± 0% +62.41% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 986.0m ± 0% 1014.0m ± 0% +2.84% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 1.777 ± 0% 2.331 ± 0% +31.18% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 3.057 ± 0% 3.993 ± 0% +30.62% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 5.766 ± 0% 8.511 ± 0% +47.61% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.55 ± 0% 15.35 ± 0% +45.50% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 20.28 ± 0% 28.08 ± 0% +38.46% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 26.59 ± 0% 40.64 ± 0% +52.84% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 34.14 ± 0% 51.44 ± 0% +50.67% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 1.993 ± 0% 2.333 ± 0% +17.06% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 3.263 ± 0% 4.569 ± 0% +40.02% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 5.291 ± 0% 7.995 ± 0% +51.11% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.275 ± 0% 13.410 ± 0% +44.58% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 17.17 ± 0% 25.44 ± 0% +48.17% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 25.11 ± 0% 37.39 ± 0% +48.90% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 33.09 ± 0% 49.40 ± 0% +49.29% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1.498 ± 0% 1.999 ± 0% +33.44% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 1.998 ± 0% 2.666 ± 0% +33.43% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 3.180 ± 0% 4.285 ± 0% +34.75% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 5.040 ± 0% 7.199 ± 0% +42.84% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.058 ± 0% 13.160 ± 0% +45.29% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 1.997 ± 0% 1.999 ± 0% +0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 36.60 ± 0% 40.42 ± 0% +10.44% (p=0.000 n=10)
geomean 5.993 9.122 +52.20%
│ buffer-pool.txt │ spanned.txt │
│ wire_bytes │ wire_bytes vs base │
DiffStack/1_x_2_B-10 3.216k ± 0% 1.188k ± 0% -63.06% (p=0.000 n=10)
DiffStack/2_x_2_B-10 5.031k ± 0% 1.933k ± 0% -61.58% (p=0.000 n=10)
DiffStack/4_x_2_B-10 8.655k ± 0% 3.424k ± 0% -60.44% (p=0.000 n=10)
DiffStack/8_x_2_B-10 15.938k ± 0% 6.412k ± 0% -59.77% (p=0.000 n=10)
DiffStack/16_x_2_B-10 30.58k ± 0% 12.40k ± 0% -59.45% (p=0.000 n=10)
DiffStack/32_x_2_B-10 59.89k ± 0% 24.37k ± 0% -59.31% (p=0.000 n=10)
DiffStack/48_x_2_B-10 89.17k ± 0% 36.38k ± 0% -59.21% (p=0.000 n=10)
DiffStack/64_x_2_B-10 118.48k ± 0% 48.39k ± 0% -59.16% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 19.596k ± 0% 9.378k ± 0% -52.14% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 45.98k ± 0% 26.50k ± 0% -42.36% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 98.75k ± 0% 60.76k ± 0% -38.47% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 204.3k ± 0% 129.3k ± 0% -36.72% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 415.5k ± 0% 266.3k ± 0% -35.91% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 837.9k ± 0% 540.4k ± 0% -35.50% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 1260.4k ± 0% 814.6k ± 0% -35.37% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 1.683M ± 0% 1.089M ± 0% -35.31% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 68.75k ± 0% 33.95k ± 0% -50.61% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 168.9k ± 0% 100.2k ± 0% -40.64% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 369.1k ± 0% 232.8k ± 0% -36.93% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 769.6k ± 0% 497.9k ± 0% -35.30% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 1.571M ± 0% 1.028M ± 0% -34.53% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 3.173M ± 0% 2.089M ± 0% -34.16% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 4.775M ± 0% 3.149M ± 0% -34.04% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 6.377M ± 0% 4.210M ± 0% -33.98% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 660.4k ± 0% 395.1k ± 0% -40.16% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1450.4k ± 0% 920.9k ± 0% -36.51% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.031M ± 0% 1.972M ± 0% -34.91% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 6.191M ± 0% 4.076M ± 0% -34.17% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 12.512M ± 0% 8.282M ± 0% -33.81% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 18.83M ± 0% 12.49M ± 0% -33.69% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 25.15M ± 0% 16.69M ± 0% -33.63% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 1051.8k ± 0% 525.5k ± 0% -50.04% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 2.626M ± 0% 1.575M ± 0% -40.04% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 5.776M ± 0% 3.673M ± 0% -36.40% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 12.075M ± 0% 7.871M ± 0% -34.82% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 24.67M ± 0% 16.27M ± 0% -34.07% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 6.612M ± 0% 6.606M ± 0% -0.10% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 4.259M ± 0% 3.833M ± 0% -10.00% (p=0.000 n=10)
geomean 601.7k 347.8k -42.19%
│ buffer-pool.txt │ spanned.txt │
│ B/op │ B/op vs base │
DiffStack/1_x_2_B-10 31.85Ki ± 0% 15.79Ki ± 0% -50.42% (p=0.000 n=10)
DiffStack/2_x_2_B-10 61.81Ki ± 0% 28.01Ki ± 0% -54.68% (p=0.000 n=10)
DiffStack/4_x_2_B-10 147.58Ki ± 0% 55.60Ki ± 0% -62.33% (p=0.000 n=10)
DiffStack/8_x_2_B-10 425.9Ki ± 0% 116.9Ki ± 0% -72.55% (p=0.000 n=10)
DiffStack/16_x_2_B-10 1429.9Ki ± 0% 307.8Ki ± 0% -78.47% (p=0.000 n=10)
DiffStack/32_x_2_B-10 4955.7Ki ± 0% 934.1Ki ± 0% -81.15% (p=0.000 n=10)
DiffStack/48_x_2_B-10 10.746Mi ± 0% 1.682Mi ± 0% -84.34% (p=0.000 n=10)
DiffStack/64_x_2_B-10 17.664Mi ± 0% 2.822Mi ± 0% -84.03% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 202.50Ki ± 0% 41.18Ki ± 0% -79.66% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 511.5Ki ± 0% 134.1Ki ± 0% -73.78% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1513.5Ki ± 0% 309.4Ki ± 0% -79.56% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 4886.0Ki ± 0% 674.2Ki ± 0% -86.20% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 16.287Mi ± 0% 1.427Mi ± 0% -91.24% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 57.541Mi ± 0% 3.198Mi ± 0% -94.44% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 121.703Mi ± 0% 5.810Mi ± 0% -95.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 200.588Mi ± 0% 7.495Mi ± 0% -96.26% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 562.5Ki ± 0% 136.3Ki ± 0% -75.78% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 2041.0Ki ± 0% 505.5Ki ± 0% -75.23% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 5.874Mi ± 0% 1.129Mi ± 0% -80.78% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 18.837Mi ± 0% 2.410Mi ± 0% -87.21% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 61.898Mi ± 0% 5.015Mi ± 0% -91.90% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 221.69Mi ± 0% 10.36Mi ± 0% -95.32% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 430.13Mi ± 0% 18.50Mi ± 0% -95.70% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 722.07Mi ± 0% 21.63Mi ± 0% -97.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 8.098Mi ± 0% 1.656Mi ± 0% -79.55% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 22.662Mi ± 0% 3.934Mi ± 0% -82.64% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 65.217Mi ± 0% 8.425Mi ± 0% -87.08% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 204.90Mi ± 0% 17.22Mi ± 1% -91.60% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 703.12Mi ± 0% 34.89Mi ± 0% -95.04% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 1505.36Mi ± 0% 61.24Mi ± 0% -95.93% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 2586.04Mi ± 0% 70.48Mi ± 1% -97.27% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 11.854Mi ± 1% 2.127Mi ± 0% -82.06% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 32.932Mi ± 1% 6.558Mi ± 1% -80.09% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 90.04Mi ± 0% 15.15Mi ± 0% -83.17% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 252.94Mi ± 0% 32.27Mi ± 1% -87.24% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 801.77Mi ± 0% 66.46Mi ± 1% -91.71% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 103.67Mi ± 0% 35.52Mi ± 0% -65.74% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 913.90Mi ± 0% 53.53Mi ± 1% -94.14% (p=0.000 n=10)
geomean 19.16Mi 2.409Mi -87.43%
│ buffer-pool.txt │ spanned.txt │
│ allocs/op │ allocs/op vs base │
DiffStack/1_x_2_B-10 344.0 ± 0% 184.0 ± 0% -46.51% (p=0.000 n=10)
DiffStack/2_x_2_B-10 616.0 ± 0% 295.0 ± 0% -52.11% (p=0.000 n=10)
DiffStack/4_x_2_B-10 1381.0 ± 0% 524.0 ± 0% -62.06% (p=0.000 n=10)
DiffStack/8_x_2_B-10 3.922k ± 0% 1.032k ± 0% -73.69% (p=0.000 n=10)
DiffStack/16_x_2_B-10 12.791k ± 0% 2.233k ± 0% -82.54% (p=0.000 n=10)
DiffStack/32_x_2_B-10 43.416k ± 0% 5.410k ± 0% -87.54% (p=0.000 n=10)
DiffStack/48_x_2_B-10 95.803k ± 0% 9.036k ± 0% -90.57% (p=0.000 n=10)
DiffStack/64_x_2_B-10 156.99k ± 0% 13.99k ± 0% -91.09% (p=0.000 n=10)
DiffStack/1_x_8.2_kB-10 335.0 ± 0% 182.0 ± 0% -45.67% (p=0.000 n=10)
DiffStack/2_x_8.2_kB-10 597.5 ± 0% 295.0 ± 0% -50.63% (p=0.000 n=10)
DiffStack/4_x_8.2_kB-10 1304.0 ± 0% 527.0 ± 0% -59.59% (p=0.000 n=10)
DiffStack/8_x_8.2_kB-10 3.475k ± 0% 1.036k ± 0% -70.19% (p=0.000 n=10)
DiffStack/16_x_8.2_kB-10 10.677k ± 0% 2.239k ± 0% -79.03% (p=0.000 n=10)
DiffStack/32_x_8.2_kB-10 35.357k ± 0% 5.414k ± 0% -84.69% (p=0.000 n=10)
DiffStack/48_x_8.2_kB-10 72.782k ± 0% 9.296k ± 0% -87.23% (p=0.000 n=10)
DiffStack/64_x_8.2_kB-10 119.24k ± 0% 14.26k ± 0% -88.04% (p=0.000 n=10)
DiffStack/1_x_33_kB-10 309.0 ± 0% 182.0 ± 0% -41.10% (p=0.000 n=10)
DiffStack/2_x_33_kB-10 594.0 ± 0% 296.0 ± 0% -50.17% (p=0.000 n=10)
DiffStack/4_x_33_kB-10 1324.5 ± 0% 528.0 ± 0% -60.14% (p=0.000 n=10)
DiffStack/8_x_33_kB-10 3.554k ± 0% 1.047k ± 0% -70.54% (p=0.000 n=10)
DiffStack/16_x_33_kB-10 10.511k ± 0% 2.232k ± 0% -78.77% (p=0.000 n=10)
DiffStack/32_x_33_kB-10 35.147k ± 0% 5.194k ± 0% -85.22% (p=0.000 n=10)
DiffStack/48_x_33_kB-10 66.519k ± 0% 8.958k ± 0% -86.53% (p=0.000 n=10)
DiffStack/64_x_33_kB-10 110.71k ± 0% 13.28k ± 0% -88.00% (p=0.000 n=10)
DiffStack/2_x_131_kB-10 644.0 ± 0% 296.0 ± 0% -54.04% (p=0.000 n=10)
DiffStack/4_x_131_kB-10 1418.5 ± 0% 535.0 ± 0% -62.28% (p=0.000 n=10)
DiffStack/8_x_131_kB-10 3.497k ± 0% 1.047k ± 0% -70.06% (p=0.000 n=10)
DiffStack/16_x_131_kB-10 9.805k ± 0% 2.182k ± 0% -77.75% (p=0.000 n=10)
DiffStack/32_x_131_kB-10 30.996k ± 0% 5.040k ± 0% -83.74% (p=0.000 n=10)
DiffStack/48_x_131_kB-10 63.751k ± 0% 8.656k ± 0% -86.42% (p=0.000 n=10)
DiffStack/64_x_131_kB-10 107.97k ± 0% 13.03k ± 0% -87.93% (p=0.000 n=10)
DiffStack/1_x_524_kB-10 356.0 ± 0% 186.0 ± 0% -47.75% (p=0.000 n=10)
DiffStack/2_x_524_kB-10 671.0 ± 0% 300.0 ± 0% -55.29% (p=0.000 n=10)
DiffStack/4_x_524_kB-10 1467.5 ± 0% 536.0 ± 0% -63.48% (p=0.000 n=10)
DiffStack/8_x_524_kB-10 3.553k ± 0% 1.038k ± 0% -70.80% (p=0.000 n=10)
DiffStack/16_x_524_kB-10 9.971k ± 0% 2.182k ± 0% -78.12% (p=0.000 n=10)
DiffStackRecorded/two-large-checkpoints.json-10 489.16k ± 0% 73.65k ± 0% -84.94% (p=0.000 n=10)
DiffStackRecorded/checkpoints.json-10 7236.9k ± 0% 857.8k ± 0% -88.15% (p=0.000 n=10)
geomean 7.989k 1.934k -75.79%
```
2023-05-05 02:15:09 +00:00
|
|
|
testOrBenchmarkDiffStack(t, testMarshalDeployment, dynamicCases)
|
|
|
|
testOrBenchmarkDiffStack(t, testMarshalDeployment, recordedCases)
|
|
|
|
}
|