Commit Graph

31 Commits

Author SHA1 Message Date
Fraser Waters d771acf707
Add tokens.StackName ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
 
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.

This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.

There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.

Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 07:44:54 +00:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
Fraser Waters 39aaf0604f Change secret.Manager State to json.RawMessage
The state from a secret manager _always_ has to be something json
serialisable because we store it into the json state files. Rather than
allowing any `interface{}` to be returned here and then error'ing if it
happens to not be something that can be marshalled to JSON this changes
the interface to return a `json.RawMessage` moving the
marshalling/unmarshalling concerns into the individual implementations
that can pretty much guarantee valid structures.
2023-05-22 11:21:15 +01:00
Abhinav Gupta e395deef6b
all: Assert => Assertf
Migrates all remaining usages of
`contract.Assert*` and `contract.Require*` to the f variants,
which require adding meaningful error messages.

There were a couple cases where a `testing.T` or `testing.B`
was already available.
For those, this uses t.FailNow or require.NoError.

Refs 
2023-03-03 14:37:43 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
bors[bot] e84522340c
Merge
12038: Move the project file handling for secret config to one place r=Frassle a=Frassle



<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

A continuing clean up of secrets.

This means the secret providers just work in terms of workspace.ProjectStack, mutate as they wish and let the higher level work out if it should save the file or not. Rather than having each secret provider have to deal with loading and saving the stack config files all that that logic is now moved out to `getStackSecretsManager` and `createSecretsManager`.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works - Should be covered by existing tests. No behavioral changes intended. 
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - Not user facing
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-02-07 21:17:31 +00:00
Fraser Waters 7d53b479ed Add `--insecure` flag to `pulumi login`
Add `--insecure` flag to `pulumi login` which disables https certificate checks
2023-02-03 16:09:37 -08:00
Fraser Waters a3f7a342b2 Move the project file handling for secret config to one place
This means the secret providers just work in terms of
workspace.ProjectStack, mutate as they wish and let the higher level
work out if it should save the file or not. Rather than having each
secret manager maintain "should I save the file" code.
2023-02-01 17:03:38 +00:00
Abhinav Gupta c97d9401f2
getDiffInfo: Fix slice length check
Checking if the slice is non-nil instead of checking length
causes breakage when we pre-allocate diffs based on `len(md.Diffs)`.
2023-01-11 21:53:05 -08:00
Robbie McKinstry 1f78baae71
Preallocate slices with a known capacity.
Enable the prealloc linter, which identifies slices
with a known capacity, but are not preallocated, which
results in unnecessary allocations and memcpys.
2023-01-11 12:52:51 -08:00
bors[bot] 2957b6b1b3
Merge
11806: all: Drop ioutil r=abhinav a=abhinav

Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.

Specifically, the template and the template input
had to be updated manually.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-01-09 17:15:34 +00:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Fraser Waters 54c10ba71d Pull NewServiceSecretsManager out of httpstate
A sister PR to https://github.com/pulumi/pulumi/pull/11767. This pulls
as much of NewServiceSecretsManager out of httpstate and to the
secrets/service module.

What's left in httpstate is the DefaultSecretManager method pulling off
the client and stack identifier to pass to NewServiceSecretsManager.
That couldn't be done in secrets/service because that would cause a
dependency chain loop (secrets/service would depend on the
httpstate.Stack, but that would depend on secrets/service).

Arguably the `client.Client` ought to just be a string URL and the
secret manager should make it's own client (like it does in
NewServiceSecretsManagerFromState). But trying to keep each individual
change here small.
2023-01-04 23:04:30 +00:00
Anton Tayanovskyy 96a3783fb2
Fix orphan bulk decrypt traces ()
* Propagate context through crypter to correctly parent service crypter tracing spans

* Remove one of the context.TODO that is needed

* Fix new_test.go

* Fix pulumi_test.go

* Fix resources_test.go

* Fix pkg/resource/stack tests

* Add CHANGELOG_PENDING

* Fix value_test.go

* Fix stack_test.go

* Feedback
2022-07-18 09:36:31 -04:00
Fraser Waters 5528cde977
Change BulkDecrypt to not rely on type tests ()
* Change BulkDecrypt to not rely on type tests

Fixes https://github.com/pulumi/pulumi/issues/9350

* Add to CHANGELOG
2022-04-11 08:59:46 +01:00
Fraser Waters 170862ccc6
Ensure that FormatOptions.Color is set before use in diag.defaultSink ()
* Ensure that FormatOptions.Color is set before use in diag.Sink

* lint
2022-04-01 20:22:35 +01:00
Pat Gavlin 87b3f44590
[secrets] Fix + refactor bulk decryption. ()
The bulk decryption logic in deployment deserialization did not recur
into slices or maps. This prevented the bulk decryption of nested secret
values. These changes fix that bug, improve test coverage, and refactor
the bulk decryption code for simplicity, clarity, and separation of
concerns.

Related to https://github.com/pulumi/home/issues/1842.
2022-02-25 08:51:52 -08:00
Paul Stack e3720b3a93
Using a decryptAll functionality when deserializing a deployment () 2022-01-24 22:33:40 +02:00
Ian Wahbe 272c4643b2
Update error handling ()
This is the result of a change applied via `go-rewrap-errors`.
2021-11-12 18:37:17 -08:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Levi Blackstone 709fcbad51
Document Go packages ()
Co-authored-by: Pat Gavlin <pat@pulumi.com>
2021-01-11 11:07:59 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle d3f5bbce48 go fmt 2020-03-18 17:27:02 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle f754b486b8 move pkg/resource/config -> sdk/go/common/resource/config 2020-03-18 15:03:37 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle 8fb3f428b0 move pkg/workspace -> sdk/go/common/workspace 2020-03-18 14:35:53 -07:00
Pat Gavlin 460ff875ab
Cache account information. ()
- Cache the username and last verified time associated with each logged-in
  backend
- In the HTTP backend, verify the access token explicitly at most once
  per hour

This trades off a little bit of usability for improved inner-loop
latency: if a user's API token becomes invalid less than an hour after
it was last verified, the user will see 4xx errors when attempting stack
operations rather than seeing the login prompt.
2019-10-15 15:37:57 -07:00
Chris Smith 17ee050abe
Refactor the way secrets managers are provided () 2019-08-01 10:33:52 -07:00
Matt Ellis e5d3a20399 Use "passphrase" and "service" instead of "local" and "cloud" 2019-05-10 17:07:52 -07:00
Matt Ellis 88012c4d96 Enable "cloud" and "local" secrets managers across the system
We move the implementations of our secrets managers in to
`pkg/secrets` (which is where the base64 one lives) and wire their use
up during deserialization.

It's a little unfortunate that for the passphrase based secrets
manager, we have to require `PULUMI_CONFIG_PASSPHRASE` when
constructing it from state, but we can make more progress with the
changes as they are now, and I think we can come up with some ways to
mitigate this problem a bit (at least make it only a problem for cases
where you are trying to take a stack reference to another stack that
is managed with local encryption).
2019-05-10 17:07:52 -07:00