Commit Graph

177 Commits

Author SHA1 Message Date
Julien 7e2ccd8757
Prepare golangci-lint upgrade ()
The latest version of golangci-lint (1.60.3) flags a bunch of new issues
in our code base. This PR addresses part of them ahead of the upgrade.

* A dynamic string passed to printf style functions as first argument,
this can lead to bad `%` interpolations. The fix is typically to use
`"%s"` as first argument and pass the dynamic string as 2nd argument.
* Using `os.ModePerm` in tests instead of more restricted file
permissions. The fix is to use 0o600 for files, or 0o700 for
directories.
* Int conversion overflows. The fix has to be done case by case,
checking that no overflow can occur.
2024-08-28 07:57:38 +00:00
Will Jones 1be4888f7d
Replace `result.Result` with native errors ()
`result.Result` is a type that was introduced to enable us to
distinguish between *expected* errors (or *bails* in Pulumi parlance)
and *unexpected* errors or exceptions. Prior to Go 1.13, this made a lot
of sense, since there was no standard story on "wrapping" errors, and
thus no way to answer the question "did I end up here (in an error path)
because I meant to, or did I end up here because of a program bug?".

With the introduction of `%w`, `interface { Unwrap() []error }` and
company in Go 1.13, a simpler interface is available: one can use
`errors.Is` and `errors.As` to ask if an error at any point wraps an
error of a certain type. With this, we can simply ask "is there a bail
at any point in this error tree?" rather than having to track this
explicitly using a type such as `result.Result`. The `IsBail` function
was introduced a while ago to this end, but its rollout was not
completed and several uses of `result.Result` remained.

This commit completes the rollout of this simplified interface,
replacing all uses of `result.Result` with native Go errors that may or
may not wrap bails, and all uses of e.g. `res.IsBail` with
`IsBail(err)`. Doing so allows us to remove `result.Result` entirely.
2024-08-22 14:39:59 +00:00
Will Jones 3f0849f7d2
Clean up multi-error rendering ()
When an error occurs in Go, there is often a need to "wrap" the error
and propagate it upwards. For instance, if a "file not found" error
occurs while trying to read a `Pulumi.yaml`, we want to surface both
those pieces of information to the consumer of the error. Since 1.13, Go
provides support for natively wrapping errors using e.g. the `%w` format
specified in `fmt.Errorf` and the `Unwrap() []error` interface method.
Prior to 1.13. libraries such as `multierror` provided a similar
function in the absence of standard-library support.

Due to its age, the `pulumi/pulumi` codebase uses both Go-native error
wrapping and `multierror`. It would likely be good to converge on the
former and lose a dependency/remove extraneous code paths. One key
difference between the two approaches is in how multiple errors are
combined into a new parent. In `multierror`, the `Append` function takes
a set of errors and combines them, _flattening any existing trees of
errors_ so that the result is always a flat list of errors. In contrast,
Go's built-in `errors.Join` does not flatten and so can produce trees of
errors. Practically speaking, this is unlikely to be a huge problem for
this codebase, but there are some changes we can make proactively to
reduce the risks of trees-vs-lists if and when we migrate. Specifically,
when rendering errors, we don't consider the possibility of there being
a tree at present. This means that an input such as:

```go
errors.Join(
  errors.Join(
    errors.New("foo"),
    errors.New("bar"),
  ),
  errors.New("baz"),
)
```

results in something like:

```
  1) 2 errors occurred:
  1) foo
  2) bar
  2) baz
```

This commit fixes this so that errors are flattened at the point of
rendering, meaning that it shouldn't matter as we move from
`multierror.Append` to `errors.Join` whether or not an error chain is a
tree or a list.
2024-08-22 10:32:45 +00:00
Julien 487b4a8494
Install missing python versions using pyenv during installation ()
Adds the flag `--use-language-version-tools` to `pulumi install`. When
passed, and `pyenv` is installed, and a `.python-version` file is in the
project directory or any of its parent directories, Pulumi will install
the requested python version if it is not already installed.

`LanguageRuntime.InstallDependencies` now takes a struct
`InstallDependenciesRequest` as argument to make it easier to extend
this call with options.

See https://github.com/pulumi/pulumi-docker-containers/pull/232

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-08-19 15:55:54 +00:00
Fraser Waters 7bed5ddbcc
Fix potential nil panic in FindExecutable ()
Found by https://github.com/uber-go/nilaway

If Stat returned an error other than `IsNotExist` we would attempt to
dereference the fileInfo and trigger a nil access.

This fixes it so Stat errors are returned as errors (except for
`IsNotExist`).
2024-08-14 08:25:06 +00:00
Thomas Gummerer 73e073efc6
disallow OutputState to be deepcopied ()
We have this deepcopy utility in our public API, but it does not work
properly on structs with unexported values. Make a note of that, and
also disallow OutputState from being copied, since OutputState has
required unexported fields that we can't deepcopy.

Not sure if we should special case `OutputState` here, which I've done
since that has come up in particular, or maybe even disallow
deepcopy'ing any values that have unexported fields? That would be a
breaking change though, so I've shied away from it for now.

Fixes https://github.com/pulumi/pulumi/issues/16634
2024-07-18 12:56:36 +00:00
Fraser Waters e0da9cb39a
Adding conformance tests for Go ()
This adds the framework for running conformance tests for Go. Currently
_all_ the tests are skipped, but I'll raise small PRs on top of this one
to start fixing some of the issues.
2024-07-17 09:00:08 +00:00
Julien ff5fe13fe2
Fix incorrect caching of git auth method in error cases ()
urlAuthParser.Parse caches auth methods. To avoid caching the auth
method when running into an error, we first checked that the auth method
is not nil. However we ran into a classic Go issue where a nil value of
a concrete type is assigned to a variable of an interface type, making 
it not equal to nil.

https://go.dev/doc/faq#nil_error

https://go.dev/play/p/AOSdCWd3XC1

Fixes https://github.com/pulumi/pulumi/issues/16637

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-07-16 14:53:32 +00:00
Fraser Waters 0a83daaa08
Fix CopyFile to copy symlink directories ()
Noticed while working on https://github.com/pulumi/pulumi/pull/16662.
CopyFile couldn't handle a symlink'd directory.
2024-07-16 11:57:14 +00:00
Germán Lena 956428f690
Update deployment settings git configuration to support directories that are not a git repo ()
Change how the deployment settings configures git repos to support
directories that are not git repositories

Fix https://github.com/pulumi/pulumi-service/issues/20675

---------

Co-authored-by: Levi Blackstone <levi@pulumi.com>
2024-07-11 14:14:18 +00:00
Germán Lena a7d5e238b8
New deployment settings wizards and environment variables management comands ()
- Turns `deployment settings init` command a wizard
- Adds new `deployment settings env` command to manage env variables
(including secrets encryption)
- Adds new `deployment settings set` command to configure individual
settings (including secrets encryption)

https://asciinema.org/a/QhuWHAvkmeAmVJkYqkCP0P6wb

Fix https://github.com/pulumi/pulumi-service/issues/20567
Fix https://github.com/pulumi/pulumi-service/issues/20576
2024-07-03 20:24:26 +00:00
Thomas Gummerer 0dc51aadd9
retry post requests that timeout during handshake ()
We've had quite a few test flakes that happen because of TLS handshake
timeouts to the service backend. These are usually during stack creation
(which happens a lot during tests), which are POST requests.

POST requests are not generally safe to retry, because the request could
already have been sent to the backend and an action taken, but the
client couldn't read the response. However when there is a TLS handshake
failure, the request can never have made it to the backend, so these
errors are safe to retry.

This should hopefully help with
https://github.com/pulumi/pulumi/issues/16529.
2024-07-03 16:22:24 +00:00
Julien P a26c010589
Fix TestTerminate_gracefulShutdown/python flake ()
It is unsafe to call `print` in a Python signalhandler. Instead we set a
flag and check this in the main busy loop.

Fixes https://github.com/pulumi/pulumi/issues/16472

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2024-06-26 05:50:58 +00:00
Pat Gavlin 7273bc02c9
[display] Enable WASM compilation ()
These changes contain some minor refactorings to conditionally disable
the use of packages that are cannot be built for `GOOS=js GOARCH=wasm`.
With these edits, `pkg/display` can be built targeting WASM.

These changes act as a safeguard to ensure that we are not adding
additional code that will _prevent_ building `pkg/display` for WASM
targets. They are not sufficient to produce a version of the display
renderer that is appropriate for actual use in a WASM environment:

- The current renderer API is not well-suited for use outside the
context of the CLI
- The current event stream format has no versioning data
- Actually building this code into a WASM module results in an
unpleasantly large file (70M uncompressed, 13M gzipped)

These changes also add a size gate for the built WASM module. The gate
is set to the 110% of the size of the WASM module as of this commit. Our
goal is to lower the size of the WASM module over time; as we do so we
will tighten this gate.

Part of .
2024-05-24 20:27:56 +00:00
Thomas Gummerer 1dd0f7fdc7
Allow CopyFile to copy symlinks ()
Bazel builds create symlinks of each file. We still want to copy these
into test directories etc., so allow CopyFile to do that. This is a
prerequisite for getting pulumi-yaml to build in Bazel.

We hide this with an environment variable, as that's easy to set during
Bazel builds, but does not affect this function during regular usage.
2024-04-16 07:54:36 +00:00
Will Jones eadf1ec185
Support always qualifying stack names () ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This commit adds the `--fully-qualify-stack-names` (or `-Q` for short)
global command-line argument, which when supplied will always print
stack names in the fully-qualified form of `organization/project/stack`.
Fixes .

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [X] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-04 10:11:46 +00:00
Ringo De Smet d513dea5ee
Use `SYSTEM_PULLREQUEST_SOURCECOMMITID` for AZ pipeline on GH PR ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Use `SYSTEM_PULLREQUEST_SOURCECOMMITID` when AZ pipelines checks against
a Github PR. Fix made based on investigation from @blampe in 

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-26 14:55:54 +00:00
Luke Hoban bb3aa1c529
Add `--suppress-progresss` flag to CLI ()
Adds support for suppressing the periodic "..." printing that can
disrupt normal output stream. This output is still deemed necessary to
include by default for CI systems that might otherwise cancel an update
that goes for a long time without printing anything. But we now have an
option to turn this off.

Notes:
1. We want to expose this via Automtation API, and may even want to
default to off via Automation API?

Fixes https://github.com/pulumi/pulumi/issues/14069.
Related https://github.com/pulumi/pulumi/issues/11139.
2024-02-05 11:48:10 +00:00
Fraser Waters 7ba0541e1c
Rename self-managed to diy-backend ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

We're moving away from referring to filestate as "self managed"
backends, preferring to refer to this as "DIY" backends going forward.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-30 09:00:15 +00:00
Paul C. Roberts ad4bec1b46
Use the Depth variants in glog calls ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This allows pulumi's log output to contain the location of where the log
call was made from, rather than it always reporting the line in `log.go`
where the glog call was made.
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This changes

```
I0124 20:53:35.244319  169013 log.go:81] skipping update check
I0124 20:53:36.262079  169013 log.go:81] found username for access token
```
to

```
I0129 14:08:42.918230   38925 pulumi.go:231] skipping update check
I0129 14:08:45.056866   38925 backend.go:615] found username for access token
```

## Checklist

- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
  - [X] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
- This is hard to test because it is difficult to reliably intercept
stderr from within go tests
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

Co-authored-by: Paul Roberts <proberts@pulumi.com>
2024-01-29 22:51:45 +00:00
Fraser Waters 8bee243efb
Move buildutil out of sdk/common to the go language host ()
Same as https://github.com/pulumi/pulumi/pull/15165, a small bit of code
we can move just to the language host.
2024-01-17 22:25:47 +00:00
Thomas Gummerer baecc85eaf
turn on the golangci-lint exhaustive linter ()
Turn on the golangci-lint exhaustive linter.  This is the first step
towards catching more missing cases during development rather than
in tests, or in production.

This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.

I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
 
Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 16:50:41 +00:00
Fraser Waters 6162d16eb2
Move goversion out of sdk/common to the go language host ()
Another small part we can pull out of sdk/go/common.
2024-01-17 14:56:18 +00:00
Fraser Waters 3be1b6289c
Remove deprecated Protobufs imports ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

github.com/golang/protobuf is marked deprecated and I was getting
increasingly triggered by the inconsistency of importing the `Empty`
type from "github.com/golang/protobuf/ptypes/empty" or
"google.golang.org/protobuf/types/known/emptypb" as "pbempty" or "empty"
or "emptypb". Similar for the struct type.

So this replaces all the Protobufs imports with ones from
"google.golang.org/protobuf", normalises the import name to always just
be the module name (emptypb), and adds the depguard linter to ensure we
don't use the deprecated package anymore.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 09:35:20 +00:00
Fraser Waters 167e0ade10
Remove Colorization.Auto ()
This value was never actually used or correctly implemented, it was just
being used as a sentinal value for `Set/GetGlobalColorization`. We can
just use a pointer instead.
2024-01-03 16:33:07 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Fraser Waters 0f4ddc2ccf
Use EqualError/ErrorContains instead of Error ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This is a pass over all of /sdk to replace asserts that just checked we
had an error with asserts for what the error value is.

Just checking for an error is a weak test that can result in error paths
being broken and tests not detecting it.
2023-12-08 06:40:14 +00:00
Thomas Kappler 967f92b566
Don't swallow file not found in fsutil.FileCopy ()
# Description

`FileCopy` silently swallows file not found errors which leads to hard
to debug issues. For instance, as described in , tests with edit
steps unexpectedly run twice against the first step.

Fixes 
Fixes  

Merging this will break plenty of existing tests if they're not fixed
first. [This
query](https://github.com/search?q=org%3Apulumi+path%3A_test.go+%2F%5E%5Ct*Dir%3A+*%22%2F&type=code)
is a rough indication.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-12-06 13:29:40 +00:00
Fraser Waters d078735823
Reimport appdash from our mirror ()
Fixes https://github.com/pulumi/pulumi/issues/14646.
2023-11-30 14:21:35 +00:00
Fraser Waters c56bb05174
Update golangci-lint () 2023-11-21 15:16:13 +00:00
Fraser Waters d771acf707
Add tokens.StackName ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This adds a new type `tokens.StackName` which is a relatively strongly
typed container for a stack name. The only weakly typed aspect of it is
Go will always allow the "zero" value to be created for a struct, which
for a stack name is the empty string which is invalid. To prevent
introducing unexpected empty strings when working with stack names the
`String()` method will panic for zero initialized stack names.
 
Apart from the zero value, all other instances of `StackName` are via
`ParseStackName` which returns a descriptive error if the string is not
valid.

This PR only updates "pkg/" to use this type. There are a number of
places in "sdk/" which could do with this type as well, but there's no
harm in doing a staggered roll out, and some parts of "sdk/" are user
facing and will probably have to stay on the current `tokens.Name` and
`tokens.QName` types.

There are two places in the system where we panic on invalid stack
names, both in the http backend. This _should_ be fine as we've had long
standing validation that stacks created in the service are valid stack
names.

Just in case people have managed to introduce invalid stack names, there
is the `PULUMI_DISABLE_VALIDATION` environment variable which will turn
off the validation _and_ panicing for stack names. Users can use that to
temporarily disable the validation and continue working, but it should
only be seen as a temporary measure. If they have invalid names they
should rename them, or if they think they should be valid raise an issue
with us to change the validation code.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 07:44:54 +00:00
Pat Gavlin 3b1150bcdc
[cli] Colorize table headers ()
These changes colorize the header row of each table printed by the CLI
using the `SpecHeadline` style (bold + pinkish). This improves the
readability of the tables on rich terminals.
2023-11-14 22:52:57 +00:00
Eron Wright 2fcdaf8445
Fix panic in mapper_encode ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description
Fixes a panic that occurs when attempting to encode certain values, for
example a map that contains a nil value:
```go
	bag := bagtag{
		String:    "something",
		StringOpt: "ohmv",
		MapOpt: map[string]interface{}{
			"a": "something",
			"b": nil,   // this reflect.Value is of type Interface and IsNil is true.
		},
	}
        _ = resource.NewPropertyMap(bag) // panic here
```

New tests are provided to cover the above scenario and to cover each
type of value, including nil cases where appropriate.

Fixes: https://github.com/pulumi/pulumi-kubernetes/issues/2622
Follow-up to: https://github.com/pulumi/pulumi/pull/9810

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-04 18:31:22 +00:00
Fraser Waters cf5b4a2790
Use `assert.NoError` rather than `assert.Nil` ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Likewise `require.NoError` instead of `require.Nil`, and `assert.Error`
rather than `assert.NotNil`.

The error variants of these functions print the errors nicer for test
failures using `Error()` rather than `GoString()`.

For bail errors this is _much_ better than the `result.Result` days
where we now get errors like:
```
Error:      	Received unexpected error:
            	BAIL: inner error
```
instead of:
```
Error:      	Expected nil, but got: &simpleResult{}
```

Also print the bail error in `TestPlan.Run` so we can see the
description of it.
2023-10-13 09:46:07 +00:00
Abhinav Gupta 615bd0f0e7
cmdutil: Support errors.Join-based multi-errors ()
cmdutil has some special handling for hashicorp/go-multierror
so that multi-errors are printed cleanly in the form:

    %d errors occurred:
        1) foo
        2) bar
        ...

In Go 1.20, the errors package got a native `errors.Join` function.
This adds support for errors.Join-based multi-errors to this logic.

These errors implement an `Unwrap() []error` method
which can be used to access the full list of errors.
We use that and then implement the same logic for formatting as before.
2023-09-01 19:01:16 +00:00
Kyle Dixler 0f5dbd2a56
Revert "resource/plugin: Shut down plugins gracefully ()" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes 

This reverts commit 25901d95c4.

On linux, `pulumi up` hung during a `pulumi-command provider` panic
after this change was introduced.
2023-08-31 16:29:55 +00:00
Abhinav Gupta b51caa6ab4
cmdutil.ReadConsole[NoEcho]: Use bubbletea ()
Switch the cmdutil.ReadConsole and cmdutil.ReadConsoleNoEcho functions
to use the bubbletea library to render the prompt,
using the textinput widget provided by the accompanying bubbles library.
The resulting input widgets support arrow keys, back space,
and some basic readline-style bindings including Ctrl-A, Alt-B, etc.

I went through all uses of ReadConsole or ReadConsoleNoEcho.
Only the one in new.go had a non-compliant prompt that I had to adjust.

Note: One divergence in behavior I opted for was that
password prompts will echo '*' characters as the user is typing
and then no echo once they've accepted or canceled the value.
Previously, the prompt did not echo anything in either case.

<details>

  <summary>
  Introduction if you're unfamiliar with bubbletea
  </summary>

  bubbletea operates by modeling the widget state as
  an immutable data structure that receives messages for events.
  On receiving a message (key press, e.g.) the model's Update method
  returns a new model instance representing its new state.
  Update may also optionally return additional commands for the program,
  e.g. stop running, or print something and move on.
  The model's View method returns what should be drawn in the terminal
  based on the model's current state.
This programming model makes it reasonably straightforward to unit test
  some of the core functionality of independent widgets
  as demonstrated in this PR.

</details>

Resolves 

---

Demos:

<details>
  <summary>Plain text</summary>
  

![prompt-plain](https://github.com/pulumi/pulumi/assets/41730/66258fc8-f772-4d01-bc7c-1f7b116aebaa)
</details>

<details>
  <summary>Secret</summary>


![prompt-secret](https://github.com/pulumi/pulumi/assets/41730/372f862e-9186-4d47-ba7d-0107c47f52f6)
</details>

<details>
  <summary>Secret prompt with padding</summary>


![prompt-secret-2](https://github.com/pulumi/pulumi/assets/41730/e9b7c253-4c9d-4235-9fa6-197aa0522033)
</details>
2023-08-30 17:08:44 +00:00
Abhinav Gupta 25901d95c4
resource/plugin: Shut down plugins gracefully ()
Uses the new TerminateProcessGroup functionality introduced in 
to shut down plugins gracefully.

Graceful shutdown takes the following form:

- Send a termination signal (SIGINT or CTRL_BREAK_EVENT)
- Wait up to 1 second for the plugin to exit
- Kill it with SIGKILL

Note that TerminateProcessGroup kills the entire group
so we don't need a separate KillChildren and cmd.Process.Kill().

This change also deprecates cmdutil.KillChildren
since we shouldn't really be using SIGKILL as a first resort anyway.

Note that this does not modify the behavior of individual plugins.
Those will exit as usual but with a SIGINT instead of SIGKILL
terminating them.

Resolves 
2023-08-29 12:02:54 +00:00
Fraser Waters 3d9ddb2981
Support bailing from RunFunc ()
**Background**

The result.Result type is used by our CLI implementation to communicate
how we want to exit the program.

Most `result.Result` values (built from errors with `result.FromError`)
cause the program to print the message to stderr and exit the program
with exit code -1.
The exception is `result.Bail()`, which indicates that we've already
printed the error message, and we simply need to `exit(-1)` now.

Our CLI command implementation use `cmdutil.RunResultFunc` which takes a
`func(...) result.Result` to implement this logic.

`cmdutil` additionally includes a `cmdutil.RunFunc` which takes a
`func(...) error` and wraps it in `RunResultFunc`, relying on
`result.FromError` for the conversion:

    func RunFunc(run func(...) error) func(...) {
        return RunResultFunc(func(...) result.Result {
            if err := run(...); err != nil {
                return result.FromError(err)
            }
            return nil
        })
    }

**Problem**

In CLI contexts where we're using an `error`, and we want to print an
error message to the user and exit, it's desirable to use diag.Sink to
print the message to the user with the appropriate level (error,
warning, etc.) and exit without printing anything else.

However, the only way to do that currently is by converting that
function to return `result.Result`, turn all error returns to
`result.FromError`, and then return `result.Bail()`.

**Solution**

This change introduces a `result.BailError` error that gets converted
into a `result.Bail()` when it passes through `result.FromError`.

It allows commands implementations that use `error` to continue
returning errors and still provide an ideal CLI experience.

It relies on `errors.As` for matching, so even if an intermediate layer
wraps the error with `fmt.Errorf("..: %w", ErrBail)`, we'll recognize
the request to bail.

BailError keep track of the internal error that triggered it, which
(when everything is moved off of result and onto error) means we'll
still be able to see the internal errors that triggered a bail during
debugging.

Currently debugging engine tests is pretty horrible because you often
just get back a `result.Result{err:nil}` with no information where in
the engine stack that came from.

**Testing**

Besides unit tests, this includes an end-to-end test for using
RunResultFunc with a bail error.
The test operates by putting the mock behavior in a fake test, and
re-running the test binary to execute *just that test*.

**Demonstration**

This change also ports the following commands to use BailError: cancel,
convert, env, policy rm, stack rm.

These command implementations are simple and were able to switch easily,
without bubbling into a change to a bunch of other code.
2023-08-29 07:43:40 +00:00
Abhinav Gupta b67c72149c
feat(cmdutil): TerminateProcessGroup for graceful termination ()
Adds a new **currently unused** function TerminateProcessGroup
that terminates all processes in a group gracefully.

It does so by first sending the process
a SIGINT on Unix systems, and CTRL_BREAK_EVENT on Windows,
and waiting a specified duration for the process to exit.
The choice of signals was very deliberate
and is documented in the comments for TerminateProcessGroup.

If the process does not exit in the given duration,
it and its child processes are forcibly terminated
with SIGKILL or equivalent.

Testing:
The core behaviors are tested against Python, Go, and Node.
The corner cases of signal handling are tested
against rogue Go processes.
The changes were experimented with in .

Refs 
2023-08-27 22:05:44 +00:00
Abhinav Gupta f0f38d6f18
chore(cmdutil): Rename build-tagged files ()
Files tagged with OS-specific build tags
should usually have a suffix in their name
indicating which OS they're narrowing to.

child_windows.go already does this,
but child.go aimed at Linux and macOS does not.
This change renames child.go to child_unix.go.

Further, it moves the go:build directives
outside the copyright block.
2023-08-25 02:14:34 +00:00
Bryce Lampe 9a8a677a39 [cli/new] Support SSH-style Git URLs
This adds support for SSH-style Git URLs, enabling folks to use
private repos for their templates.

For instance,

    $ pulumi new git@github.com:acmecorp/templates/website

will now work as intended.

The `ssh_config` library handles finding the relevant SSH key for the
given host.

If the SSH key is protected by a password, the user will be prompted to
supply the password on-demand. (It is memoized to avoid asking multiple
times, as the template workflow requires using it more than once.) To
avoid prompting, the `PULUMI_GITSSH_PASSPHRASE` env var can be set.

Fixes  and .
2023-08-02 09:29:34 -07:00
Pat Gavlin 88290d8cb2 [cli] Add project root update metadata
These changes add the path to the current project relative to the root
of the VCS repository to update metadata. If VCS info is not available,
this metadata is not added.

Part of https://github.com/pulumi/home/issues/2946.
2023-07-17 17:06:41 -07:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
Fraser Waters 46332e7d17 Make convert more lenient
Fixes https://github.com/pulumi/pulumi/issues/13117

This adds a new "--strict" flag to `pulumi convert` which defaults to
false. When strict is NOT set we bind the PCL with the extra options of
`SkipResourceTypechecking`, `AllowMissingVariables`, and
`AllowMissingProperties`. This will change some errors to warnings in
code generation.

The `strict` flag is sent over the gRPC interface to the Go/Node plugins
for their `GenerateProject` methods as they have to do PCL binding
plugin side currently.
2023-06-08 11:14:31 +01:00
bors[bot] 4c91d24aab
Merge
13030: [sdk] Update uniseg r=pgavlin a=pgavlin

The latest version of this module dramatically improves its allocation volume and offers a friendlier API for measuring string width.

Co-authored-by: Pat Gavlin <pat@pulumi.com>
2023-05-26 05:26:03 +00:00
Pat Gavlin a8f41f031b [sdk] Update uniseg
The latest version of this module dramatically improves its allocation
volume and offers a friendlier API for measuring string width.
2023-05-25 22:24:13 -07:00
Pat Gavlin 864da4bdf9 [logging] Guard verbose logs on glog.Verbose
`glog.Verbose(int)` returns a bool that indicates whether or not
logging is enabled at the given level. Check that result prior to
actually calling the logger in order to avoid extra work when verbose
logging is disabled.
2023-05-25 16:37:06 -07:00
Pat Gavlin 6a4740e8cb [cli] Better memory profiling
The current memory profile reports the live objects and their sources at
the time at which it is captured. This is almost never what we want:
instead, we want a profile of _all_ allocations over the lifetime of the
process. These changes replace the heap profile with an allocation
profile and add a parameter, --memprofilerate, to control the rate at
which allocations are profiled.
2023-05-25 09:20:12 -07:00