Commit Graph

53 Commits

Author SHA1 Message Date
Thomas Gummerer baecc85eaf
turn on the golangci-lint exhaustive linter ()
Turn on the golangci-lint exhaustive linter.  This is the first step
towards catching more missing cases during development rather than
in tests, or in production.

This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.

I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
 
Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 16:50:41 +00:00
Fraser Waters 301c9e7b2c
Enable importas linter ()
Follow on from https://github.com/pulumi/pulumi/pull/15164 after team
discussion that importas was more value.

Opted to not make importas too strict to start. It still allows
individual go files to import modules with ad-hoc names, but at least
makes sure the "pulumirpc" module is always named the same.
2024-01-17 14:56:37 +00:00
Fraser Waters 3be1b6289c
Remove deprecated Protobufs imports ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

github.com/golang/protobuf is marked deprecated and I was getting
increasingly triggered by the inconsistency of importing the `Empty`
type from "github.com/golang/protobuf/ptypes/empty" or
"google.golang.org/protobuf/types/known/emptypb" as "pbempty" or "empty"
or "emptypb". Similar for the struct type.

So this replaces all the Protobufs imports with ones from
"google.golang.org/protobuf", normalises the import name to always just
be the module name (emptypb), and adds the depguard linter to ensure we
don't use the deprecated package anymore.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 09:35:20 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta 69762acedf
grpc: WithInsecure is deprecated
grpc.WithInsecure is deprecated [1].
Instead, we're expected to use:

    grpc.WithTransportCredentials(insecure.NewCredentials())

This updates all instances across pkg/, sdk/ and tests/.

  [1]: https://pkg.go.dev/google.golang.org/grpc#WithInsecure

In service to 
2023-01-11 12:15:40 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Aaron Friel a90e3cf193 Clean up unnecessary not-implemented methods 2023-01-05 18:00:20 -08:00
Aaron Friel 67f4dda5c5 Embed default gRPC server implementations 2023-01-05 17:54:11 -08:00
Fraser Waters 14d8b59ea2 First pass of adding the convert mapper 2022-11-29 09:46:44 +00:00
Anton Tayanovskyy 9276574c80 Introduce PULUMI_DEBUG_GRPC support 2022-11-08 14:01:15 -05:00
Anton Tayanovskyy fc52744a0d
Add tracing to `npm install` and fix orphan tracing ()
* WIP: trace npm install

* Fixes orphan GetPluginInfo span in pulumi new

* Extend interceptors to streaming gRPC calls

* Intercept streaming client calls

* Fix lint

* PR feedback

* Update pkg/cmd/pulumi/new.go

Co-authored-by: Robbie McKinstry <thesnowmancometh@gmail.com>

* Lint

Co-authored-by: Robbie McKinstry <thesnowmancometh@gmail.com>
2022-09-12 17:42:27 -04:00
Fraser Waters 55be89e36a
Poll gRPC healthchecks to auto-cancel sub-processes ()
* Poll gRPC healthchecks to auto-cancel sub-processes

This hooks the rpcutils cancellation to a polling process that asks for the health of the engine.

If the engine dies, the health check fails, rpcutil gets cancelled, sub-process quits.
This is hooked into both the langauge plugins, and the shared provider main method.

N.B. Providers started in "attach" mode do not poll the health check as it is expected that these processes we're explictly started by the user and the user will shut them down.

* Fix cancel usage

* Add to CHANGELOG

* if nil check not needed
2022-06-06 13:28:00 +01:00
Fraser Waters c9ba17d450
Allow attaching to running providers ()
* Add Attach call

* Regenerate grpc

* Start plumbing in changes

* Main doens't need a port

* Split Attach into grpc interface

* Change envvar format

* Type test for attach

* lint

* Reformat python

* Implement provider debug for nodejs

* Fix plugin close

* lint

* Add to CHANGELOG

* Set Kill

Co-authored-by: Daniel Bradley <daniel@pulumi.com>
2022-04-19 12:41:18 +01:00
Ian Wahbe 272c4643b2
Update error handling ()
This is the result of a change applied via `go-rewrap-errors`.
2021-11-12 18:37:17 -08:00
Liam White f545f56f1b
Turn off grpc info log in Provider ()
Resolves https://github.com/pulumi/pulumi/issues/7156

Signed-off-by: Liam White <liam@tetrate.io>
2021-10-06 09:10:00 -07:00
Justin Van Patten d812d16329
[sdk/go] Unmarshal output values in component provider ()
This adds support for unmarshaling output values in the CopyTo helper used when copying inputs to an args struct
2021-09-17 19:46:06 -07:00
James Nugent 11fc7e7f2b
Add provider.MainWithOptions helper function ()
This commit adds a new counterpart to `ComponentMain` which accepts
an options struct for specifying callback functions. Currently it
supports `construct` (for components) and `call` (for methods), but is
extensible in a non-breaking fashion in future to support all other
provider methods as they become useful to implement.

The original `ComponentMain` still exists, though it may be desirable to
deprecate it in future in favor of `MainWithOptions`.
2021-07-29 10:31:09 -07:00
Justin Van Patten 84b574f0df
Initial support for resource methods (authoring from Node.js, calling from Python) ()
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
2021-06-30 07:48:56 -07:00
Justin Van Patten 780a0c8a3d
Support defining remote components in Go () 2021-04-16 11:49:21 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Levi Blackstone 709fcbad51
Document Go packages ()
Co-authored-by: Pat Gavlin <pat@pulumi.com>
2021-01-11 11:07:59 -07:00
Levi Blackstone 19a113de7a
Handle invalid UTF-8 characters before RPC calls ()
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.

This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
2020-06-17 13:30:59 -06:00
Komal Ali cc96df7485 update channel options in go 2020-04-23 11:16:36 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle c1d3a8524b move pkg/util/cmdutil -> sdk/go/common/util/cmdutil 2020-03-18 15:39:00 -07:00
evanboyle 7ff46cb4fa move pkg/util/rpcutil -> sdk/go/common/util/rpcutil 2020-03-18 15:37:13 -07:00
evanboyle c3f6ae2451 move pkg/util/logging -> sdk/go/common/util/logging 2020-03-18 15:34:58 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle fba783caf9 move pkg/resource -> sdk/go/common/resource, but leave nested resource packages 2020-03-18 13:36:19 -07:00
Pat Gavlin 82204230e1
Improve tracing support. ()
* Fix some tracing issues.

- Add endpoints for `startUpdate` and `postEngineEventsBatch` so that
  spans for these invocations have proper names
- Inject a tracing span when walking a plan so that resource operations
  are properly parented
- When handling gRPC calls, inject a tracing span into the call's
  metadata if no span is already present so that resource monitor and
  engine spans are properly parented
- Do not trace client gRPC invocations of the empty method so that these
  calls (which are used to determine server availability) do not muddy
  the trace. Note that I tried parenting these spans appropriately, but
  doing so broke the trace entirely.

With these changes, the only unparented span in a typical Pulumi
invocation is a single call to `getUser`. This span is unparented
because that call does not have a context available. Plumbing a context
into that particular call is surprisingly tricky, as it is often called
by other context-less functions.

* Make tracing support more flexible.

- Add support for writing trace data to a local file using Appdash
- Add support for viewing Appdash traces via the CLI
2019-09-16 14:16:43 -07:00
Alex Clemmer dea68b8b37 Implement status sinks
This commit reverts most of  and replaces it with functionally
identical logic, using the notion of status message-specific sinks.

In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.

The original commit message in that PR was:

> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
2018-08-31 15:56:53 -07:00
Alex Clemmer 9e58fd1aaa Revert "Plumb `LogRequest.IsStatus` through the logging subsystem"
This reverts commit 3066cbcbd7.
2018-08-31 15:56:53 -07:00
Alex Clemmer 3066cbcbd7 Plumb `LogRequest.IsStatus` through the logging subsystem 2018-08-30 17:17:20 -07:00
Alex Clemmer b0c383e1e8 Add URN argument to `HostClient#Log`
This will make it possible to correlate log messages to specific
resources.
2018-08-01 11:00:48 -07:00
joeduffy 5967259795 Add license headers 2018-05-22 15:02:47 -07:00
CyrusNajmabadi 72e00810c4
Filter the logs we emit to glog so that we don't leak out secrets. () 2018-05-15 15:28:00 -07:00
Pat Gavlin 97ace29ab1
Begin tracing Pulumi API calls. ()
These changes enable tracing of Pulumi API calls.

The span with which to associate an API call is passed via a
`context.Context` parameter. This required plumbing a
`context.Context` parameter through a rather large number of APIs,
especially in the backend.

In general, all API calls are associated with a new root span that
exists for essentially the entire lifetime of an invocation of the
Pulumi CLI. There were a few places where the plumbing got a bit hairier
than I was willing to address with these changes; I've used
`context.Background()` in these instances. API calls that receive this
context will create new root spans, but will still be traced.
2018-05-07 18:23:03 -07:00
Pat Gavlin a23b10a9bf
Update the copyright end date to 2018. ()
Just what it says on the tin.
2018-03-21 12:43:21 -07:00
joeduffy d9a143c8a1 Implement the Python langhost RPC server
This change adds a basic Python langhost RPC server.  It's fairly
barebones and merely acts as a jumping off point for the Pulumi engine
to spawn a Python program.  The host is written in Go, in contrast to
implementing the host in Python, and more closely resembles how I
expect the Node.js language host to work once  is done.
2018-02-23 19:33:02 -08:00
Luke Hoban af5298f4aa
Initial work on tracing support ()
Adds OpenTracing in the Pulumi engine and plugin + langhost subprocesses.

We currently create a single root span for any `Enging.plan` operation - which is a single `preview`, `update`, `destroy`, etc.

The only sub-spans we currently create are at gRPC boundaries, both on the client and server sides and on both the langhost and provider plugin interfaces.

We could extend this to include spans for any other semantically meaningful sections of compute inside the engine, though initial examples show we get pretty good granularity of coverage by focusing on the gRPC boundaries.

In the future, this should be easily extensible to HTTP boundaries and to track other bulky I/O like datastore read/writes once we hook up to the PPC and Pulumi Cloud.

We expose a `--trace <endpoint>` option to enable tracing on the CLI, which we will aim to thread through to subprocesses.

We currently support sending tracing data to a Zipkin-compatible endpoint.  This has been validated with both Zipkin and Jaeger UIs.

We do not yet have any tracing inside the TypeScript side of the JS langhost RPC interface.  There is not yet automatic gRPC OpenTracing instrumentation (though it looks like it's in progress now) - so we would need to manually create meaningful spans on that side of the interface.
2017-11-08 17:08:51 -08:00
Joe Duffy f6e694c72b Rename pulumi-fabric to pulumi
This includes a few changes:

* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.

* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.

* The CLI is renamed from lumi to pulumi.
2017-09-21 19:18:21 -07:00
joeduffy f189c40f35 Wire up Lumi to the new runtime strategy
🔥 🔥 🔥  🔥 🔥 🔥

Getting closer on .
2017-09-04 11:35:21 -07:00
joeduffy 35aa6b7559 Rename pulumi/lumi to pulumi/pulumi-fabric
We are renaming Lumi to Pulumi Fabric.  This change simply renames the
pulumi/lumi repo to pulumi/pulumi-fabric, without the CLI tools and other
changes that will follow soon afterwards.
2017-08-02 09:25:22 -07:00
joeduffy a6caef973a Make assets and archives 1st class
This change recognizes assets and archives as 1st class resource
property values.  This is necessary to support them in the new bridge
work, and lays the foundation for fixing .

I also took the opportunity to clean up some old cruft in the
resource properties area.
2017-07-14 12:28:43 -07:00
joeduffy aeefc27a08 Add some ReadLocations tracing; and skip nulls 2017-07-06 00:02:33 -04:00
joeduffy 06ad983541 Add a ReadLocations engine-side RPC function
This adds a ReadLocations RPC function to the engine interface, alongside
the singular ReadLocation.  The plural function takes a single token that
represents a module or class and we will then return all of the module
or class (static) properties that are currently known.
2017-07-01 13:26:49 -07:00
joeduffy 0ea55328f6 Specialize the null message for config reads 2017-06-27 10:45:29 -07:00
joeduffy 2daea4c3d8 Clarify aspects of using the DCO 2017-06-26 14:46:34 -07:00