Commit Graph

16 Commits

Author SHA1 Message Date
Fraser Waters fe2c6c0515 Support subdirectories in proto 2023-03-05 16:05:58 +00:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Aaron Friel 9f9989af97 Update Go gRPC, require impl to embed UnimplementedXServer 2023-01-05 18:00:16 -08:00
Fraser Waters 920e928886 Fix the go_package option on the .proto files
This doesn't really effect anything with our current usage, but if we
ever put proto files in another package and try to import the current
set it wouldn't actually be able to find them.

I noticed this while working on
https://github.com/pulumi/pulumi/pull/10792 where I added a new
"pulumirpc.engine" package. The proto file could refer to things like
"pulumirpc.PluginDependency" but the go code then tried to import it
like `_go "github.com/pulumi/pulumi/v3/proto/go/pulumirpc"` which isn't
actually the correct import path (missing the sdk folder part, and there
isn't acutally a folder called pulumirpc).
2022-09-20 09:31:03 +01:00
Fraser Waters 1beb4f827e
Compile protobufs with "pulumi" namespace ()
* Move proto file to top level

* Regenerate go protobufs

* Regenerate python protobufs

* Fix dotnet protobuf references

* Update README

* Fix up nodejs protobufs

* Add to CHANGELOG

* lint

* Remove google/protobuf/status

* Revert server changes

* Restore pyi files

* Revert "Remove google/protobuf/status" for nodejs

This reverts commit d692f00bd2.
2022-07-12 14:45:03 +01:00
Fraser Waters d56ba6c918
Update protoc-gen-go to 1.5.2 ()
* Update protoc-gen-go to 1.5.2

* lint
2022-06-11 08:52:24 +01:00
James Nugent a1b55192dd Regenerate protocol buffers code 2020-02-28 11:53:47 +00:00
Pat Gavlin a222705143
Implement first-class providers. ()
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.

### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.

These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
  or unknown properties. This is necessary because existing plugins
  only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
  values are known or "must replace" if any configuration value is
  unknown. The justification for this behavior is given
  [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
  configures the provider plugin if all config values are known. If any
  config value is unknown, the underlying plugin is not configured and
  the provider may only perform `Check`, `Read`, and `Invoke`, all of
  which return empty results. We justify this behavior becuase it is
  only possible during a preview and provides the best experience we
  can manage with the existing gRPC interface.

### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.

All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.

Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.

### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
  provider plugins. It must be possible to add providers from a stack's
  checkpoint to this map and to register new/updated providers during
  the execution of a plan in response to CRUD operations on provider
  resources.
- In order to support updating existing stacks using existing Pulumi
  programs that may not explicitly instantiate providers, the engine
  must be able to manage the "default" providers for each package
  referenced by a checkpoint or Pulumi program. The configuration for
  a "default" provider is taken from the stack's configuration data.

The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").

The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.

During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.

While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.

### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
  to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
  a particular provider instance to manage a `CustomResource`'s CRUD
  operations.
- A new type, `InvokeOptions`, can be used to specify options that
  control the behavior of a call to `pulumi.runtime.invoke`. This type
  includes a `provider` field that is analogous to
  `ResourceOptions.provider`.
2018-08-06 17:50:29 -07:00
Alex Clemmer 0e39b3c868 Add `Cancel` to gRPC resource provider interface 2018-07-15 11:05:44 -10:00
Sean Gillespie 8b9e24cd85 Allow dynamic-provider to send structured errors
A critical part of the partial update protocol is to return a structured
error when a resource is successfully created, but fails to initialize.
This structured error contains the properties of the
partially-initialized resource, and instructs the engine to halt.

Most languages implement this by attaching "details" to the error, i.e.,
an arbitrary proto message attached to the error. The JavaScript
implementation is not mature enough to include all the facilities
required to use this, so here we must add a `Status` message, which
protobuf requires as part of its structure for returning details.
2018-07-02 13:32:23 -07:00
Alex Clemmer 3bd2e6f235 Represent init errors in resource provider proto 2018-07-02 13:32:23 -07:00
Sean Gillespie c2b2f3b117
Initial Python 3 port of the Python SDK ()
* Machine-assisted Python 3 port

* Hack around protoc python imports

* Regenerate gRPC and protobuf

* CR: Bash code cleanup
2018-06-26 11:14:03 -07:00
Joe Duffy 479a2e6ad5
Add an ID property to ReadResponse ()
The RPC provider interface needs a way to convey back to the engine
that a resource being read no longer exists.  To do this, we'll return
the ID property that was read back.  If it is empty, it means the
resource is gone.  If it is non-empty, we expect it to match the input.
2018-04-10 12:58:50 -07:00
CyrusNajmabadi a759f2e085
Switch to a resource-progress oriented view for pulumi preview/update/destroy () 2018-04-10 12:03:11 -07:00
joeduffy 22584e7e37 Make some resource model changes
This commit changes two things about our resource model:

* Stop performing Pulumi Engine-side diffing of resource state.
  Instead, we defer to the resource plugins themselves to determine
  whether a change was made and, if so, the extent of it.  This
  manifests as a simple change to the Diff function; it is done in
  a backwards compatible way so that we continue with legacy diffing
  for existing resource provider plugins.

* Add a Read RPC method for resource providers.  It simply takes a
  resource's ID and URN, plus an optional bag of further qualifying
  state, and it returns the current property state as read back from
  the actual live environment.  Note that the optional bag of state
  must at least include enough additional properties for resources
  wherein the ID is insufficient for the provider to perform a lookup.
  It may, however, include the full bag of prior state, for instance
  in the case of a refresh operation.

This is part of .
2018-04-05 08:14:25 -07:00
Sean Gillespie 91c550f1e0
Send structured errors across RPC boundaries ()
* Send structured errors across RPC boundaries

This brings us closer to gRPC best practices where we send structured
errors with error codes across RPC endpoints. The new "rpcerrors"
package can wrap errors from RPC endpoints, so RPC servers can attach
some additional context as to why a request failed.

* Code review feedback:

1. Rename rpcerrors -> rpcerror, better package name
2. Rename RPCError -> Error, RPCErrorCause -> ErrorCause, names
suggested by gometalinter to improve their package-qualified names
3. Fix import organization in rpcerror.go
2018-03-28 17:07:35 -07:00