Commit Graph

21 Commits

Author SHA1 Message Date
Fraser Waters 183b47bf06
More efficent mapping lookup ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Inspired by a comment from Zaid. This allows providers to return what
providers they have mapping information for without having to marshal
all their mapping data to the engine at the same time, this could save
transmitting a lot of data that the engine might not ever need (for
example if it's not converting code for that specific provider). It also
allows provider to support mulitple mappings.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-21 11:45:07 +00:00
Pat Gavlin 948bb36e7e [engine] Add support for source positions
These changes add support for passing source position information in
gRPC metadata and recording the source position that corresponds to a
resource registration in the statefile.

Enabling source position information in the resource model can provide
substantial benefits, including but not limited to:

- Better errors from the Pulumi CLI
- Go-to-defintion for resources in state
- Editor integration for errors, etc. from `pulumi preview`

Source positions are (file, line) or (file, line, column) tuples
represented as URIs. The line and column are stored in the fragment
portion of the URI as "line(,column)?". The scheme of the URI and the
form of its path component depends on the context in which it is
generated or used:

- During an active update, the URI's scheme is `file` and paths are
  absolute filesystem paths. This allows consumers to easily access
  arbitrary files that are available on the host.
- In a statefile, the URI's scheme is `project` and paths are relative
  to the project root. This allows consumers to resolve source positions
  relative to the project file in different contexts irrespective of the
  location of the project itself (e.g. given a project-relative path and
  the URL of the project's root on GitHub, one can build a GitHub URL for
  the source position).

During an update, source position information may be attached to gRPC
calls as "source-position" metadata. This allows arbitrary calls to be
associated with source positions without changes to their protobuf
payloads. Modifying the protobuf payloads is also a viable approach, but
is somewhat more invasive than attaching metadata, and requires changes
to every call signature.

Source positions should reflect the position in user code that initiated
a resource model operation (e.g. the source position passed with
`RegisterResource` for `pet` in the example above should be the source
position in `index.ts`, _not_ the source position in the Pulumi SDK). In
general, the Pulumi SDK should be able to infer the source position of
the resource registration, as the relationship between a resource
registration and its corresponding user code should be static per SDK.

Source positions in state files will be stored as a new `registeredAt`
property on each resource. This property is optional.
2023-07-10 14:35:40 -07:00
Fraser Waters 1839e15e4a Phase 3 of the convert mapper 2022-12-01 23:43:43 +00:00
Fraser Waters 1beb4f827e
Compile protobufs with "pulumi" namespace ()
* Move proto file to top level

* Regenerate go protobufs

* Regenerate python protobufs

* Fix dotnet protobuf references

* Update README

* Fix up nodejs protobufs

* Add to CHANGELOG

* lint

* Remove google/protobuf/status

* Revert server changes

* Restore pyi files

* Revert "Remove google/protobuf/status" for nodejs

This reverts commit d692f00bd2.
2022-07-12 14:45:03 +01:00
Levi Blackstone be26786e1d
Update docstring for Cancel gRPC method ()
Add a more comprehensive description about the semantics of the provider Cancel operation. This description was copied from the pulumi-kubernetes provider where this feature was originally implemented.
2022-05-25 19:17:21 -06:00
Fraser Waters c9ba17d450
Allow attaching to running providers ()
* Add Attach call

* Regenerate grpc

* Start plumbing in changes

* Main doens't need a port

* Split Attach into grpc interface

* Change envvar format

* Type test for attach

* lint

* Reformat python

* Implement provider debug for nodejs

* Fix plugin close

* lint

* Add to CHANGELOG

* Set Kill

Co-authored-by: Daniel Bradley <daniel@pulumi.com>
2022-04-19 12:41:18 +01:00
Justin Van Patten 84b574f0df
Initial support for resource methods (authoring from Node.js, calling from Python) ()
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
2021-06-30 07:48:56 -07:00
Pat Gavlin 2585b86aa4
Initial support for remote component construction. ()
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.

Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.

The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.

These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).

An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.

This is the core of .
2020-09-07 19:33:55 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
James Nugent a1b55192dd Regenerate protocol buffers code 2020-02-28 11:53:47 +00:00
Pat Gavlin ccde31b713
Add support for extracting schemas from providers. ()
These changes add a new method to the resource provider gRPC interface,
`GetSchema`, that allows consumers of these providers to extract
JSON-serialized schema information for the provider's types, resources,
and functions.
2020-02-27 16:10:47 -08:00
Justin Van Patten 3bf9067bac
Expose options, parent, deps, and provider config to policies () 2020-02-07 16:11:34 -08:00
Alex Clemmer 25d27d09f9 Add `StreamInvoke` to Provider gRPC interface 2019-11-05 10:47:48 -08:00
Matt Ellis 2ad9afc572 Regnerate gRPC generated code 2019-05-10 17:07:52 -07:00
Pat Gavlin 905e7353e4
Update the provider RPC interface ()
These changes add two new methods to the provider interface and extend
the results of three others.

The new methods are `CheckConfig` and `DiffConfig`, which fill out the
set of methods required for a complete implementation of the
first-class provider design. Though these methods are optional for
backwards compatibility, they should be implemented by all future
providers for the best possible user experience.

The adjusted result types are `DiffResponse`, `ReadResponse`, and
`ErrorResourceInitFailed`. The first has been updated to include a list
of the properties that changed (if any). The latter two now include
an estimated set of inputs for the resource as well as the resource's
state. Together, these three changes enable the engine to determine the
set of inputs that should be specified by a user in order to match those
that describe the resource's current state.

This contributes to , , , and .
2019-03-05 10:49:24 -08:00
Sean Gillespie f284112b4e
Use nightly protoc gRPC plugin for node ()
* Use nightly protoc gRPC plugin for Node

Newer versions of the Node gRPC plugin accept the 'minimum_node_version'
flag, which we can use to instruct protoc to not support Node versions
earlier than Node 6. This allows the compiler to use 'Buffer.from'
instead of the deprecated 'Buffer' constructor, which fixes a
deprecation warning on Node 10.

* Protobuf changes
2018-09-17 15:16:31 -07:00
Alex Clemmer 0e39b3c868 Add `Cancel` to gRPC resource provider interface 2018-07-15 11:05:44 -10:00
Sean Gillespie c2b2f3b117
Initial Python 3 port of the Python SDK ()
* Machine-assisted Python 3 port

* Hack around protoc python imports

* Regenerate gRPC and protobuf

* CR: Bash code cleanup
2018-06-26 11:14:03 -07:00
joeduffy 22584e7e37 Make some resource model changes
This commit changes two things about our resource model:

* Stop performing Pulumi Engine-side diffing of resource state.
  Instead, we defer to the resource plugins themselves to determine
  whether a change was made and, if so, the extent of it.  This
  manifests as a simple change to the Diff function; it is done in
  a backwards compatible way so that we continue with legacy diffing
  for existing resource provider plugins.

* Add a Read RPC method for resource providers.  It simply takes a
  resource's ID and URN, plus an optional bag of further qualifying
  state, and it returns the current property state as read back from
  the actual live environment.  Note that the optional bag of state
  must at least include enough additional properties for resources
  wherein the ID is insufficient for the provider to perform a lookup.
  It may, however, include the full bag of prior state, for instance
  in the case of a refresh operation.

This is part of .
2018-04-05 08:14:25 -07:00
Pat Gavlin a23b10a9bf
Update the copyright end date to 2018. ()
Just what it says on the tin.
2018-03-21 12:43:21 -07:00
joeduffy a045e2fb1e Implement more of the Python runtime
This change includes a lot more functionality.  Enough to actually
run the webserver-py example through previews, updates, and destroys!

* Actually wire up the gRPC connections to the engine/monitor.

* Move the Node.js and Python generated Protobuf/gRPC files underneath
  the actual SDK directories to simplify this generally.  No more
  copying during `make` and, in fact, this was required to give a smoother
  experience with good packages/modules for the Python's SDK development.

* Build the Python egg during `make build`.

* Add support for program stacks.  Just like with the Node.js runtime,
  we will auto-parent any resources without explicit parents to a single
  top-level resource component.

* Add support for component resource output properties.

* Add get_project() and get_stack() functions for retrieving the current
  project and stack names.

* Properly use UNKNOWN sentinels.

* Add a set_outputs() function on Resource.  This is defined by the
  code-generator and allows custom logic for output property setting.
  This is cleaner than the way we do this in Node.js, and gives us a
  way to ensure that output properties are "real" properties, complete
  with member documentation.  This also gives us a hook to perform
  name demangling, which the code-generator typically controls anyway.

* Add package dependencies to setuptools.py and requirements.txt.
2018-02-24 08:58:34 -08:00