Commit Graph

17 Commits

Author SHA1 Message Date
Will Jones 15184d702e
Add `Handshake` to the provider protocol ()
Fixes https://github.com/pulumi/pulumi/issues/16876

The pulumi provider protocol gRPC always starts by calling CheckConfig
and then Configure. The problem is that CheckConfig accepts a property
bag, which could contain secrets or outputs or resource references, etc.
However, the engine doesn't know if the provider supports these items
(and vice versa) until Configure is called, since Configure is the call
where the engine and the provider agree on which parts of the protocol
they support.

This introduces a Handshake component to the provider protocol that
establishes which version of the protocol the provider and engine
support. This also adds the plugins root and program directories to that
handshake request allowing a provider to know where it was started up
from. This _also_ replaces `Provider.Attach` as the handshake request
includes the engine address to connect to.

Tasting notes:

* Add `Handshake` with request/response to the protocol. `Handshake`
starts as a high watermark for accepts secrets, accepts resources, etc
but includes the plugins root and program directories (if possible), as
well as the engine address. This pretty much replaces the need for
`Attach`.
* Modify `dialPlugin` (where the engine establishes gRPC connections to
plugins) to take a callback that is used to initialise the connection
* For non-provider plugins, pass `testConnection`, which captures the
logic we have today -- call a dummy gRPC method and observed not
implemented error to confirm connection is live
* For provider plugins, pass `handshake`, which sends a `Handshake` and
captures the response
* All providers thus handshake at boot, as opposed to `Configure`, which
a. happens later and b. is asynchronous
* Modify provider implementation to track a `protocol`; move
`acceptSecrets` and company from `configSource` to there
* If `Handshake` is implemented, populate `protocol` on `dialPlugin`. If
not, fallback to populating in `Configure`
* Invariant: `Configure` implies `protocol`
* The rest is largely plumbing
* We can add similar Handshake methods for the other plugin types as
well

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-11-26 17:35:47 +00:00
Germán Lena d7f24dfcfb
Refactor: move plugin kind to apitype ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR moves PluginKind to apitype to prevent circular dependencies
when adding apitype as a dependency of the workspace module.
It also re-exports PluginKind to keep backward compatibility

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-25 17:30:30 +00:00
Fraser Waters f3e6e9a9dd
Plumb Remote, Component, and LogicalName into the import plugin system ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14532.

14532 was just for Remote and Component, but since raising that we've
added LogicalName as well so this PR also adds support for that.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-24 17:15:30 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Fraser Waters 1ae75580d4
Support diagnostics in state conversion ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Part of fixing
https://github.com/pulumi/pulumi-converter-terraform/issues/42. Allow
the converter to warn the user which resources have missing mapping
data, and thus are likely to not import.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-17 08:08:39 +00:00
Fraser Waters d2cf01284b
Send args from convert to converter plugin ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This matches up with how we send args to converter plugins from
`import`. Looks like this will be useful in at least two places already.
Firstly the bicep converter needs to know which file to convert.
Secondly the terraform convert needs a --convert-examples (or the like)
flag to turn on it's bulk converter mode for the terraform bridge to
use.

Also fixes that we we're limiting `import` to three args, but a
converter may take more.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-18 11:01:13 +00:00
Fraser Waters 981be6c458
Add args to state conversions ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

When importing from other ecosystems we need someway in `import` for the
user to tell the converter plugin _what_ to import.

Simply forwarding on the args pass to `import` looks to be the simplest
option to enable this.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-01 18:07:49 +00:00
Fraser Waters 818caae680 Pass loader to convert plugins 2023-08-09 22:02:57 +01:00
Fraser Waters a691975202 Warn about ambient plugins loaded from $PATH
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.

While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.

This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.

Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.

Re-instates https://github.com/pulumi/pulumi/pull/13607 with a fix for
symlinks included.
2023-08-08 13:11:34 +01:00
Kyle Dixler 86ebe1bbd3 Revert "Warn about ambient plugins loaded from $PATH" 2023-08-04 16:54:16 -07:00
Fraser Waters a5b1590499 Warn about ambient plugins loaded from $PATH
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.

While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.

This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.

Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.
2023-07-27 17:59:44 +01:00
Fraser Waters a28066fea6 Make MapperAddress consistent everywhere by calling it MapperTarget 2023-07-27 11:48:28 +01:00
Fraser Waters ef7b123bba Don't return empty hcl.Diagnostics
Fixes https://github.com/pulumi/pulumi-terraform-bridge/issues/1201

hcl.Diagnostics implements the Error interface, but this means a list of
zero diagnostics still looks like a non-nil error, which throws of
normal "if err == nil" checks.

This changes a few use sites of hcl.Diagnostics to ensure we return nil
when there aren't any diagnostics rather than an empty slice. This
ensures if they get cast to `Error` they don't trip up standard `err ==
nil` checks.
2023-06-08 19:14:41 +01:00
Fraser Waters 3af6ba04d4 Fix converter diagnostics
The converter diagnostics were appending to a pre-allocated but also
pre-sized slice and so returning a load of `nil`s over RPC. This fixes
that and adds some sanity tests for both sides of the interface.
2023-06-05 22:49:53 +01:00
Fraser Waters dbc67b9bc9 Allow converter plugins to return diagnostics 2023-06-05 17:38:59 +01:00
Fraser Waters 02489dbbb7 Add mapping service to converter 2023-03-24 17:09:17 +00:00
Fraser Waters 5013d427e6 Add converter plugin 2023-03-09 22:16:21 +00:00