Commit Graph

13 Commits

Author SHA1 Message Date
Will Jones b85a92cd5c
Have `Host.Provider` accept a `PackageDescriptor` ()
Plugins are the core means by which Pulumi may be extended. Language
hosts, resource providers, analyzers, and converters, for instance, are
all kinds of plugin. Plugins are loaded by a plugin `Host`, which also
offers convenience methods for loading specific kinds of plugin such as
those mentioned above.

The `Provider` method on `Host` currently accepts a name and version.
This is not ideal, since there are several other parameters that may
affect the plugin to be loaded, as well as what operations may be run on
it when it is loaded:

* Custom download URLs and checksums may be desirable to control where a
plugin is retrieved from, and to verify a plugin's integrity.
* Parameterization means that while the `aws` provider is desired, it is
actually provided by a dynamically-bridging `terraform` plugin which is
to be supplied with a parameter such as
`{"name":"aws","version":"..."}`.

This PR begins reworking the `Host` interface so that its `Provider`
method accepts a more complete `PackageDescriptor`, consisting of a full
`PluginSpec` and an optional `Parameterization`. Presently this PR just
replicates existing call sites to use the new data structure -- if this
merges successfully then several of these call sites can likely be
cleaned up further by moving duplicated logic that handles things like
custom download URLs, etc. _into_ the newly capable `Provider`
implementation.
2024-09-12 13:17:30 +00:00
Ian Wahbe 78c48204e0
Normalize plugin.Provider methods to (Context, Request) -> (Response, error) ()
Normalize methods on plugin.Provider to the form:

```go
Method(context.Context, MethodRequest) (MethodResponse, error)
```

This provides a more consistent and forwards compatible interface for
each of our methods.

---

I'm motivated to work on this because the bridge maintains a copy of
this interface: `ProviderWithContext`. This doubles the pain of dealing
with any breaking change and this PR would allow me to remove the extra
interface. I'm willing to fix consumers of `plugin.Provider` in
`pulumi/pulumi`, but I wanted to make sure that we would be willing to
merge this PR if I get it green.

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-07 19:47:49 +00:00
Germán Lena d7f24dfcfb
Refactor: move plugin kind to apitype ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR moves PluginKind to apitype to prevent circular dependencies
when adding apitype as a dependency of the workspace module.
It also re-exports PluginKind to keep backward compatibility

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-25 17:30:30 +00:00
Fraser Waters 40ae5b98ca
Fix flakyness in TestPluginMapper_MappedNamesDifferFromPulumiName ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes flakiness in `TestPluginMapper_MappedNamesDifferFromPulumiName`.
2024-01-25 15:17:38 +00:00
Fraser Waters 26bf308dd0
Fix infinite loop bug in plugin_mapper ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Found by @GeoffMillerAZ a while back. Just tracked down exactly what
caused it. The test added hangs if ran on the current version of the
code.

Issue was we used to assume that we'd only stop looking for plugins once
there weren't any more plugins to look at. But then we added
`GetMappings` so we could see which providers a plugin mapped to
up-front. So the check needs to actually be stop looking when there's no
plugins _or_ when all the plugins left explicitly say they aren't for
the current lookup.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-23 09:11:33 +00:00
Fraser Waters d0c3cfba8a
Lock in the plugin mapper ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Quick fix for https://github.com/pulumi/pulumi/issues/14718.

This could probably be smarter but this will unblock the use of the new
converter for providers.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works - I don't think a method level lock _needs_ a test, but
I've left a comment that if we try to make this smarter it will need
parallel tests written for it.
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change - users won't have
observed this behavior, just pulumi internal usage.
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-08 11:02:04 +00:00
Fraser Waters 183b47bf06
More efficent mapping lookup ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Inspired by a comment from Zaid. This allows providers to return what
providers they have mapping information for without having to marshal
all their mapping data to the engine at the same time, this could save
transmitting a lot of data that the engine might not ever need (for
example if it's not converting code for that specific provider). It also
allows provider to support mulitple mappings.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-21 11:45:07 +00:00
Fraser Waters 1aaa7dc707 Fix plugin mapper to not retry installs
Fixes https://github.com/pulumi/pulumi/issues/13105

When we failed to find a mapping for a provider we wouldn't save that
result, so on calling GetMapping again we'd try installing and querying
that provider again.
2023-06-06 09:42:09 +01:00
Fraser Waters a66a7f50eb Make GetMapping contextful
Now that we've removed the build link to terraform-bridge we can fix
this TODO. Pass a `context.Context` to `GetMapping` because it's
normally an async method.

This will require a small fix up in terraform-bridge when it updates to
this version of pulumi/pkg.
2023-06-05 21:34:08 +01:00
Fraser Waters 8446f2208c Allow mapping requests to install plugins
Enabling fix for https://github.com/pulumi/pulumi-terraform-bridge/issues/1130

This adds two new bits of logic to the plugin mapper.

Firstly when getting a mapping request for a given package X, as well as
searching to see if we have an X plugin already installed to call it's
`GetMapping` method first, we now also try to install X if we don't
already have it. The install failing is not considered an error, we'll
just try the other plugins to see if they map and then possibly return
empty (as before).

Secondly the converter can pass a hint to what the pulumi plugin for the
ecosystem plugin is. If no hint is given we assume the pulumi plugin
name is the same as the ecosystem name. But this can be used for example
in the terraform converter to tell the engine that for example the
terraform provider "azurerm" is called "azure" in pulumi.

The hint is just a hint, if the name doesn't match all the other plugins
will be searched as before.
2023-05-29 14:53:04 +01:00
Fraser Waters ae2efd4a73 Speed up mapping lookup
Rather than doing a linear search through all the plugins and asking
each in turn for their mapping information, use the heuristic that
normally the Pulumi provider name matches the mapped provider name and
try that plugin first.
2023-04-26 16:38:01 +01:00
Fraser Waters ab5e23b26b Add some tests for the PluginMapper 2023-04-20 09:39:04 +01:00
Abhinav Gupta f9c1c52879 refactor(codegen/convert): Extract PluginMapper to file
Extracts the implementation of PluginMapper
to its own file in anticipation changes to this code.
2023-04-20 09:37:23 +01:00