mirror of https://github.com/pulumi/pulumi.git
996afa5388
`pulumi convert` enables users to convert existing programs written in
other languages and tools, such as Terraform, to Pulumi programs which
manage the same infrastructure. In order to support Pulumi's wide array
of target languages and providers, and the possible multitude of source
programs (Terraform, CloudFormation, and so on), conversion is
architected as follows:
* When invoked through `pulumi convert`, the Pulumi CLI boots up a
*converter plugin* responsible for handling the source language. E.g. in
the case of converting `--from terraform`, the
`pulumi-converter-terraform` plugin will be run.
* Converter plugins implement a common gRPC service, which the CLI will
call to ask for a PCL (Pulumi configuration language; Pulumi's internal
representation) program based on the source.
* As part of producing the PCL program, the converter may ask the CLI to
*map* names from the source language (e.g. `aws_s3_bucket` in a
Terraform program) to equivalent Pulumi names (e.g.
`aws:s3/bucket:Bucket`).
* As is often the case, when it comes to mapping names, the CLI does not
do any work itself -- it instead brokers mapping requests to the set of
plugins installed. So in the above example, the CLI might find the
`pulumi-resource-aws` plugin and ask it to service the mapping request,
before returning to the converter plugin so it can continue to produce
PCL.
* When PCL is produced, the CLI can proceed with code generation
("Programgen"), driven by an appropriate language host and its
`GenerateProgram`/`GenerateProject` gRPC methods.
This changeset focuses on the ability for converter plugins to request
mappings from the CLI, specifically in the case of *parameterized
provider plugins*. A parameterized provider plugin is a plugin that can
be sent some data ("parameterization") before any of its other interface
methods (e.g. `GetSchema`, `Configure`, and in this case `GetMappings`
or `GetMapping`) are called. Parameterized providers underpin Pulumi's
"Any Terraform provider" features, in which Terraform providers are
dynamically bridged at runtime by the
`pulumi-resource-terraform-provider` plugin in response to a
parameterization request.
When a converter plugin requests mappings, it can offer a "hint" to the
CLI as to the plugin that it thinks will provide such a mapping. So e.g.
when the Terraform converter sees `aws_s3_bucket`, it will currently
send a hint `"aws"` to the CLI so that the CLI will search for (and
prioritise) asking a plugin named `aws` for its mappings (as opposed to
e.g. enumerating the random set of other plugins it might encounter
first). Here, we modify the `Mapper` interface so that hints can be full
package descriptors. In doing so, we will be able to modify the
converter to ask for the `terraform-provider` *plugin*, but with
*parameterization* that causes it to dynamically bridge a Terraform
provider such that it can return mappings for that provider. These
changes will be made to the `pulumi-converter-terraform` codebase
separately after this work is merged and released (the bridge may also
need to accommodate this change to the interface, but it should be
mechanical).
As part of this change, it should be noted that the way mappings are
*cached* has been changed. Prior to this work, we would aggressively
cache mappings, including those we retrieved "en route" to finding a
correct one. So, for instance, if we were asked for `"gcp"` mappings, in
the absence of a hint we might first ask for such mappings from the
`aws` and `azure` providers (say). Even though these mappings wouldn't
match, we'd cache them so that if someone later came to us asking for
`aws` mappings, we'd just return them from the cache instead of booting
up plugins and making gRPC calls. In the presence of parameterization,
it's not clear that being this aggressive is safe any more. There's no
guarantee, for instance, that a provider which has provided a set of
mappings will never later be able to provide a different set due to e.g.
a different parameterization. This change thus refactors the caching
behaviour to cache providers that have been requested (and succeeded),
but not to cache mappings "found along the way". Hopefully this should
still cover the majority of common cases without significant performance
costs (regression behaviour such as not double installing plugins should
be preserved, for instance).
Part of #18187
|
||
|---|---|---|
| .. | ||
| architecture | ||
| contributing | ||
| debugging | ||
| documentation | ||
| references | ||
| sphinx | ||
| .gitignore | ||
| Makefile | ||
| README.md | ||
| diagrams.md | ||
| documentation.md | ||
| make.py | ||
| requirements.txt | ||
| writing.md | ||
README.md
Pulumi developer documentation
Welcome to the Pulumi developer documentation! This site is designed to provide complete documentation for maintainers of and contributors to Pulumi.
:::{toctree} :maxdepth: 2 :titlesonly:
Home /docs/contributing/README /docs/architecture/README /docs/references/README :::