![]() `pulumi convert` enables users to convert existing programs written in other languages and tools, such as Terraform, to Pulumi programs which manage the same infrastructure. In order to support Pulumi's wide array of target languages and providers, and the possible multitude of source programs (Terraform, CloudFormation, and so on), conversion is architected as follows: * When invoked through `pulumi convert`, the Pulumi CLI boots up a *converter plugin* responsible for handling the source language. E.g. in the case of converting `--from terraform`, the `pulumi-converter-terraform` plugin will be run. * Converter plugins implement a common gRPC service, which the CLI will call to ask for a PCL (Pulumi configuration language; Pulumi's internal representation) program based on the source. * As part of producing the PCL program, the converter may ask the CLI to *map* names from the source language (e.g. `aws_s3_bucket` in a Terraform program) to equivalent Pulumi names (e.g. `aws:s3/bucket:Bucket`). * As is often the case, when it comes to mapping names, the CLI does not do any work itself -- it instead brokers mapping requests to the set of plugins installed. So in the above example, the CLI might find the `pulumi-resource-aws` plugin and ask it to service the mapping request, before returning to the converter plugin so it can continue to produce PCL. * When PCL is produced, the CLI can proceed with code generation ("Programgen"), driven by an appropriate language host and its `GenerateProgram`/`GenerateProject` gRPC methods. This changeset focuses on the ability for converter plugins to request mappings from the CLI, specifically in the case of *parameterized provider plugins*. A parameterized provider plugin is a plugin that can be sent some data ("parameterization") before any of its other interface methods (e.g. `GetSchema`, `Configure`, and in this case `GetMappings` or `GetMapping`) are called. Parameterized providers underpin Pulumi's "Any Terraform provider" features, in which Terraform providers are dynamically bridged at runtime by the `pulumi-resource-terraform-provider` plugin in response to a parameterization request. When a converter plugin requests mappings, it can offer a "hint" to the CLI as to the plugin that it thinks will provide such a mapping. So e.g. when the Terraform converter sees `aws_s3_bucket`, it will currently send a hint `"aws"` to the CLI so that the CLI will search for (and prioritise) asking a plugin named `aws` for its mappings (as opposed to e.g. enumerating the random set of other plugins it might encounter first). Here, we modify the `Mapper` interface so that hints can be full package descriptors. In doing so, we will be able to modify the converter to ask for the `terraform-provider` *plugin*, but with *parameterization* that causes it to dynamically bridge a Terraform provider such that it can return mappings for that provider. These changes will be made to the `pulumi-converter-terraform` codebase separately after this work is merged and released (the bridge may also need to accommodate this change to the interface, but it should be mechanical). As part of this change, it should be noted that the way mappings are *cached* has been changed. Prior to this work, we would aggressively cache mappings, including those we retrieved "en route" to finding a correct one. So, for instance, if we were asked for `"gcp"` mappings, in the absence of a hint we might first ask for such mappings from the `aws` and `azure` providers (say). Even though these mappings wouldn't match, we'd cache them so that if someone later came to us asking for `aws` mappings, we'd just return them from the cache instead of booting up plugins and making gRPC calls. In the presence of parameterization, it's not clear that being this aggressive is safe any more. There's no guarantee, for instance, that a provider which has provided a set of mappings will never later be able to provide a different set due to e.g. a different parameterization. This change thus refactors the caching behaviour to cache providers that have been requested (and succeeded), but not to cache mappings "found along the way". Hopefully this should still cover the majority of common cases without significant performance costs (regression behaviour such as not double installing plugins should be preserved, for instance). Part of #18187 |
||
---|---|---|
.. | ||
README.md | ||
binder.go | ||
binder_component.go | ||
binder_nodes.go | ||
binder_resource.go | ||
binder_resource_test.go | ||
binder_schema.go | ||
binder_schema_test.go | ||
binder_test.go | ||
binding.md | ||
call.go | ||
component.go | ||
config.go | ||
diagnostics.go | ||
functions.go | ||
functions_test.go | ||
intrinsics.go | ||
invoke.go | ||
local.go | ||
output.go | ||
program.go | ||
resource.go | ||
rewrite_apply.go | ||
rewrite_apply_test.go | ||
rewrite_convert.go | ||
rewrite_convert_test.go | ||
rewrite_properties.go | ||
type.go | ||
utilities.go | ||
utilities_test.go |
README.md
(pcl)=
Pulumi Configuration Language (PCL)
Pulumi Configuration Language (PCL) is an internal representation of Pulumi programs which supports all core concepts of the Pulumi programming model in a minimal form. Although not exposed directly to users today, this intermediate representation is used to support a variety of program conversion tasks, from and to various supported Pulumi languages.
Pulumi supports a number of operations pertaining to PCL programs:
-
Lexing and parsing, in which PCL source is parsed into an abstract syntax tree (AST).
-
Type checking and binding, in which a PCL AST is verified to be well-formed (describing resources, outputs, and so on), associated with ("bound to") a set of Pulumi schema, and type checked and resolved.
-
Code generation ("programgen"), in which a PCL program is converted into a program written in a supported target language. Much of this underpins the core of tools such as
pulumi convert
.
(pcl-program)=
PCL programs
PCL programs are comprised of PCL source files which typically have the suffix
.pp
. This repository contains a large number of examples -- in particular, a
large amount of PCL is used to drive language conformance
tests. PCL programs support the following concepts:
(pcl-resource)=
Resources
Resources are defined as blocks of type resource
. They
correspond to stack resource registrations (see resource
registration and
for more). A resource block must
have exactly two labels -- the first is the resource's name, to be used both as
a reference to it elsewhere in the program and as its name in the resulting
Pulumi stack (see the sections in this documentation on URNs and
resource registration). The resource's body contains a
set of attributes, which correspond to the resource's input properties. An
optional options
block may be used to specify resource options (e.g.
provider
, protect
, ignoreChanges
, etc.).
resource "r" "pkg:index:Resource" {
name = "r"
options {
protect = true
}
}
(pcl-component)=
Components
A component in the context of a PCL program is equivalent to the notion of a
"local" component resource in a Pulumi program.
Components are defined as blocks of type component
. A component block must
have two labels. The first is the component's name, to be used both as a
reference to it elsewhere in the program and as its name in the resulting Pulumi
stack. The second is a path to a directory containing a PCL program (set of
.pp
files) that defines the component's implementation. The component's body
contains a set of attributes, which correspond to the component's input
properties. An optional options
block may be used to specify component options
(e.g. provider
, protect
, etc.).
component "c" "./c" {
name = "c"
value = 42
}
Program generation for components results in e.g. the generation of local
component types, such as those extending the ComponentResource
class in SDKs
exposing that type.
(pcl-output)=
Outputs
Outputs are defined as blocks of type output
. They
correspond exactly to exported stack outputs, such as pulumi.export
in Python
or ctx.Export
in Go, which correspond to
on the stack resource
under the hood. Output blocks must have exactly one label, which is the output's
name. The output's body must contain a single attribute, value
, which is the
output's value.
resource "r" "pkg:index:Resource" {
name = "r"
}
output "o" {
value = r.name
}
(pcl-config)=
Configuration
Configuration variables define the names and schema of values that are
expected to be supplied to a program through Pulumi configuration (e.g. from a
Pulumi.<stack>.yaml
). They are defined as blocks of type
config
, with each block taking labels for the variable's name and type. The
block's body may be used to specify other aspects of the variable, such as a
description and default value. config
blocks are typically used to generate
program code for referring to Pulumi configuration, e.g. by creating and using a
pulumi.Config()
object in NodeJS.
config "key" "string" {
description = "The key to use when encrypting data"
default = ""
}
(pcl-invoke)=
invoke
The invoke
intrinsic is used to express provider function calls and
corresponds to the 's
method.
result1 = invoke("pkg:index:Function", { x = 3, y = 4 })
result2 = invoke("pkg:index:Function", { z = 5 }, { parent = p })
(pcl-call)=
call
The call
intrinsic is used to express a method invocation on a resource and
corresponds to the 's
method.
resource "c" "pkg:index:Component" {
...
}
result = call(c, "method", { x = 3, y = 4 })
(pcl-builtin-function)=
Built-in functions
PCL offers a number of built-in functions for use in expressions. These are
captured by the pulumiBuiltins
function in
.
:::{toctree} :maxdepth: 1 :titlesonly:
Syntax and type checking </pkg/codegen/hcl2/README> Binding </pkg/codegen/pcl/binding> :::