Pulumi - Infrastructure as Code in any programming language. Build infrastructure intuitively on any cloud using familiar languages 🚀
Go to file
Pat Gavlin 9d0b0fed91 [engine] Support remapping envvars for providers
These changes add support for remapping environment variables when
launching providers. This allows users to work around problems with
dynamic provider configuration that is stored in statefiles causing
problems during refresh and destroy operations.

For a bit of background: `pulumi up` is distinctly different from
`pulumi destroy` and `pulumi refresh` in that it involves running the
Pulumi program associated with the stack's project. As it runs, the
Pulumi program defines the desired state for resources--including
provider resources--using values computed by the program in coordination
with the Pulumi engine. When the program creates a provider resource,
the inputs for the provider are either sourced from the program itself
(i.e. from values provided by the program) or are read out-of-band by
the provider plugin. The exact set of configuration that may be sourced
from the environment is particular to each provider--for example, the
Kubernetes provider uses the ambient `kubeconfig` by default, the AWS
provider reads various AWS-specific environment variables, etc. Any
_explicitly-provided inputs_ are written into the stack's statefile.

For example, consider the following program:

```typescript
import * as aws from "@pulumi/aws";

const usEast1 = new aws.Provider("us-east-1", { region: "us-east-1" });
const defaultRegion = new aws.Provider("default-region");
```

The `usEast1` provider's `region` is explicitly specified by the
program, but the `defaultRegion` provider's `region` will be read from
the environment (e.g. from the `AWS_REGION` environment variable). In
the resulting statefile, the `usEast1` provider's state will include the
`region` input, but the `defaultRegion` provider's state will not.

Because `pulumi refresh` and `pulumi destroy` do not run the Pulumi
program associated with the stack's project, they are unable to
recompute configuration values that were explicitly provided by the
program, and must use the values stored in the statefile. Unfortunately,
this may include credential information, which is what causes the errors
described here. The current workaround--which is certainly not
sufficient for explicitly-instantiated providers--is to use environment
variables to provide credentials out-of-band.

The clearest/most complete solution here is to run the Pulumi program
associated with a stack's project as part of `pulumi refresh` and
`pulumi destroy`. Unfortunately, this is a _major_ behavioral change,
and the exact semantics of the run are not clear.

These changes allow explicitly-instantiated providers to make use of the
same workaround that is available to default providers: pass dynamic,
environmentally-sourced provider configuration in environment variables
rather than as provider inputs. The environment variable remapping allows
users to replace the value for a provider environment variable with the
value of a different environment variable before the provider is loaded.
This allows users to place configuration in environment variables that
the provider would not normally read and remap them to
provider-supported envvars, which allows multiple distinct sets of
environment variables for providers.

For the example above, this might look like so:

```typescript
import * as aws from "@pulumi/aws";

const usEast1 = new aws.Provider("us-east-1", {
    pluginEnvVars: { "AWS_REGION": { from: "US_EAST_1_REGION" } },
});
const defaultRegion = new aws.Provider("default-region");
```

Or, if the providers needed different credentials (much more common):

```typescript
import * as aws from "@pulumi/aws";

const usEast1 = new aws.Provider("us-east-1", {
    pluginEnvVars: {
	"AWS_ACCESS_KEY_ID": { from: "US_EAST_1_AWS_ACCESS_KEY_ID" },
	"AWS_SECRET_ACCESS_KEY": { from: "US_EAST_1_AWS_SECRET_ACCESS_KEY" },
	"AWS_SESSION_TOKEN": { from: "US_EAST_1_AWS_SESSION_TOKEN" },
    },
});
const defaultRegion = new aws.Provider("default-region");
```
2024-06-13 13:44:11 -07:00
.devcontainer Update pulumictl 2023-06-29 11:09:56 +01:00
.github avoid new issues being created when dispatched acceptance tests fail (#16318) 2024-06-05 08:12:30 +00:00
.gitpod Update pulumictl 2023-06-29 11:09:56 +01:00
.vscode Remove experimentalWorkspaceModule from vscode settings 2023-03-09 21:00:00 +00:00
build Revert "Revert "Run integration tests and dev builds with race detection" (#15998)" (#16148) 2024-05-09 16:15:41 +00:00
changelog upgrade pulumi-yaml to 1.8.0 (#16325) 2024-06-05 10:58:07 +00:00
cmd/pulumi-test-language [chore] Update esc to v0.9.1 (#16322) 2024-06-05 06:22:01 +00:00
coverage Toward replacing MSBuild with make+bash on Windows (#8617) 2022-01-07 22:27:14 -05:00
developer-docs Typos fixed in implementers-guide.md file (#14288) 2023-10-23 15:26:55 +00:00
docker Cleanup of all docker operations since moving to pulumi/pulumi-docker-containers (#8252) 2021-10-26 20:37:33 +03:00
pkg [engine] Support remapping envvars for providers 2024-06-13 13:44:11 -07:00
proto RegisterProvider engine work (#16241) 2024-05-23 06:16:59 +00:00
scripts Update dotnet language host (#15955) 2024-04-16 15:07:33 +00:00
sdk [engine] Support remapping envvars for providers 2024-06-13 13:44:11 -07:00
tests upgrade pulumi-yaml to 1.8.0 (#16325) 2024-06-05 10:58:07 +00:00
.dockerignore Add a Dockerfile for the Pulumi CLI 2018-09-29 11:48:21 -07:00
.envrc.template fix: Allows for parallel pulumi programs to run in the node runtime 2022-10-13 07:15:25 -04:00
.gitignore [display] Enable WASM compilation (#16246) 2024-05-24 20:27:56 +00:00
.gitpod.yml Move `PULUMI_ROOT` to `$HOME/.pulumi-dev` (#8512) 2021-12-15 12:32:41 -08:00
.golangci.yml fix deprecation warnings in golangci-lint (#16068) 2024-04-26 13:01:58 +00:00
.goreleaser.yml [display] Enable WASM compilation (#16246) 2024-05-24 20:27:56 +00:00
.readthedocs.yaml Bump the RTD Python version down to 3.6. 2021-08-25 15:23:46 -07:00
.yarnrc Pass --network-concurrency 1 to yarn 2018-01-29 11:49:42 -08:00
CHANGELOG.md Changelog and go.mod updates for v3.118.0 (#16301) 2024-05-31 18:49:15 +00:00
CODE-OF-CONDUCT.md Update to "learning in public" on CoC 2021-07-06 11:03:19 -05:00
CONTRIBUTING.md chore: remove repetitive words (#15653) 2024-03-13 05:14:04 +00:00
LICENSE Remove appendix from LICENSE 2023-03-01 16:43:41 +00:00
Makefile fix name of Makefile target (#16272) 2024-05-28 08:41:57 +00:00
README.md Update language runtime versions in README.md (#15331) 2024-02-01 23:40:29 +00:00
codecov.yml sdk/go: Add pulumix subpackage (#13509) 2023-08-28 15:38:23 +00:00
youtube_preview_image.png Make youtube preview smaller 2020-05-15 09:52:23 -07:00

README.md

Slack GitHub Discussions NPM version Python version NuGet version GoDoc License Gitpod ready-to-code

Pulumi's Infrastructure as Code SDK is the easiest way to build and deploy infrastructure, of any architecture and on any cloud, using programming languages that you already know and love. Code and ship infrastructure faster with your favorite languages and tools, and embed IaC anywhere with Automation API.

Simply write code in your favorite language and Pulumi automatically provisions and manages your resources on AWS, Azure, Google Cloud Platform, Kubernetes, and 120+ providers using an infrastructure-as-code approach. Skip the YAML, and use standard language features like loops, functions, classes, and package management that you already know and love.

For example, create three web servers:

const aws = require("@pulumi/aws");
const sg = new aws.ec2.SecurityGroup("web-sg", {
    ingress: [{ protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] }],
});
for (let i = 0; i < 3; i++) {
    new aws.ec2.Instance(`web-${i}`, {
        ami: "ami-7172b611",
        instanceType: "t2.micro",
        vpcSecurityGroupIds: [sg.id],
        userData: `#!/bin/bash
            echo "Hello, World!" > index.html
            nohup python -m SimpleHTTPServer 80 &`,
    });
}

Or a simple serverless timer that archives Hacker News every day at 8:30AM:

const aws = require("@pulumi/aws");

const snapshots = new aws.dynamodb.Table("snapshots", {
    attributes: [{ name: "id", type: "S", }],
    hashKey: "id", billingMode: "PAY_PER_REQUEST",
});

aws.cloudwatch.onSchedule("daily-yc-snapshot", "cron(30 8 * * ? *)", () => {
    require("https").get("https://news.ycombinator.com", res => {
        let content = "";
        res.setEncoding("utf8");
        res.on("data", chunk => content += chunk);
        res.on("end", () => new aws.sdk.DynamoDB.DocumentClient().put({
            TableName: snapshots.name.get(),
            Item: { date: Date.now(), content },
        }).promise());
    }).end();
});

Many examples are available spanning containers, serverless, and infrastructure in pulumi/examples.

Pulumi is open source under the Apache 2.0 license, supports many languages and clouds, and is easy to extend. This repo contains the pulumi CLI, language SDKs, and core Pulumi engine, and individual libraries are in their own repos.

Welcome

  • Get Started with Pulumi: Deploy a simple application in AWS, Azure, Google Cloud, or Kubernetes using Pulumi.

  • Learn: Follow Pulumi learning pathways to learn best practices and architectural patterns through authentic examples.

  • Examples: Browse several examples across many languages, clouds, and scenarios including containers, serverless, and infrastructure.

  • Docs: Learn about Pulumi concepts, follow user-guides, and consult the reference documentation.

  • Registry: Find the Pulumi Package with the resources you need. Install the package directly into your project, browse the API documentation, and start building.

  • Pulumi Roadmap: Review the planned work for the upcoming quarter and a selected backlog of issues that are on our mind but not yet scheduled.

  • Community Slack: Join us in Pulumi Community Slack. All conversations and questions are welcome.

  • GitHub Discussions: Ask questions or share what you're building with Pulumi.

Getting Started

Watch the video

See the Get Started guide to quickly get started with Pulumi on your platform and cloud of choice.

Otherwise, the following steps demonstrate how to deploy your first Pulumi program, using AWS Serverless Lambdas, in minutes:

  1. Install:

    To install the latest Pulumi release, run the following (see full installation instructions for additional installation options):

    $ curl -fsSL https://get.pulumi.com/ | sh
    
  2. Create a Project:

    After installing, you can get started with the pulumi new command:

    $ mkdir pulumi-demo && cd pulumi-demo
    $ pulumi new hello-aws-javascript
    

    The new command offers templates for all languages and clouds. Run it without an argument and it'll prompt you with available projects. This command created an AWS Serverless Lambda project written in JavaScript.

  3. Deploy to the Cloud:

    Run pulumi up to get your code to the cloud:

    $ pulumi up
    

    This makes all cloud resources needed to run your code. Simply make edits to your project, and subsequent pulumi ups will compute the minimal diff to deploy your changes.

  4. Use Your Program:

    Now that your code is deployed, you can interact with it. In the above example, we can curl the endpoint:

    $ curl $(pulumi stack output url)
    
  5. Access the Logs:

    If you're using containers or functions, Pulumi's unified logging command will show all of your logs:

    $ pulumi logs -f
    
  6. Destroy your Resources:

    After you're done, you can remove all resources created by your program:

    $ pulumi destroy -y
    

To learn more, head over to pulumi.com for much more information, including tutorials, examples, and details of the core Pulumi CLI and programming model concepts.

Platform

Languages

Language Status Runtime Versions
JavaScript Stable Node.js Current, Active and Maintenance LTS versions
TypeScript Stable Node.js Current, Active and Maintenance LTS versions
Python Stable Python Supported versions
Go Stable Go Supported versions
.NET (C#/F#/VB.NET) Stable .NET Supported versions
Java Public Preview JDK 11+
YAML Stable n/a n/a

EOL Releases

The Pulumi CLI v1 and v2 are no longer supported. If you are not yet running v3, please consider migrating to v3 to continue getting the latest and greatest Pulumi has to offer! 💪

Clouds

Visit the Registry for the full list of supported cloud and infrastructure providers.

Contributing

Visit CONTRIBUTING.md for information on building Pulumi from source or contributing improvements.