Commit Graph

74 Commits

Author SHA1 Message Date
Ian Wahbe 9a06694676
Move assets and archives to their own package ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR is motivated by https://github.com/pulumi/pulumi/pull/15145.

`resource.*` should be built on top of `property.Value`,[^1] which means
that `resource`
needs to be able to import `property.Value`, and so `property` cannot
import
`resource`. Since Assets and Archives are both types of properties, they
must be moved out
of `resource`.

[^1]: For example:
a1d686227c/sdk/go/common/resource/resource_state.go (L35-L36)

## Open Question

This PR moves them to their own sub-folders in `resource`. Should
`asset` and `archive`
live somewhere more high level, like `sdk/go/property/{asset,archive}`?

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-25 20:39:31 +00:00
Thomas Gummerer baecc85eaf
turn on the golangci-lint exhaustive linter ()
Turn on the golangci-lint exhaustive linter.  This is the first step
towards catching more missing cases during development rather than
in tests, or in production.

This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.

I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
 
Fixes  

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 16:50:41 +00:00
Fraser Waters 88b256d5b1
Move some asset code to pkg ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by https://github.com/pulumi/pulumi/pull/15157.

Trying to move as much of the code in assets and archives to pkg.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 11:30:37 +00:00
Fraser Waters 54c956af6d
Send all events to the engine event stream ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Currently the engine skips sending some resource events to the event
stream. Currently that's any "RemovePendingDelete" steps and anything to
do with default providers.

This was added so that we wouldn't display "internal implemntation
details" like default providers to the user in the tree or diff views.

However we wanted to use the engine event stream to support generating
an import file from preview deployments (make an import for every
resource that needs to be created). This mostly works except for the
imports we also need to know some of the provider details, and while the
event stream will tell us about explicit providers the skipping of
default providers means we can't get their information in the import
generater code.

So this moves this filtering out of the engine and to the display logic
instead. We still leave it up to the engine to mark what events it
considers "internal" but they're always sent to the event stream.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works - This shouldn't change anything user visible so should be
covered by existing tests
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change - No user visible
change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-20 21:55:59 +00:00
Fraser Waters 6f21cac6f3
Make `engine.NewEvent` type safe ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Small refactor I noticed while writing a test with engine events. We
always had to call `NewEvent` with the tag and payload value for an
event and these _had_ to match up else the engine panics. But we can
just pass the payload and type switch to work out the tag. Means one
less parameter to pass to `NewEvent` and pretty much no chance of it
going wrong. To ensure there's really no chance I've added a generic
union type so you can only pass payload types to this method now.

Cancel had to be handled separately because it doesn't have a payload
type, it's just nil.
2023-11-16 16:54:03 +00:00
Thomas Gummerer 529ea588ea
add output for running policy packs ()
# Description

🚧 

This is a work in progress attempting to add some output for when policy
packs are being loaded/compiled (which I realized was the actual reason
for the slowdown, not that they were being run already). There's still
some pretty obvious code quality issues as I've been throwing this
together, and the URN's are not constructed properly yet either, so some
of the output is less helpful than it could be. I'm mainly opening this
to get some early feedback on whether this is the appropriate place to
add the output, or if there are better alternatives.

Also including a gif here of what this looks like. In particular the
Name here needs to be improved, and the "Plan" might want to say
"loading/compiling" or something similar instead?


![policy-pack](https://github.com/pulumi/pulumi/assets/191004/85cca146-d740-4e4c-96dc-cc26dca6bb90)

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-15 11:19:31 +00:00
Joe Duffy 96a9a77167
Policy remediations feature ()
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
2023-10-09 18:31:17 +00:00
Fraser Waters 0ab7f9e8bd
Deepcopy event payloads on construction _not_ on access ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14036.

Not sure this is perfect (it's pretty hard to reason about mutation of
the whole system given what Go gives us to work with), but it seems
better.

We copy event payloads on construction, which should be synchronous with
step execution, rather than on access which could be in parallel with
the step executor or deployment executor mutating resource state.

Just copying on construction flagged up that the deployment executor and
step executor both race to modify resource state when the deployment
executor starts scheduling deletes. We add a synchronisation point
between the two parts in this change so that the deployment executor
waits for the step executor to go quite, and keeps it quite while it
computes the deletes, once it starts scheduling them the step executor
can run again.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-02 14:15:10 +00:00
Fraser Waters 4ebf61f896
Move sdk/go/common/display to /pkg/display ()
Similar to how https://github.com/pulumi/pulumi/pull/13953 moves some
code from sdk/go/common to /pkg. This display code is only used in /pkg,
another simple reduction of what's in sdk/go/common.
2023-09-18 11:01:28 +00:00
Pat Gavlin af8f3234de [apitype] Add RetainOnDelete to StepEventStateMetadata
This flag is necessary for parity between the CLI's event renderer and
the renderer used by the Cloud.
2023-06-26 12:50:39 -07:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta 021a48e495
pkg/engine: Prefer contract.Assertf over Assert
Incremental step towards 

Migrates uses of contract.{Assert, AssertNoError, Require} in pkg/engine
to `*f` variants so that we're required to provide more error context.

Refs 
2023-02-15 10:23:38 -08:00
Anton Tayanovskyy 1bc10eb75d
Do not panic when sending events to closed emitter ()
* Do not panic when sending events to closed emitter

* Mark unit tests Parallel
2022-08-25 12:43:10 -04:00
Richard Shade a5687d16a9
Moving previewDigest to sdk/go/common/display, and exporting it. ()
* Moving previewDigest and exporting it, closes 

* Moving previewDigest and exporting it, closes 

* Updating changelog-pending

* Go Mod Tidy

* replacing to local

* more go.mod changes

* reseting go mod

* full move

* Fixing golint

* No go.mod changes needed
2022-06-27 09:08:06 -05:00
Pat Gavlin 251a75b971
[events] Retain `Secret` values. ()
Instead of replacing `Secret` values with the text `[secret]`, retain
the `Secret` value and replace its element with the text `[secret]`.
This allows downstream consumers (e.g. the CLI's diff renderer) to
understand that the contents of the value is secret if necessary.
2022-04-06 11:01:06 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Levi Blackstone f6ec691544
Fix malformed resource value bug ()
* Fix resource-ref-as-ID marshaling. ()

This reapplies 2f0dba23ab.

* Fix malformed resource value bug

PR  introduced a bug by marshaling resource
ids as PropertyValues, but not handling that case on
the unmarshaling side. The previous code assumed
that the id was a simple string value. This bug prevents
any stack update operations (preview, update, destroy,
refresh). Since this change was already
released, we must now handle both cases in the
unmarshaling code.

* Add resource ref unit tests for the Go SDK. ()

This reapplies 3d505912b8.

Co-authored-by: Pat Gavlin <pat@pulumi.com>
2021-01-21 16:40:27 -07:00
Paul Stack 9322072146
Revert "Fix resource-ref-as-ID marshaling. ()" () 2021-01-21 20:37:40 +00:00
Pat Gavlin 2f0dba23ab
Fix resource-ref-as-ID marshaling. ()
When marshaling a resource reference as its ID (i.e. when
opts.KeepResources is false, as it will be in the case of downlevel SDKs
and resource providers), we must take care to marshal/unmarshal an empty
ID as the unknown property value.

This includes the following changes to the resource ref APIs:

- Bifurcate resource reference creation into two methods: one for
  creating references to custom resources and one for creating
  references to component resources.
- Store the ID in a resource reference as a PropertyValue s.t. it can be
  computed.
- Add a helper method for retrieving the ID as a string + an indicator of
  whether or not the reference has an ID.

Fixes .
2021-01-15 16:06:57 -08:00
Pat Gavlin f0951ce650
Rename engine.plan to engine.deployment. ()
This name better suits the semantics of the type, and aligns with the
rename of deploy.Plan to deploy.Deployment. These changes also refactor
the `update` method s.t. previews and updates are more consistent in
their behavior (e.g. duration and resource changes are now reported for
both, incl. on error paths).
2020-11-18 11:16:30 -08:00
Pat Gavlin 3d2e31289a
Add support for serialized resource references. ()
Resources are serialized as their URN, ID, and package version. Each
Pulumi package is expected to register itself with the SDK. The package
will be invoked to construct appropriate instances of rehydrated
resources. Packages are distinguished by their name and their version.

This is the foundation of cross-process resources.

Related to .

Co-authored-by: Mikhail Shilkov <github@mikhail.io>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Co-authored-by: Levi Blackstone <levi@pulumi.com>
2020-10-27 10:12:12 -07:00
Pat Gavlin 418e2291a2
Protect against engine event mutation. ()
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
2020-07-16 23:52:31 -07:00
Paul Stack 0824fc9a8b
Revert "Narrow a few interfaces." () 2020-07-09 15:19:12 +01:00
Pat Gavlin 45d2fa95d6
Narrow a few interfaces. ()
- Remove `Info` from `Source`. This method was not used.
- Remove `Stack` from `EvalSource`. This method was not used.
- Remove `Type` and `URN` from `Step`. These values are available via
  `Res().URN.Type()` and `Res().URN`, respectively. This removes the
  possibility of inconsistencies between the type, URN, and state of the
  resource associated with a `Step`.
- Remove URN from StepEventMetadata.
2020-07-01 15:32:50 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle d3f5bbce48 go fmt 2020-03-18 17:27:02 -07:00
evanboyle c3f6ae2451 move pkg/util/logging -> sdk/go/common/util/logging 2020-03-18 15:34:58 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle f754b486b8 move pkg/resource/config -> sdk/go/common/resource/config 2020-03-18 15:03:37 -07:00
evanboyle 67cb405c93 move pkg/apitype -> sdk/common/apitype 2020-03-18 15:00:30 -07:00
evanboyle 70f386a967 move pkg/tokens -> sdk/go/common/tokens 2020-03-18 14:49:56 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle dfab571aac move pkg/resource/plugin -> sdk/go/common/resource/plugin 2020-03-18 14:26:24 -07:00
evanboyle fba783caf9 move pkg/resource -> sdk/go/common/resource, but leave nested resource packages 2020-03-18 13:36:19 -07:00
Justin Van Patten c08714ffb4
Support lists and maps in config ()
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).

For example:

```yaml
config:
  proj:blah:
  - a
  - b
  - c
  proj:hello: world
  proj:outer:
    inner: value
  proj:servers:
  - port: 80
```

While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.

As always, you can specify single values with:

```shell
$ pulumi config set hello world
```

Which results in the following YAML:

```yaml
proj:hello world
```

And single value secrets via:

```shell
$ pulumi config set --secret token shhh
```

Which results in the following YAML:

```yaml
proj:token:
  secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```

Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:

```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```

Which results in:

```yaml
proj:names
- a
- b
- c
```

Values can be obtained similarly:

```shell
$ pulumi config get --path names[1]
b
```

Or setting values in a map:

```shell
$ pulumi config set --path outer.inner value
```

Which results in:

```yaml
proj:outer:
  inner: value
```

Of course, setting values in nested structures is supported:

```shell
$ pulumi config set --path servers[0].port 80
```

Which results in:

```yaml
proj:servers:
- port: 80
```

If you want to include a period in the name of a property, it can be
specified as:

```
$ pulumi config set --path 'nested["foo.bar"]' baz
```

Which results in:

```yaml
proj:nested:
  foo.bar: baz
```

Examples of valid paths:

- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'

Note: paths that contain quotes can be surrounded by single quotes.

When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.

Secure values are supported in lists/maps as well:

```shell
$ pulumi config set --path --secret tokens[0] shh
```

Will result in:

```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```

Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:

```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```

**Accessing config values from the command line with JSON**

```shell
$ pulumi config --json
```

Will output:

```json
{
  "proj:hello": {
    "value": "world",
    "secret": false,
    "object": false
  },
  "proj:names": {
    "value": "[\"a\",\"b\",\"c\"]",
    "secret": false,
    "object": true,
    "objectValue": [
      "a",
      "b",
      "c"
    ]
  },
  "proj:nested": {
    "value": "{\"foo.bar\":\"baz\"}",
    "secret": false,
    "object": true,
    "objectValue": {
      "foo.bar": "baz"
    }
  },
  "proj:outer": {
    "value": "{\"inner\":\"value\"}",
    "secret": false,
    "object": true,
    "objectValue": {
      "inner": "value"
    }
  },
  "proj:servers": {
    "value": "[{\"port\":80}]",
    "secret": false,
    "object": true,
    "objectValue": [
      {
        "port": 80
      }
    ]
  },
  "proj:token": {
    "secret": true,
    "object": false
  },
  "proj:tokens": {
    "secret": true,
    "object": true
  }
}
```

If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.

If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.

**Accessing config values from Pulumi programs**

Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:

```typescript
import * as pulumi from "@pulumi/pulumi";

interface Server {
    port: number;
}

const config = new pulumi.Config();

const names = config.requireObject<string[]>("names");
for (const n of names) {
    console.log(n);
}

const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
    console.log(s.port);
}
```
2019-11-01 13:41:27 -07:00
Alex Clemmer 628abb5636 Fix bug causing `pulumi query` to hang forever
`engine.Query` queues up a bunch of `defer` functions that (among other
things) wait for various resources to clean themselves up.

In `query`, we have a "naive" (read: bad) `eventEmitter`, whose `done`
channel is set to `nil`. Therefore, the relevent `defer` waits for an
event on a `nil` channel which in Go will famously simply hang forever.

This commit will correct this by setting this channel appropriately, so
that it signals appropriately when it's done.
2019-10-23 15:14:56 -07:00
Alex Clemmer 9c16485152 Don't require a stack to run `pulumi query` 2019-10-23 15:14:56 -07:00
Pat Gavlin 730fe8617e
Buffer events. ()
This avoids unnecessary blocking inside pre/post-step callbacks if the
reader on the other side of the event channel is slow.

We do not use a buffered channel in the event pipe because it is
empirically less likely that the goroutine reading from a buffered
channel will be scheduled when new data is placed in the channel. In the
case of our event system in which events will not be delivered to the
service and display until the copying goroutine is scheduled, this can
lead to unacceptable delay between the send of the original event and
its output.
2019-10-15 15:47:40 -07:00
Alex Clemmer 429bde332b Print PolicyPacks run as part of update summary
Fixes .
2019-09-30 14:49:21 -07:00
Alex Clemmer fc80eaaa3d Implement `GetAnalyzerInfo` in analyzer plugin 2019-07-16 00:58:33 -07:00
Pat Gavlin 6e5c4a38d8
Defer all diffs to resource providers. ()
Thse changes make a subtle but critical adjustment to the process the
Pulumi engine uses to determine whether or not a difference exists
between a resource's actual and desired states, and adjusts the way this
difference is calculated and displayed accordingly.

Today, the Pulumi engine get the first chance to decide whether or not
there is a difference between a resource's actual and desired states. It
does this by comparing the current set of inputs for a resource (i.e.
the inputs from the running Pulumi program) with the last set of inputs
used to update the resource. If there is no difference between the old
and new inputs, the engine decides that no change is necessary without
consulting the resource's provider. Only if there are changes does the
engine consult the resource's provider for more information about the
difference. This can be problematic for a number of reasons:

- Not all providers do input-input comparison; some do input-state
  comparison
- Not all providers are able to update the last deployed set of inputs
  when performing a refresh
- Some providers--either intentionally or due to bugs--may see changes
  in resources whose inputs have not changed

All of these situations are confusing at the very least, and the first
is problematic with respect to correctness. Furthermore, the display
code only renders diffs it observes rather than rendering the diffs
observed by the provider, which can obscure the actual changes detected
at runtime.

These changes address both of these issues:
- Rather than comparing the current inputs against the last inputs
  before calling a resource provider's Diff function, the engine calls
  the Diff function in all cases.
- Providers may now return a list of properties that differ between the
  requested and actual state and the way in which they differ. This
  information will then be used by the CLI to render the diff
  appropriately. A provider may also indicate that a particular diff is
  between old and new inputs rather than old state and new inputs.

Fixes .
2019-07-01 12:34:19 -07:00
Alex Clemmer 0fc4bc7885 Remove policy ID from policy API 2019-06-13 17:39:30 -07:00
Alex Clemmer 8b7d329c69 Use Analyzer PB in analyzer code 2019-06-13 16:04:13 -07:00
Matt Ellis 529645194e Track secrets inside the engine
A new `Secret` property value is introduced, and plumbed across the
engine.

- When Unmarshalling properties /from/ RPC calls, we instruct the
  marshaller to retain secrets, since we now understand them in the
  rest of the engine.

- When Marshalling properties /to/ RPC calls, we use or tracked data
  to understand if the other side of the connection can accept
  secrets. If they can, we marshall them in a similar manner to assets
  where we have a special object with a signiture specific for secrets
  and an underlying value (which is the /plaintext/ value). In cases
  where the other end of the connection does not understand secrets,
  we just drop the metadata and marshal the underlying value as we
  normally would.

- Any secrets that are passed across the engine events boundary are
  presently passed as just `[secret]`.

- When persisting secret values as part of a deployment, we use a rich
  object so that we can track the value is a secret, but right now the
  underlying value is not actually encrypted.
2019-05-10 17:07:52 -07:00
joeduffy 250bcb9751 Add a --json flag to the preview command
This change adds a --json flag to the preview command, enabling
basic JSON serialization of preview plans. This effectively flattens
the engine event stream into a preview structure that contains a list
of steps, diagnostics, and summary information. Each step contains
the deep serialization of resource state, in addition to metadata about
the step, such as what kind of operation it entails.

This is a partial implementation of . In particular,
we only support --json on the `preview` command itself, and not `up`,
meaning that it isn't possible to serialize the result of an actual
deployment yet (thereby limiting what you can do with outputs, etc).
2019-04-25 17:36:31 -07:00
Pat Gavlin 4b33a45561
Filter diff keys based on provider info ()
If a provider returns information about the top-level properties that
differ, use those keys to filter the diffs that are rendered to the
user.

Fixes .
2019-03-06 16:41:19 -08:00
Alex Clemmer dea68b8b37 Implement status sinks
This commit reverts most of  and replaces it with functionally
identical logic, using the notion of status message-specific sinks.

In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.

The original commit message in that PR was:

> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
2018-08-31 15:56:53 -07:00
Alex Clemmer 9e58fd1aaa Revert "Plumb `LogRequest.IsStatus` through the logging subsystem"
This reverts commit 3066cbcbd7.
2018-08-31 15:56:53 -07:00
Alex Clemmer 3066cbcbd7 Plumb `LogRequest.IsStatus` through the logging subsystem 2018-08-30 17:17:20 -07:00
Alex Clemmer 2fa98a8dad Generate empty update steps for partial failures
This commit will greatly improve the experience of dealing with partial
failures by simply re-trying to initialize the relevant resources on
every subsequent `pulumi up`, instead of printing a list of reasons the
resource had previously failed to initialize.

As motivation, consider our behavior in the following common, painful
scenario:

  * The user creates a `Service` and a `Deployment`.
  * The `Pod`s in the `Deployment` fail to become live. This causes the
    `Service` to fail, since it does not target any live `Pod`s.
  * The user fixes the `Deployment`. A run of `pulumi up` sees the
    `Pod`s successfully initialize.
  * Users will expect that the `Service` is now in a state of success,
    as the `Pod`s it targets are alive. But, because we don't update the
    `Service` by default, it perpetually exists in a state of error.
  * The user is now required to change some trivial feature of the
    `Service` just to trigger an update, so that we can see it succeed.

There are many situations like this. Another very common one is waiting
for test `Pod`s that are meant to successfully complete when some object
becomes live.

By triggering an empty update step for all resources that have any
initialization errors, we avoid all problems like this.

This commit will implement this empty-update semantics for partial
failures, as well as fix the display UX to correctly render the diff in
these cases.
2018-08-28 18:00:35 -07:00