pulumi

History

Justin Van Patten 472e6bdd50 Allow Python dynamic providers to capture secrets (#15864 ) As of https://github.com/pulumi/pulumi/pull/13315 (which shipped in [v3.75.0](https://github.com/pulumi/pulumi/releases/tag/v3.75.0)), `__provider` (the serialized provider string) is _always_ set to a secret in the state. This can lead to poor performance when there are a lot of dynamic resources and the serialized provider does not actually have any secrets. This change does two things: 1. Provides a way to opt-out of always serializing the provider as a secret. 2. Allows Outputs to be captured during serialization of the provider, which wasn't previously possible. ## 1. Opt-out of always serializing as secret A new attribute, `serialize_as_secret_always`, can be set on a subclass of `ResourceProvider` to opt-out of always serializing the provider as a secret. If you know you don't have any secrets being serialized into the provider and want to avoid the encryption overhead, you can set this attribute to `False`. ```python class MyProvider(ResourceProvider): serialize_as_secret_always = False def create(self, props): # Doesn't have any secrets that need encrypting ... ``` ## 2. Allow Outputs to be captured If you currently try to capture an `Output`, it fails when serializing the provider with: ``` TypeError: cannot pickle '_asyncio.Future' object ``` This change allows Outputs to be captured/serialized for dynamic providers, including secret Outputs. This aligns Python dynamic providers with [Node.js](https://github.com/pulumi/pulumi/pull/13329). ```python import pulumi from pulumi.dynamic import CreateResult, Resource, ResourceProvider config = pulumi.Config() password = config.require_secret("password") class SimpleProvider(ResourceProvider): def create(self, props): # Need to use `password.get()` to get the underlying value of the secret from within the serialized code. # This simulates using this as a credential to talk to an external system. return CreateResult("0", { "authenticated": "200" if password.get() == "s3cret" else "401" }) class SimpleResource(Resource): authenticated: pulumi.Output[str] def __init__(self, name): super().__init__(SimpleProvider(), name, { "authenticated": None }) r = SimpleResource("foo") pulumi.export("out", r.authenticated) ``` Note: In the above example, we didn't have to specify `serialize_as_secret_always` since the default behavior is to always serialize the provider as a secret. If we wanted to, we could have specified `serialize_as_secret_always = False` and it still would have serialized the provider as a secret, since it captured `password` which is a secret Output. If `serialize_as_secret_always = False` was specified and no secrets were captured, then the provider would not be serialized as a secret. We plan to recommend this approach to capturing secrets for both Node.js and Python dynamic providers after this change. Fixes #15539	2024-08-02 23:51:52 +00:00
..
README.md	Allow Python dynamic providers to capture secrets (#15864 )	2024-08-02 23:51:52 +00:00

Justin Van Patten 472e6bdd50

Allow Python dynamic providers to capture secrets (#15864 )

As of https://github.com/pulumi/pulumi/pull/13315 (which shipped in
[v3.75.0](https://github.com/pulumi/pulumi/releases/tag/v3.75.0)),
`__provider` (the serialized provider string) is _always_ set to a
secret in the state. This can lead to poor performance when there are a
lot of dynamic resources and the serialized provider does not actually
have any secrets.

This change does two things:
1. Provides a way to opt-out of always serializing the provider as a
secret.
2. Allows Outputs to be captured during serialization of the provider,
which wasn't previously possible.

## 1. Opt-out of always serializing as secret

A new attribute, `serialize_as_secret_always`, can be set on a subclass
of `ResourceProvider` to opt-out of always serializing the provider as a
secret. If you know you don't have any secrets being serialized into the
provider and want to avoid the encryption overhead, you can set this
attribute to `False`.

```python
class MyProvider(ResourceProvider):
    serialize_as_secret_always = False

    def create(self, props):
        # Doesn't have any secrets that need encrypting
        ...
```

## 2. Allow Outputs to be captured

If you currently try to capture an `Output`, it fails when serializing
the provider with:

```
TypeError: cannot pickle '_asyncio.Future' object
```

This change allows Outputs to be captured/serialized for dynamic
providers, including secret Outputs.

This aligns Python dynamic providers with
[Node.js](https://github.com/pulumi/pulumi/pull/13329).

```python
import pulumi
from pulumi.dynamic import CreateResult, Resource, ResourceProvider

config = pulumi.Config()
password = config.require_secret("password")

class SimpleProvider(ResourceProvider):
    def create(self, props):
        # Need to use `password.get()` to get the underlying value of the secret from within the serialized code.
        # This simulates using this as a credential to talk to an external system.
        return CreateResult("0", { "authenticated": "200" if password.get() == "s3cret" else "401" })

class SimpleResource(Resource):
    authenticated: pulumi.Output[str]

    def __init__(self, name):
        super().__init__(SimpleProvider(), name, { "authenticated": None })


r = SimpleResource("foo")
pulumi.export("out", r.authenticated)
```

Note: In the above example, we didn't have to specify
`serialize_as_secret_always` since the default behavior is to always
serialize the provider as a secret. If we wanted to, we could have
specified `serialize_as_secret_always = False` and it still would have
serialized the provider as a secret, since it captured `password` which
is a secret Output. If `serialize_as_secret_always = False` was
specified and no secrets were captured, then the provider would not be
serialized as a secret.

We plan to recommend this approach to capturing secrets for both Node.js
and Python dynamic providers after this change.

Fixes #15539

2024-08-02 23:51:52 +00:00

README.md

Allow Python dynamic providers to capture secrets (#15864 )

2024-08-02 23:51:52 +00:00

README.md

Intentionally make no changes.