Commit Graph

22 Commits

Author SHA1 Message Date
Will Jones 79e814fe0f
Don't lift dunder attributes on Python `Output`s ()
`Output`s are a central part of Pulumi programs. As well as tracking
dependencies between resources (e.g. that `A`'s input `x` comes from
`B`'s output `y`), they allow us to specify that some value will only be
available at a future point (e.g. after a resource has been created or
updated). This is typically done in each language by implementing
`Output`s using some asynchronous or future value, such as NodeJS's
`Promise`. In Python, we use `asyncio` `Task`s.

In order to make `Output`s ergonomic to use, we implement "lifting" of
properties in languages that support it. Suppose for instance we have an
object `c` that is an instance of the following class `C`:

```python
class C:
    x: str
    y: int

    def __init__(self, x: str, y: int) -> None:
        self.x = x
        self.y = y

c = C("x", 42)
```

Because `c: C`, we have that `c.x == "x"` and `c.y == 42` as we might
expect. Consider though some output property of a resource that produces
a `C`. This property will be of type `Output[C]`, since the value of
type `C` won't be available until the resource has been set up by Pulumi
as part of program execution. If we want to pass that output's `x` value
to some other resource, we might have to write:

```python
r1 = ... # r1 has a property c: Output[C]

r2 = R("r", RArgs(x=r1.c.apply(lambda cc: cc.x)))
```

Observe that we have to use `apply` to unwrap the output and access the
property inside. This is tedious and ugly, and exactly the problem
lifting solves. Lifting allows us to write `r1.c.x` and have it be
implemented as `r1.c.apply(lambda cc: cc.x)` under the hood. In Python,
this is achieved using Python's `__getattr__` "dunder" method ("dunder"
being short for "double underscore", the convention adopted for
"special" identifiers in Python). `__getattr__` allows us to perform
dynamic property lookup on an object, which in the case of `Output` we
use to delegate property access to the underlying value using `apply`.

This works really well and contributes significantly to Pulumi programs
being easy to read and write in Python. Unfortunately, it has a flaw:
`__getattr__` is also used to check whether attributes exist on an
object (using `hasattr`), and thus has a contract whereby it is
_synchronously_ expected to raise an `AttributeError` if an attribute
does not exist. In returning an _asynchronous_ task (that may _later_
resolve to an `AttributeError`), `Output` is in violation of this
contract. This means that code which calls e.g. `hasattr(r1.c, "z")`
will yield a future task that if resolved, will blow up with an
`AttributeError`.

Historically, this hasn't really been a problem. With the advent of
https://github.com/pulumi/pulumi/pull/15744 and subsequent
improvements/fixes, however, this is now an issue, since we explicitly
await all outstanding outputs before program termination. In the example
above, for instance, the orphaned task will be forced at the end of
program execution. The `AttributeError` will rear its head and Pulumi
will exit with an error and stack trace.

This commit implements a fix proportional to the cases where this
appears to be a problem. Namely, libraries such as Pydantic that use
dunder attributes in their implementation and check for the presence of
these attributes when executing. When `__getattr__` is called with a
dunder attribute, we synchronously `raise AttributeError` rather than
lifting it. There are (we believe) no cases where this would affect a
Pulumi-generated SDK (since dunder names aren't generally used for
public resource properties) and the dunder properties on the `Output`
itself (e.g. `__dict__`, etc.) will continue to work since their
resolution will succeed normally and a call to `__getattr__` will thus
not be made.

Fixes 
2024-06-22 00:04:34 +00:00
Fraser Waters f0085de0c7
Handful of mypy fixes in the python test code () 2024-05-02 17:46:52 +00:00
Fraser Waters 48dbd6c596
Use black to format lib/test ()
Test code should be formatted and linted the same as library code. This
is the first step of that, simply including ./lib/test to the folder
that the black formatter runs on.
2024-04-23 08:29:58 +00:00
Fraser Waters 50eacf3c2f
Add a test and fix that we don't wait for tasks, just outputs ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/6762.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-26 14:07:37 +00:00
Fraser Waters bba69e2f38 Make `Output.from_input` recurse into tuples.
Fixes https://github.com/pulumi/pulumi/issues/6635.
2023-07-27 10:13:40 +01:00
bors[bot] 772a057ee1
Merge
13463: Fix links to outputs docs r=cnunciato a=cnunciato

Fixes some broken links pointing to the Inputs & Outputs docs.

Fixes .

Co-authored-by: Christian Nunciato <chris@nunciato.org>
2023-07-13 12:34:13 +00:00
Fraser Waters 8db3087d2b Make pythons RPCManager a context variable 2023-07-13 09:59:12 +01:00
Christian Nunciato 202fbd946f Fix links outputs docs 2023-07-12 14:59:07 -07:00
Fraser Waters 5685291373 Handle Outputs in dict keys passed to from_input 2023-01-27 04:51:05 +00:00
Fraser Waters ecaee6eeba Fix a TypeError in Output.from_input 2023-01-13 18:14:50 +00:00
Fraser Waters 9961796909 Add json_loads to python sdk
Partner method to json_dumps.
2022-12-28 20:24:07 +00:00
Fraser Waters 2ffdcd0322 Add json_dumps to python sdk 2022-12-14 13:53:15 +00:00
Fraser Waters c63813e0bc Add Output.format to python SDK 2022-10-04 22:45:49 +01:00
Kyle Pitzen 122d7c25b9
fix(sdk/python): Makes SETTINGS attributes context-aware () 2022-08-23 13:45:37 -04:00
Guinevere Saenger bbbab1c9a1
Correct Python syntax in help text ()
* Correct Python syntax in help text

Replace arrow with colon, as is correct for Python

* fix test
2022-08-15 18:03:15 +01:00
Anton Tayanovskyy ef97271249
Fix Python SDK apply causing program hangs ()
* Fix Python SDK apply causing program hangs

* PR feedback
2022-08-15 10:43:10 -04:00
Fraser Waters cd2a0b7934
Fix typo in __str__ error message ()
* Fix typo in __str__ error message

* Fix test
2022-06-24 22:45:19 +01:00
Fraser Waters 6bbe7f6071
Change str(output) to return a warning message ()
* Change str(output) to return a warning message

* Update CHANGELOG
2022-06-13 16:08:13 +01:00
Luke Hoban 965d23ba2a
[sdk/python] Ensure Output objects are not iterable ()
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop.  To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).

Fixes .
2021-06-15 09:25:24 +10:00
Luke Hoban 26e252f241
Ensure `Output.from_input({})` returns `{}` instead of `[]` ()
Fixes .
2021-06-09 19:48:10 +10:00
Luke Hoban bd6410e2fb
[sdk/python] Avoid exponential complexity for `from_input`/`all` ()
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.

Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.

Also simplify the implementation to aid readability.

Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes .
2021-06-01 13:11:22 +10:00
Justin Van Patten 2779de38ea
[sdk/python] from_input: Unwrap nested outputs in input types ()
`Output.from_input` deeply unwraps nested output values in dicts and lists, but doesn't currently do that for the more recently added "input types" (i.e. args classes). This leads to errors when using args classes with output values with `Provider` resources, which uses `Output.from_input` on each input property and then serializes the value to JSON in an `apply`. This changes fixes `Output.from_input` to recurse into values within args classes to properly unwrap any nested outputs.
2021-01-29 15:44:00 -08:00