`Output`s are a central part of Pulumi programs. As well as tracking
dependencies between resources (e.g. that `A`'s input `x` comes from
`B`'s output `y`), they allow us to specify that some value will only be
available at a future point (e.g. after a resource has been created or
updated). This is typically done in each language by implementing
`Output`s using some asynchronous or future value, such as NodeJS's
`Promise`. In Python, we use `asyncio` `Task`s.
In order to make `Output`s ergonomic to use, we implement "lifting" of
properties in languages that support it. Suppose for instance we have an
object `c` that is an instance of the following class `C`:
```python
class C:
x: str
y: int
def __init__(self, x: str, y: int) -> None:
self.x = x
self.y = y
c = C("x", 42)
```
Because `c: C`, we have that `c.x == "x"` and `c.y == 42` as we might
expect. Consider though some output property of a resource that produces
a `C`. This property will be of type `Output[C]`, since the value of
type `C` won't be available until the resource has been set up by Pulumi
as part of program execution. If we want to pass that output's `x` value
to some other resource, we might have to write:
```python
r1 = ... # r1 has a property c: Output[C]
r2 = R("r", RArgs(x=r1.c.apply(lambda cc: cc.x)))
```
Observe that we have to use `apply` to unwrap the output and access the
property inside. This is tedious and ugly, and exactly the problem
lifting solves. Lifting allows us to write `r1.c.x` and have it be
implemented as `r1.c.apply(lambda cc: cc.x)` under the hood. In Python,
this is achieved using Python's `__getattr__` "dunder" method ("dunder"
being short for "double underscore", the convention adopted for
"special" identifiers in Python). `__getattr__` allows us to perform
dynamic property lookup on an object, which in the case of `Output` we
use to delegate property access to the underlying value using `apply`.
This works really well and contributes significantly to Pulumi programs
being easy to read and write in Python. Unfortunately, it has a flaw:
`__getattr__` is also used to check whether attributes exist on an
object (using `hasattr`), and thus has a contract whereby it is
_synchronously_ expected to raise an `AttributeError` if an attribute
does not exist. In returning an _asynchronous_ task (that may _later_
resolve to an `AttributeError`), `Output` is in violation of this
contract. This means that code which calls e.g. `hasattr(r1.c, "z")`
will yield a future task that if resolved, will blow up with an
`AttributeError`.
Historically, this hasn't really been a problem. With the advent of
https://github.com/pulumi/pulumi/pull/15744 and subsequent
improvements/fixes, however, this is now an issue, since we explicitly
await all outstanding outputs before program termination. In the example
above, for instance, the orphaned task will be forced at the end of
program execution. The `AttributeError` will rear its head and Pulumi
will exit with an error and stack trace.
This commit implements a fix proportional to the cases where this
appears to be a problem. Namely, libraries such as Pydantic that use
dunder attributes in their implementation and check for the presence of
these attributes when executing. When `__getattr__` is called with a
dunder attribute, we synchronously `raise AttributeError` rather than
lifting it. There are (we believe) no cases where this would affect a
Pulumi-generated SDK (since dunder names aren't generally used for
public resource properties) and the dunder properties on the `Output`
itself (e.g. `__dict__`, etc.) will continue to work since their
resolution will succeed normally and a call to `__getattr__` will thus
not be made.
Fixes#16399
Test code should be formatted and linted the same as library code. This
is the first step of that, simply including ./lib/test to the folder
that the black formatter runs on.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/6762.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
13463: Fix links to outputs docs r=cnunciato a=cnunciato
Fixes some broken links pointing to the Inputs & Outputs docs.
Fixes#13461.
Co-authored-by: Christian Nunciato <chris@nunciato.org>
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop. To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).
Fixes#5028.
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.
Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.
Also simplify the implementation to aid readability.
Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes#3987.
`Output.from_input` deeply unwraps nested output values in dicts and lists, but doesn't currently do that for the more recently added "input types" (i.e. args classes). This leads to errors when using args classes with output values with `Provider` resources, which uses `Output.from_input` on each input property and then serializes the value to JSON in an `apply`. This changes fixes `Output.from_input` to recurse into values within args classes to properly unwrap any nested outputs.