mirror of https://github.com/pulumi/pulumi.git
fef43d10cf
### Description This PR extends program-gen to start emitting deferred outputs for references of mutually dependant components in PCL for nodejs, python and dotnet. Addresses the following: - [[TF circular references] .NET program-gem emitting DeferredOutput from mutually dependant components](https://github.com/pulumi/pulumi/issues/17789) - [[TF circular references] NodeJS program-gem emitting DeferredOutput from mutually dependant components](https://github.com/pulumi/pulumi/issues/17790) - [[TF circular references] python program-gem emitting DeferredOutput from mutually dependant components](https://github.com/pulumi/pulumi/issues/17857) The main idea when extracting references to mutually dependant components is to replace with variables that are defined as deferred outputs and later in the program (after the declaration of the dependant component) we resolve the value of that deferred variable. The `pcl.ExtractDeferredOutputVariables` utility function contains the core logic for this implementation and it is what each language generator uses Example in PCL: ```tf component "first" "./first" { passwordLength = second.passwordLength } component "second" "./second" { petName = first.petName } ``` ### Generated TypeScript ```typescript const [secondPasswordLength, resolveSecondPasswordLength] = pulumi.deferredOutput<number>(); const first = new First("first", {passwordLength: secondPasswordLength}); const second = new Second("second", {petName: first.petName}); resolveSecondPasswordLength(second.passwordLength); ``` ### Generated Python ```python second_password_length, resolve_second_password_length = pulumi.deferred_output() first = First("first", { 'passwordLength': second_password_length}) second = Second("second", { 'petName': first.pet_name}) resolve_second_password_length(second.password_length); ``` ### Generated C# ```csharp var secondPasswordLength = new Pulumi.DeferredOutput<int>(); var first = new Components.First("first", new() { PasswordLength = secondPasswordLength.Output, }); var second = new Components.Second("second", new() { PetName = first.PetName, }); secondPasswordLength.Resolve(second.PasswordLength); ``` ### Tackling a reference to a _list_ of mutually dependant components For simple references, the above generated code works. However we also need to consider when lists of mutually dependant components are being referenced. Take the following PCL ```tf component "another" "./first" { passwordLength = length([ for _, v in many : v.passwordLength ]) } component "many" "./second" { options { range = 10 } petName = another.petName } ``` In this case the reference `many` is the _collection_ being iterated on which is a mutually dependant component. This is unfortunately a thing that happens in the real-world case described in https://github.com/pulumi/pulumi/issues/13581. What we do here is extract the entire list comprehension / generator into a variable and resolve the computation later <details> <summary>Generated TypeScript</summary> ```ts const [loopingOverMany, resolveLoopingOverMany] = pulumi.deferredOutput<Array<number>>(); const another = new First("another", {passwordLength: loopingOverMany.apply(loopingOverMany => loopingOverMany.length)}); const many: Second[] = []; for (const range = {value: 0}; range.value < 10; range.value++) { many.push(new Second(`many-${range.value}`, {petName: another.petName})); } resolveLoopingOverMany(pulumi.output(many.map((v, k) => [k, v]).map(([_, v]) => (v.passwordLength)))); ``` </details> <details> <summary>Generated Python</summary> ```py looping_over_many, resolve_looping_over_many = pulumi.deferred_output() another = First("another", { 'passwordLength': looping_over_many.apply(lambda looping_over_many: len(looping_over_many)}) many = [] for range in [{"value": i} for i in range(0, 10)]: many.append(Second(f"many-{range['value']}", { 'petName': another.pet_name })) resolve_looping_over_many(pulumi.Output.from_input([v["passwordLength"] for _, v in many])) ``` </details> <details> <summary>Generated C#</summary> ```csharp var loopingOverMany = new Pulumi.DeferredOutput<List<int>>(); var another = new Components.First("another", new() { PasswordLength = loopingOverMany.Output.Apply(loopingOverMany => loopingOverMany.Length), }); var many = new List<Components.Second>(); for (var rangeIndex = 0; rangeIndex < 10; rangeIndex++) { var range = new { Value = rangeIndex }; many.Add(new Components.Second($"many-{range.Value}", new() { PetName = another.PetName, })); } loopingOverMany.Resolve(Output.Create(many.Select((value, i) => new { Key = i.ToString(), Value = pair.Value }).Select(v => { return v.PasswordLength; }).ToList())); ``` </details> ### Typing and Lifting issues When extracting the reference expressions and rewriting variables, for some reason the typing information seem to get lossy and lifting variables isn't generating the right code 🤔 in the previous examples with lists of components (ts) the expression `loopingOverMany.length` should have been _lifted_ into `loopingOverMany.apply(many => many.length)` similarly python and C# examples should have been lifted. Currently this is why I've skipped the compilation step in the program test options. EDIT: fixed some of the lifting issues, now we correctly use `.apply` when necessary For the sake of not making this PR even bigger, I will open another issue to tackle these related typing and lifting problems. - [ ] ~TODO link to issue about lifting replaced variables~ Lifting deferred output variables is working |
||
|---|---|---|
| .. | ||
| deferred-outputs.py | ||
| first.py | ||
| second.py | ||