Commit Graph

33 Commits

Author SHA1 Message Date
Zaid Ajaj b4c00f7e30
[go/program-gen] Fix union type type resolution in Go program generation ()
# Description

Fixes https://github.com/pulumi/pulumi-azure-native/issues/1554 

### Context
The problem here is that when we compute `InputType: model.Type` in
`pcl.Resource`, we map the types of input properties of resources from
`schema.Type` into `model.Type`. When one of these properties is a
`schema.UnionType` (union of objects to be exact), we map that _as is_
to `model.UnionType` which trips up Go program-gen as it doesn't know
how to reduce the type to the actual _selected_ object type based on the
resource inputs.

### Resolution
The way to fix this is not in Go program-gen, instead we _reduce_ the
computed union types during the binding phase into the actual object
types based on the resource inputs so that all program generators only
work against explicit objects rather than having to deal with unions of
objects

### Example:

```pcl
resource "example" "azure-native:eventgrid:EventSubscription" {
    destination = {
        endpointType = "EventHub"
        resourceId = "example"
    }
    expirationTimeUtc = "example"
    scope = "example"
}
```

Before:
```go
pulumi.Run(func(ctx *pulumi.Context) error {
	_, err := eventgrid.NewEventSubscription(ctx, "example", &eventgrid.EventSubscriptionArgs{
		Destination: eventgrid.EventHubEventSubscriptionDestination{
			EndpointType: "EventHub",
			ResourceId:   "example",
		},
		ExpirationTimeUtc: pulumi.String("example"),
		Scope:             pulumi.String("example"),
	})
	if err != nil {
		return err
	}
	return nil
})
```

After:
```go
pulumi.Run(func(ctx *pulumi.Context) error {
	_, err := eventgrid.NewEventSubscription(ctx, "example", &eventgrid.EventSubscriptionArgs{
		Destination: &eventgrid.EventHubEventSubscriptionDestinationArgs{
			EndpointType: pulumi.String("EventHub"),
			ResourceId:   pulumi.String("example"),
		},
		ExpirationTimeUtc: pulumi.String("example"),
		Scope:             pulumi.String("example"),
	})
	if err != nil {
		return err
	}
	return nil
})
```

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-06-06 19:13:19 +00:00
Zaid Ajaj b96f19b2fa
[program-gen/pcl] Fixes type-annotating nested resource properties when these have quoted keys ()
# Description

When binding resource properties and annotating these with types from
their schemas, we seem to skip this annotation process when resource
properties and their nested objects use _quoted_ keys `{ "key" = <value>
}` rather than using _literal_ keys `{ key = <value> }`.

This results in issues such as
https://github.com/pulumi/kube2pulumi/issues/60 where a csharp property
name override was not correctly applied because
1) kube2pulumi generated properties for resources that are quoted (this
should be fine)
2) binding the resource properties skipped annotating the nested object
with its corresponding schema type
3) program-gen in dotnet didn't have access to the schema type of the
nested object to correctly apply the property override

This PR fixes this issue by extending PCL resource binding to also check
for properties which have quoted keys.

Fixes https://github.com/pulumi/kube2pulumi/issues/60 and potentially
other issues that arise from converters generating PCL with quoted keys.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-27 12:42:52 +00:00
Zaid Ajaj 1430093ab0
[program-gen/pcl] Avoid pretty printing large object graphs when encountering bind error in resource properties ()
# Description

When we bind PCL program and a resource property doesn't type-check, we
pretty print the type of the resource property in its _entirety_ in the
error message and that includes nested objects, nested maps, nested
lists, unions etc. instead of using type references: we pretty print the
`model.Type` derived from `schema.Type` and this resolves the entire
graph.

This PR changes it such that we print the `schema.Type` in string form
in the error message when we encounter a bind error
in resource properties instead of fully resolving and printing the full
`model.Type`.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-20 01:20:42 +00:00
Zaid Ajaj fe5e65f0f4 Allow binding unsupported range and collection types in non-strict mode for pulumi convert 2023-07-12 19:13:57 +02:00
Zaid Ajaj e832a505f1 Implement lenient traversal for resources 2023-06-20 13:50:18 +02:00
Zaid Ajaj 464b07508e Allow traversing unknown properties from resources when skipping resource type checking 2023-06-16 15:06:08 +02:00
Zaid Ajaj 7e5e452909 Extend SkipResourceTypechecking to allow generating unknown resources 2023-06-14 19:02:56 +02:00
bors[bot] d69e5f2d26
Merge
13131: [pcl/components] Fixes range scoping for PCL components r=Zaid-Ajaj a=Zaid-Ajaj

# Description

This PR implements proper `range` block scoping for `pcl.Component` so that when binding the component body, it understands references to the `range` expression. This fixes a couple of the PCL binder issues we have been seeing in TF converter such as  https://github.com/pulumi/pulumi-terraform-bridge/issues/1184, https://github.com/pulumi/pulumi-terraform-bridge/issues/1150, https://github.com/pulumi/pulumi-terraform-bridge/issues/1148 at least locally these errors are resolved for the linked TF example. 

Also small fix for the error we are seeing for `length(...)` because fields of object-typed config were implicitly optional and errored before. Now it flattens `option<T>` to just `T` and no longer errors. https://github.com/pulumi/pulumi-terraform-bridge/issues/1186

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Zaid Ajaj <zaid.naom@gmail.com>
2023-06-08 23:37:53 +00:00
Zaid Ajaj 604d0f9b77 Fixes range scoping for PCL components 2023-06-08 21:43:54 +02:00
Fraser Waters ef7b123bba Don't return empty hcl.Diagnostics
Fixes https://github.com/pulumi/pulumi-terraform-bridge/issues/1201

hcl.Diagnostics implements the Error interface, but this means a list of
zero diagnostics still looks like a non-nil error, which throws of
normal "if err == nil" checks.

This changes a few use sites of hcl.Diagnostics to ensure we return nil
when there aren't any diagnostics rather than an empty slice. This
ensures if they get cast to `Error` they don't trip up standard `err ==
nil` checks.
2023-06-08 19:14:41 +01:00
Zaid Ajaj cb573d2aa2 Fix stack overflow panic when pretty printing recursive types 2023-05-11 16:28:16 +02:00
Abhinav Gupta acaf79bc10
all: Drop //nolint:goconst
Drops the nolint:goconst directive
now that the goconst linter is no longer enabled.
2023-03-09 11:15:21 -08:00
Zaid Ajaj ba639d555a PCL components and dotnet program-gen implementation 2023-03-08 14:21:34 +01:00
bors[bot] 2e3b732558
Merge
12305: pcl/options: Support retainOnDelete r=abhinav a=abhinav

Adds support for the retainOnDelete option to PCL.
With this in place, we'll be able to update code generators
to generate the retainOnDelete option.

Refs 


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-03-04 02:33:44 +00:00
Abhinav Gupta e395deef6b
all: Assert => Assertf
Migrates all remaining usages of
`contract.Assert*` and `contract.Require*` to the f variants,
which require adding meaningful error messages.

There were a couple cases where a `testing.T` or `testing.B`
was already available.
For those, this uses t.FailNow or require.NoError.

Refs 
2023-03-03 14:37:43 -08:00
Abhinav Gupta 7d1a9ec3b0
pcl/options: Support retainOnDelete
Adds support for the retainOnDelete option to PCL.
With this in place, we'll be able to update code generators
to generate the retainOnDelete option.

Refs 
2023-03-03 10:29:59 -08:00
Abhinav Gupta 0bff0b8716 sdk/go: Remove 'nolint' directives from package docs
Go treats comments that match the following regex as directives.

    //[a-z0-9]+:[a-z0-9]

Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)

Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.

    // nolint:foo
    //nolint: foo

This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.

Verification:
Here's the output of `go doc` on some entities
before and after this change.

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

nolint: lll, interfacer

nolint: lll, interfacer

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```

After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine. nolint: revive
```

After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine.
```

Note that golangci-lint offers a 'nolintlint'  linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.

Resolves 

Related: https://github.com/golangci/golangci-lint/issues/892

[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
  tee "$FILES" |
  xargs perl -p -i -e '
    s|// nolint|//nolint|g;
    s|//nolint: |//nolint:|g;
  '
rg '.go$' < "$FILES" | xargs gofmt -w -s
2023-01-06 09:06:47 -08:00
Fraser Waters 64d571c8fa Fix PCL binding of options.range = number 2022-12-07 19:47:44 +00:00
Ian Wahbe 3d27c75de8 Don't emit missing var errors for const vars
We test that consts behave correctly by omitting them from an example
and assigning from the omitted value.
2022-11-21 17:20:54 -08:00
Ian Wahbe d770162254 Remove traverse errors on dynamic types
Preserve traverser src range

Improve testing diags

Re-enable resource type checking

Warn instead of skipping missing resources

Improve explination for the new test

Get test to generate output

Reenable forceing resource type checking

CL

Fix TestBindProgram

Cleanup PR
2022-11-16 14:41:30 -08:00
aq17 b10b0c80a1 Test YAML PCL examples 2022-10-24 10:33:48 -07:00
Zaid Ajaj 12bb360d9b Implement pcl.AnnotateResourceInputs and apply property overrides for deeply nested objects 2022-10-13 11:47:01 +02:00
Ian Wahbe b2ce84c195 Add `modulePath` to go 2022-10-10 16:01:53 -07:00
Ian Wahbe 97075a4fe0 Use loadPackageSchema instead of accessing entries
Direct access to entries is both not threadsafe and failes without
version support.
2022-10-07 13:57:51 -07:00
aq17 525849bf17 Support options.version on pulumi convert 2022-09-23 09:41:17 -07:00
Aaron Friel a4b1d6b2a7 ci: gofmt 1.18+ clean 2022-09-21 09:48:39 -07:00
Alex Qiu f8d3bb676a
Accept options.version on pulumi convert () 2022-08-22 10:36:01 -07:00
Alex Qiu b1557bc5ec
Fix StackReference code gen () 2022-08-19 10:27:05 -07:00
Aaron Friel 301e16e277
[codegen/go] Support provider resources () 2022-07-14 15:49:36 -07:00
Pat Gavlin 36cbf572f4
[schema] Add support for on-demand binding. ()
These changes extend the public API of `pkg/codegen/schema` to support
on-demand binding of package members. On-demand binding is appropriate for
scenarios that do not require the entire package, especially those such as
program code generation or the YAML LSP server that require only specific
types/functions/etc.

The extensions to the public API consist of two new types and several new
methods. The most notable of these are `PackageReference` and
`Loader.LoadPackageReference`. The former provides the on-demand binding
interface, while the latter creates instances of the former (n.b. it was
my intent to make a breaking change to the signature of `Loader.LoadPackage`
s.t. it returns a `PackageReference`, but the circular dependency between
this Go module and those for YAML and Java prevented that change).

These changes _dramatically_ reduce the memory required to interace with
Pulumi Packages, and only require memory proportional to the number of
accessed package members. We may be able to improve on this in the future
by removing type/resource/function interning, which would allow those
values to be garbage collected at a granaular level rather than at a
package level. That is a more radical change, though, as it requires new
equality semantics for each of the affected types (some of which are
currently used as map keys).
2022-05-23 15:44:35 -07:00
Aaron Friel 98e48f4cc4
codegen: preserve externally visible names of a resources and outputs ()
* codegen: preserve externally visible names of a resources and outputs

* refactor: rename unique name to logical name
2022-04-25 15:07:25 -07:00
Ian Wahbe 7da81c1d7f
PCL: Use resource aliases when resolving a type ()
* PCL: Use resource aliases when resolving a type

* Add a test

* Fix lint

* Fetch schemas in the makefile

Because we have exposed the `TestProgramCodegen` function to the public,
we can no longer pin specific schema version from that function. In
general, we should avoid mutating global state in tests. This becomes
especially important when the tests are used by other packages.

Using the makefile to setup global state both simplifies how we fetch
schemas and ensures that the fetches are intentional.

* Fix path

* remove unused imports

* Add comments
2022-03-18 11:48:39 -07:00
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. ()
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00