Commit Graph

36 Commits

Author SHA1 Message Date
Zaid Ajaj 8b3f94b89c
[program-gen] Fix panic when generating programs for MLC packages using external types ()
# Description

For an MLC package such as `aws-static-website`, it has some types which
are referenced from the `aws` package. Program-gen assumes packages are
inferred from resources and invokes, not types which caused panics in Go
(), dotnet and python
(https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/2)

This PR fixes those panics. In Go the panic was due to using package
name instead of the package reference from the imported type. In dotnet
and python was due to assuming no external type references. Now we
generate nice code for all these languages.

That said, there is still an issue of resolving imports for the packages
of these external types. It works in Go, TypeScript doesn't need it but
dotnet and python do. That is why the latter are added in `SkipCompile`
in the test program.


Fixes 
Fixes
https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/3
Fixes
https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/2

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-10 17:23:15 +00:00
Zaid Ajaj 3b9f5c09b7
[program-gen] Fix stack overflow when binding invoke that resolves to promise ()
# Description


Fixes  because using `ContainsPromises` doesn't account for
recursive object references where as `ContainsEventuals` does.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-20 15:49:08 +00:00
Zaid Ajaj 5e5357d259
[schema] Fixes marshalling the "plain" flag from object or resource properties ()
# Description

When marshalling a schema that has resources or types with plain
properties, the `plain` flag was not preserved. This PR fixes it and
adds a roundtrip test for the plain properties.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-27 11:05:03 +00:00
Anton Tayanovskyy ba039c20f8
Support returning plain values from methods ()
Support returning plain values from methods.

Implements Node, Python and Go support.

Remaining:

- [x] test receiving unknowns
- [x] acceptance tests written and passing locally for Node, Python, Go
clients against a Go server
- [x] acceptance tests passing in CI
- [x] tickets filed for remaining languages
   - [x] https://github.com/pulumi/pulumi-yaml/issues/499
   - [x] https://github.com/pulumi/pulumi-java/issues/1193
   - [x] https://github.com/pulumi/pulumi-dotnet/issues/170 

Known limitations:

- this is technically a breaking change in case there is code out there
that already uses methods that return Plain: true

- struct-wrapping limitation: the provider for the component resource
needs to still wrap the plain-returning Method response with a 1-arg
struct; by convention the field is named "res", and this is how it
travels through the plumbing

- resources cannot return plain values yet

- the provider for the component resource cannot have unknown
configuration, if it does, the methods will not be called
- Per Luke https://github.com/pulumi/pulumi/issues/11520 this might not
be supported/realizable yet

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/12709

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-18 06:02:06 +00:00
Justin Van Patten 5e19f60665
Correctly roundtrip enums in schema marshalling ()
The code to marshal an enum was not keeping the enum's comment.

Fixes 
2023-09-12 18:34:40 +00:00
Fraser Waters 94dfb68c56 Correctly roundtip overlay type tokens in schema marshalling
Fixes https://github.com/pulumi/pulumi/issues/13671.

The code to marshal a TokenType back to a `$ref` simple copied the token
across, but that didn't correctly reconstruct the URL that `$ref`s
expect.
2023-08-08 19:28:51 +01:00
Kyle Dixler fa299fff98 Add test covering codegen expectations in
Fixes 
Note: Dotnet codegen is incorrect and tracked by 
2023-08-01 10:47:17 -07:00
Zaid Ajaj cd65005389 Fix panic in GenerateProject when version is not set in schema 2023-07-14 01:23:27 +02:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
Zaid Ajaj 3bc0e6fe38 Fix aliasing package names using dashes when schema doesn't include go package info override 2023-06-20 18:11:47 +02:00
Zaid Ajaj 8912eae4c6 Fixes python panic when emiting code for index expressions that aren't typechecked 2023-06-09 01:44:09 +02:00
Zaid Ajaj 2ce1fdb091 Fix panic in go program-gen when encountering splat expressions 2023-06-07 13:22:41 +02:00
Fraser Waters 0a33b66692 Fix version number in typerefs when marshalling schemas
Fixes https://github.com/pulumi/pulumi/issues/13000
2023-05-24 08:16:20 +01:00
Zaid Ajaj e9c9fb5e32 Support range expressions that are of type output 2023-04-26 20:00:32 +02:00
Fraser Waters 93a2767b50 Escape import names for python
Part of https://github.com/pulumi/pulumi/issues/12075
2023-03-16 14:44:13 +00:00
bors[bot] 3e22568b94
Merge
12347: all: Reformat with gofumpt r=abhinav a=abhinav

Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-03-03 19:30:12 +00:00
Abhinav Gupta 6ae75053c8
testdata/schema: Add random 4.11.2
Adds the most recent relese of pulumi-random to included schemas.
We need something newer than 4.2.0 or 4.3.1 (both from 2021)
to test options added since then, like retainOnDelete.

Refs https://github.com/pulumi/pulumi/pull/12306#issuecomment-1450923083
2023-03-03 09:13:55 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
bors[bot] 8915911825
Merge
11819: Cleanup diag error for protected resources r=dixler a=iwahbe

Change the error message for deleting protected resources so it sounds less like we tried and failed to delete the resource, and more like we chose not to. 

Fixes 


11834: Preallocate slices with a known capacity. r=dixler a=RobbieMcKinstry

Enable the prealloc linter, which identifies slices with a known capacity, but are not preallocated, which results in unnecessary allocations and memcpys.

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] N/A: enabling a linter. I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] **N/A: not a user-facing change.** I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


11842: Changelog and go.mod updates for v3.51.1 r=dixler a=justinvp



11845: Move SecretsProvider to pkg/secrets r=dixler a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Trying to push DefaultSecretsProvider up past the backends resulted in a module loop between pkg/resource/stack and pkg/resource/deploy. This places SecretProvider in the secrets module to avoid that.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works - N/A Just moving interface declaration 
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - N/A Internal code changes
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version - No
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: Robbie McKinstry <thesnowmancometh@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-01-12 20:11:42 +00:00
Zaid Ajaj 330676a0d0 Initial implementation of simplified invokes for dotnet and nodejs 2023-01-11 14:17:14 -08:00
Robbie McKinstry 1f78baae71
Preallocate slices with a known capacity.
Enable the prealloc linter, which identifies slices
with a known capacity, but are not preallocated, which
results in unnecessary allocations and memcpys.
2023-01-11 12:52:51 -08:00
Aaron Friel a682a1ab06
Revert "Simplified invokes: SDK-gen and program-gen implementation for dotnet and nodejs" 2022-12-20 17:47:29 -08:00
Zaid Ajaj a6f26d2355 Initial implementation of simplified invokes for dotnet and nodejs 2022-12-16 13:39:03 +01:00
Aaron Friel c891102f7d testing: Allow plugin testing users to define their own plugin sets 2022-12-06 16:21:48 -08:00
Ian Wahbe 7bfdd82e24 Standardize pkg loading for TestReferenceRenderer 2022-11-03 16:46:41 -07:00
aq17 77b694e5fe Fix failing tests 2022-10-28 10:18:09 -07:00
Alex Qiu 1798e771e5
Merge branch 'master' into aqiu/yaml-251 2022-10-27 10:52:41 -07:00
Ian Wahbe d6ad246e10 Add missing schemas
Missing schemas are
- aws 4.36.0
- google-native 0.18.2
- kubernetes 3.7.0
- random 4.3.1
- azure-native 1.28.0
2022-10-24 16:47:54 -07:00
aq17 76199f9e5b fix 2022-10-24 10:33:48 -07:00
aq17 b10b0c80a1 Test YAML PCL examples 2022-10-24 10:33:48 -07:00
Aaron Friel 2d90969b58 feat(ci): Enable fully offline codegen tests with versioned plugins
Commit 1 of 2: this makes the changes to every file except the schemas,
for ease of review.
2022-10-11 05:16:23 -07:00
aq17 525849bf17 Support options.version on pulumi convert 2022-09-23 09:41:17 -07:00
Ian Wahbe a76f5cbec6
Improve PCL error message when accessing a missing field ()
* Prepare for v3.39.2 release

* Release v3.39.2

* Cleanup after v3.39.2 release

* Add test case

This involves bumping the k8s and eks version to support the repro.

* Improve error message for a missing property

* Restore old versions
2022-09-08 18:45:15 +02:00
Aaron Friel 3cd0665a2f
[schema] Faster schema loading via caching alongside plugins ()
* [engine] cache schemas in schema loader to files

* [engine] single instance mmapped schemas, preventing use-after-free segfault

* fix: clean up remaining references to close method

* chore: pr feedback
2022-06-13 23:27:11 -07:00
Aaron Friel 1da317dcc9
[codegen] Improve handling of resources with outputs that are resources. () 2022-05-20 10:29:05 -07:00
Ian Wahbe a1e18dae4d
export codegen tests for internal use ()
* Export Codegen test modules

* Document pkg stability guarantee

* Expose programgen

* Recommendation: Improve wording

* Move `internal` to `testing`

* Re-rout references to codegen/internal

* Fix some other "internal" references
2022-02-07 12:10:04 +01:00