Commit Graph

59 Commits

Author SHA1 Message Date
Zaid Ajaj c7c91f023f
[program-gen/csharp] Fixes generated code for a list of resources used in resource option DependsOn ()
# Description

Fixes  by special casing how we handle `dependsOn` and generating
`DependsOn = { ... }` instead of `DependsOn = new[] { ... }`

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-03-25 15:34:54 +00:00
Zaid Ajaj 8b3f94b89c
[program-gen] Fix panic when generating programs for MLC packages using external types ()
# Description

For an MLC package such as `aws-static-website`, it has some types which
are referenced from the `aws` package. Program-gen assumes packages are
inferred from resources and invokes, not types which caused panics in Go
(), dotnet and python
(https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/2)

This PR fixes those panics. In Go the panic was due to using package
name instead of the package reference from the imported type. In dotnet
and python was due to assuming no external type references. Now we
generate nice code for all these languages.

That said, there is still an issue of resolving imports for the packages
of these external types. It works in Go, TypeScript doesn't need it but
dotnet and python do. That is why the latter are added in `SkipCompile`
in the test program.


Fixes 
Fixes
https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/3
Fixes
https://github.com/pulumi/pulumi-converter-constructor-syntax/issues/2

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-10 17:23:15 +00:00
Zaid Ajaj ed79536f65
[program-gen] Emit missing trivia for resources and local variables ()
# Description

For C#, Python and TypeScript: emit missing comments for local variable
declaration
For Go: emit missing comments for local variables and resources


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-11 17:02:12 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Thomas Meckel f83ee4a5fc
feat: Add support for language specific settings for resources ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR contains changes to support language specific settings for
resources. This PR is a prerequisite to resolve a corresponding
[bug](https://github.com/pulumi/pulumi-terraform-bridge/issues/1460) in
the Terraform Bridge.

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi-terraform-bridge/issues/1460

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Thomas Meckel <tmeckel@users.noreply.github.com>
2023-11-29 16:35:08 +00:00
Zaid Ajaj 72a333e882
[programgen/csharp] Fixes compiling an empty list of expressions from object properties ()
Fixes  

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-28 12:43:35 +00:00
Fraser Waters 99b736b55b Add localDependencies option to GenerateProject
This isn't currently actually used anywhere. I've just threaded it
through to all the program gen functions where it will be needed.

Matrix testing will be using and testing this.
2023-08-08 12:28:19 +01:00
Zaid Ajaj d66a037ba3 Fixes list initializer for plain lists in resource properties 2023-08-07 18:43:37 +02:00
bors[bot] bceabb0351
Merge
13606: [program-gen] Normalize the declaration name of generated resource components r=Zaid-Ajaj a=Zaid-Ajaj

This PR adds a new function `DeclarationName()` to PCL components which is then used as the name of the component inside of the code, distinguishing it from the file name where it lives. The function returns a valid and idiomatic name to be used and references in the generated code for all of the language generators. 

For example if you have a component of which its source code files live `./some-component` then the declaration name for that component would be `SomeComponent` etc. 

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Zaid Ajaj <zaid.naom@gmail.com>
2023-07-27 17:13:32 +00:00
Zaid Ajaj 3e39cb2820 Normalize the declaration name of generated resource components 2023-07-27 15:15:27 +02:00
Zaid Ajaj c6d1117036 Fix typing for optional and complex config variables in main program 2023-07-26 13:37:53 +02:00
Zaid Ajaj 7e5e452909 Extend SkipResourceTypechecking to allow generating unknown resources 2023-06-14 19:02:56 +02:00
Zaid Ajaj 703c53cfeb Turn error diagnostics into warnings 2023-06-02 17:17:59 +02:00
Zaid Ajaj 588e21fde7 Impleneted python program-gen for PCL components 2023-04-05 13:19:52 +02:00
Zaid Ajaj 81f2722ba1
Update comment on collectObjectTypedConfigVariables
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-03-27 12:34:48 +02:00
Zaid Ajaj 98803e4d15 Examples of iterating lists and maps of objects in components 2023-03-24 10:40:18 +01:00
Zaid Ajaj 81fe24ae5b Emit containers of object-typed config variables as plain types 2023-03-24 09:46:09 +01:00
Zaid Ajaj 726caed8ab Object-typed config variables for components 2023-03-23 23:25:21 +01:00
Zaid Ajaj 0988177853 Implement description as comments or docstring for config variables in program-gen 2023-03-21 15:01:16 +01:00
Zaid Ajaj 26e58eb9a8 nodejs program-gen implementation for components and support for nested component declarations 2023-03-14 16:17:14 +01:00
Zaid Ajaj b90b5a5bf9 Add "NotImplemented" PCL function intrinsic 2023-03-10 12:14:28 +01:00
Zaid Ajaj ba639d555a PCL components and dotnet program-gen implementation 2023-03-08 14:21:34 +01:00
Abhinav Gupta 407e6953e3
codegen/{go,js,.net,py}: Generate retainOnDelete
Uses the newly added `options.retainOnDelete` in PCL
to generate the RetainOnDelete resource option
for Go, NodeJS, .NET, and Python.

Java support cannot be added without a release of pulumi/pulumi
because the new field in pcl.ResourceOptions is not yet visible
to Java.

Resolves 
2023-03-06 13:57:23 -08:00
Abhinav Gupta e395deef6b
all: Assert => Assertf
Migrates all remaining usages of
`contract.Assert*` and `contract.Require*` to the f variants,
which require adding meaningful error messages.

There were a couple cases where a `testing.T` or `testing.B`
was already available.
For those, this uses t.FailNow or require.NoError.

Refs 
2023-03-03 14:37:43 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta f3e39d2f2f
chore: WriteString(Sprintf(..)) => Fprintf(..)
Replace `buffer.WriteString(fmt.Sprintf(..))` calls,
where buffer is one of `bytes.Buffer`, `strings.Builder`, or `bufio.Writer`,
with equivalent `fmt.Fprintf` calls -- all those types are io.Writers.
2023-03-01 13:22:33 -08:00
Abhinav Gupta 8280296699
gosimple: printf, time.Since, TrimPrefix, etc.
Fix a bunch of other gosimple issues:

- `Printf` used for static string
- `Skip(Printf(..))` => `Skipf(..)`
- `time.Now().Sub(t)` => `time.Since(t)`
- `HasPrefix; TrimPrefix` => `TrimPrefix`
- Unnecessary type casts
- `IndexRune(..) != 1` => `ContainsRune`
- `if cond { return true } else { return false }` => `return cond`
2023-01-12 09:55:34 -08:00
Abhinav Gupta 3d64521cf2
unused: Remove unused functions and types
Several functions and types were reported as unused.

Most notably, jsonTemp and the method in jsonSpiller that uses it
were all unused.
2023-01-12 09:55:34 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Ian Wahbe 6ff61ae1ae Don't use `*schema.Package` in .NET codegen 2022-12-08 17:50:45 +01:00
Ian Wahbe 6d4b3d619b Simplify `findFunctionSchema`
Update test output

Expose a token lookup method in PCL, and then use it instead of
`schema.PackageReference.Functions().Get`.
2022-12-07 15:05:28 +01:00
Ian Wahbe a601fdb080 Support a logical name for config vars 2022-11-02 12:39:57 -07:00
Zaid Ajaj 12bb360d9b Implement pcl.AnnotateResourceInputs and apply property overrides for deeply nested objects 2022-10-13 11:47:01 +02:00
aq17 525849bf17 Support options.version on pulumi convert 2022-09-23 09:41:17 -07:00
aq17 0a9e72782f fix csharp submodule codegen
fix csharp codegen

fix csharp codegen
2022-09-22 11:43:58 -07:00
Alex Qiu b1557bc5ec
Fix StackReference code gen () 2022-08-19 10:27:05 -07:00
Zaid Ajaj 846b7b88cb
dotnet program-gen now targets .NET 6 ()
* dotnet program-gen targets .NET 6

* changelog entry

* Update GH workflows to use .NET 6

* Enable DOTNET_ROLL_FORWARD=Major for GH workflows

* Expose GeneratePrograms with ability to provide generation options. Use specialized options for import

* Bump dotnet test projects TFM and relax warnings as errors

* refactor code according to Ian's comments

* Include error in generator diagnostics when unable to find function schema

* remove specialized version of GenerateProgram for import and refactor lowerExpression

* remove AcceptCodegenChanges from program driver

* dotnet integration tests updated to net6.0

* Generate return await on top-level Deployment.RunAsync
2022-07-21 21:04:02 +02:00
Aaron Friel c2e196bf3f
[codegen] Simplify program generation for explicit providers () 2022-07-19 09:26:40 -07:00
Kyle Pitzen a14dfc2742
fix: Adds support for C#-specific InputParameter names () 2022-07-06 15:27:17 +02:00
Mikhail Shilkov 544e981d48
Fix the panic on pulumi convert for package without a version () 2022-06-30 08:19:26 +02:00
Pat Gavlin 36cbf572f4
[schema] Add support for on-demand binding. ()
These changes extend the public API of `pkg/codegen/schema` to support
on-demand binding of package members. On-demand binding is appropriate for
scenarios that do not require the entire package, especially those such as
program code generation or the YAML LSP server that require only specific
types/functions/etc.

The extensions to the public API consist of two new types and several new
methods. The most notable of these are `PackageReference` and
`Loader.LoadPackageReference`. The former provides the on-demand binding
interface, while the latter creates instances of the former (n.b. it was
my intent to make a breaking change to the signature of `Loader.LoadPackage`
s.t. it returns a `PackageReference`, but the circular dependency between
this Go module and those for YAML and Java prevented that change).

These changes _dramatically_ reduce the memory required to interace with
Pulumi Packages, and only require memory proportional to the number of
accessed package members. We may be able to improve on this in the future
by removing type/resource/function interning, which would allow those
values to be garbage collected at a granaular level rather than at a
package level. That is a more radical change, though, as it requires new
equality semantics for each of the affected types (some of which are
currently used as map keys).
2022-05-23 15:44:35 -07:00
Aaron Friel e6728a7b7e codegen: workaround missing package versions, language info 2022-05-04 10:00:53 +02:00
Fraser Waters bec8f22290
Add GenerateProject to codegen ()
* Add GenerateProject to codegen

* typo

* Write directly to filesystem

* lint

* Call install deps

* lint

* lint

* Fix requirements.txt
2022-04-29 17:04:04 +01:00
Aaron Friel 98e48f4cc4
codegen: preserve externally visible names of a resources and outputs ()
* codegen: preserve externally visible names of a resources and outputs

* refactor: rename unique name to logical name
2022-04-25 15:07:25 -07:00
Ian Wahbe ef2d6ef44a
Program codegen correctly handles foreign imports ()
* Handle third party resources in nodejs

* Test nodejs change

* Add C# support

* Add python support

* Add go support

This involved extending the third-party test to accommodate modules.

* Update CHANGELOG_PENDING.md

* Nit: gitplace.org => git.example.org
2022-01-31 21:48:32 +01:00
Anton Tayanovskyy e60d6bf248
Programgen support for F.Invoke forms in .NET () ()
* Implement .NET codegen for F.Invoke forms

* Add tests for .NET Invoke

* Fixed conflict

* Accept changes reverting fargate example
2021-11-18 17:53:17 -05:00
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. ()
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00
T-Vova 87e4c92c66
Implemented `sha1()` support for Code Generator ()
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 21:40:38 +03:00
Pat Gavlin 7b1d6ec1ac
Reify `Input` and `Optional` types in the schema type system. ()
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00