Commit Graph

39 Commits

Author SHA1 Message Date
Justin Van Patten b0c276ece0
Update esc to v0.6.1 ()
First step in addressing https://github.com/pulumi/pulumi/issues/14873

Note: I suspect I'll need to temporarily disable some codegen tests to
get this merged, and then once we release v3.98.0 and the next version
of esc, we can re-enable.
2023-12-19 08:05:16 +00:00
Zaid Ajaj ac6adb98bf
[go/program-gen] Fix using inline invoke expressions inside resources, objects and arrays ()
# Description

This PR fixes an issue in Go program where if users are writing invoke
expressions inline inside other expressions, then that invoke is
extracted into a temporary variable and then later referenced the same
way it was used. This is because non-output-versioned invokes cannot be
used directly since they return a tuple (InvokeResult, err) so we need
to check for the error before proceeding.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-03 14:22:43 +00:00
Abhinav Gupta a24ad1839b
programgen(go): Handle conflicting names in imported packages
This fixes how programgen generates import statements
to handle conflicting imports when two imported packages
have the same name, e.g.

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/ecs
    github.com/pulumi/pulumi-awsx/sdk/go/awsx/ecs

To do this, we add a fileImporter type that tracks these imports.
It prefers unnamed imports for packages unless one of the following is
true:

- the name of the package has already been used by another import
- the name of the package does not match the last component
  of the import path (e.g., `example.com/foo-go` with `package foo`).

If the name has already been used by another import,
it attempts the following in-order:

- Combine the last two path components of the import path
  into an identifier and use that if available.
  e.g., `awsxs3` from `sdk/go/awsx/s3`.
- Append a number to the package name and increment it
  until an unused name is found.
  e.g. `ecs2`, `ecs3`, and so on.

There's a change in how this information is tracked as well.
Previously, this was a pull approach: various calls returned
programImports objects which all got merged together.

This change switches to a push approach:
as code is generated and imports are requested,
they're submitted to the fileImporter which keeps track of them
until the next `Reset()` call.
The above also has a nice side effect of dropping a parameter.

Another change worth explicitly calling out:
Previously, getModOrAlias partially duplicated some of the logic
implemented in getPulumiImport, and used `mod`, `originalMod`
in a non-obvious way.
This generated incorrect imports like the following
(note the two `/aws` at the end):

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/aws

This change replicates more of the logic of getPulumiImport
(now called addPulumiImport) into this function,
and addresses the discrepancy in codegen caused by `mod`/`originalMod`.
The result leaves most existing code unchanged,
except in a couple existing cases where the resulting changes make sense
given the logic for named imports outlined above.

Resolves 
2023-07-25 12:49:37 -07:00
Zaid Ajaj a5975ca585 lint 2023-07-14 01:47:25 +02:00
Zaid Ajaj cd65005389 Fix panic in GenerateProject when version is not set in schema 2023-07-14 01:23:27 +02:00
Zaid Ajaj 6528a9fe7a Go module support as component resources in program-gen 2023-05-08 16:20:04 +02:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Aaron Friel abb3763229 fix(ci): Split programgen tests over more jobs 2022-10-16 23:20:45 -07:00
Aaron Friel 4e7a887f3f chore: update test to use aws v5 2022-10-13 15:59:06 -07:00
Aaron Friel 2d90969b58 feat(ci): Enable fully offline codegen tests with versioned plugins
Commit 1 of 2: this makes the changes to every file except the schemas,
for ease of review.
2022-10-11 05:16:23 -07:00
Ian Wahbe b2ce84c195 Add `modulePath` to go 2022-10-10 16:01:53 -07:00
aq17 525849bf17 Support options.version on pulumi convert 2022-09-23 09:41:17 -07:00
Ian Wahbe 94422f97f8
Expose external package cache + global default (Take 2) ()
* Expose external package cache

* CL

* Add a default global cache

* More cache passing

* Compare `Cache` entries by pointer equality

* Add a check for providers when doing imports

* Pass the cache

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-08-12 20:04:21 +02:00
Aaron Friel cca0a39e70
Revert "Expose external package cache ()" ()
This reverts commit 16c173963a.
2022-08-11 15:04:14 -07:00
Ian Wahbe 16c173963a
Expose external package cache ()
* Expose external package cache

* CL

* Add a default global cache

* More cache passing

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-08-11 13:28:27 -07:00
Ian Wahbe e39998784a
Expose the check function () 2022-03-16 11:42:30 -07:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Ian Wahbe 8666d9c0b7
Normalize providers merge ()
* Normalize providers merge

* Update CHANGELOG_PENDING.md

* Fix CHANGELOG_PENDING.md

* Add unit tests

* Ensure go lints

* Remove `Go Lint` as redundant

* Fix go.mod for sdk

* Fix some tests

* Correctly fetch the package name from a token

* Restore unnecessary changes

* Use the correct pkg for provider
2022-02-08 15:35:36 +01:00
Ian Wahbe a1e18dae4d
export codegen tests for internal use ()
* Export Codegen test modules

* Document pkg stability guarantee

* Expose programgen

* Recommendation: Improve wording

* Move `internal` to `testing`

* Re-rout references to codegen/internal

* Fix some other "internal" references
2022-02-07 12:10:04 +01:00
Aaron Friel ee709d6c52 [codegen/go] Improve optional params in invoke
As described in , docs generated for helper functions can be incorrect because optional arguments to parameter objects are not correctly handled.

Previously, code would be generated like so:

```go
    policyDocument, err := iam.GetPolicyDocument(ctx, &iam.GetPolicyDocumentArgs{
      Statements: []iam.GetPolicyDocumentStatement{
        iam.GetPolicyDocumentStatement{
          Sid: "1",
          //...
```

However "Sid" is of type `*string`.

This four helper conversion functions, to handle the primitive types we lower from, and modifies codegen to recursively apply these functions inside of an invoke call.

In the new code generation, the above is instead rendered as:

```go
    policyDocument, err := iam.GetPolicyDocument(ctx, &iam.GetPolicyDocumentArgs{
      Statements: []iam.GetPolicyDocumentStatement{
        iam.GetPolicyDocumentStatement{
          Sid: pulumi.StringRef("1"),
          //...
```
2022-02-01 11:44:40 -08:00
Ian Wahbe f12b8fa00c
Cleanup program-gen test dependency gathering () 2021-10-19 12:17:19 -07:00
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. ()
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00
Ian Wahbe 9f6589ed8c
iwahbe/7802/compile program generator test output ()
* Move program tests into folders

* update package schema

* Enabled tests pass

* Fix lints and begin to update test cases

* Re-enable tests

* Update aws version to v4

* Refactor language specific parts

* Hook up dotnet and nodejs

* Update tests from master

* SSOT for schema/version

* Name blocking errors. Leave tests in valid state

* Give each language its own folder

* Remove SkipCompile for azure-sa (bug was fixed)

* Fix nits + changes asserts to require

* Remove unused import

* One last assert => require
2021-09-29 11:33:57 -07:00
T-Vova 90c0fa30af
Implemented file reading support for Go code generator ()
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:10:25 +03:00
Komal fddeea8a88
[codegen] Unify program codegen testing ()
* ProgramGen test driver

* Refactor to avoid exporting hcl2 entities
2021-07-27 18:00:39 -07:00
Pat Gavlin 7b1d6ec1ac
Reify `Input` and `Optional` types in the schema type system. ()
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
Mikhail Shilkov 1378c45849 Discriminated unions in schema and programgen 2021-05-05 10:01:22 +02:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
evanboyle 2f71fbbf2d traverse object and tuple cons expressions for imports 2020-08-05 15:23:36 -07:00
Evan Boyle 31770c3300
go program gen: prompt array conversion, unused range vars, id handling () 2020-06-24 11:07:26 -07:00
Evan Boyle b302dbcbca
go program gen: lift expressions where prompt optionals are required () 2020-06-23 10:59:26 -07:00
Evan Boyle 8b8170252b
[Go Program Gen] multiline strings, get/lookup disambiguation, webserver example () 2020-06-18 13:34:22 -07:00
Evan Boyle 9cdddd2566
[Go Program Gen] Support for EKS example () 2020-06-17 17:19:39 -07:00
Evan Boyle 2d61852e4f
Go program gen: resource range, readDir, template strings, etc () 2020-06-15 23:00:02 -07:00
Evan Boyle 94cd67b564
[Go Program Gen] Improved type handling and invokes () 2020-06-05 18:52:00 -07:00
Evan Boyle ea114b04ee
output generation for s3 logging example () 2020-05-26 07:07:54 -07:00
Evan Boyle 8dd329f57f
Go basic program gen () 2020-05-21 10:23:33 -07:00
Evan Boyle 0d257bb32d
go gen_program shell + test + sample () 2020-05-14 08:43:12 -07:00