First step in addressing https://github.com/pulumi/pulumi/issues/14873
Note: I suspect I'll need to temporarily disable some codegen tests to
get this merged, and then once we release v3.98.0 and the next version
of esc, we can re-enable.
# Description
This PR fixes an issue in Go program where if users are writing invoke
expressions inline inside other expressions, then that invoke is
extracted into a temporary variable and then later referenced the same
way it was used. This is because non-output-versioned invokes cannot be
used directly since they return a tuple (InvokeResult, err) so we need
to check for the error before proceeding.
Fixes#14064
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This fixes how programgen generates import statements
to handle conflicting imports when two imported packages
have the same name, e.g.
github.com/pulumi/pulumi-aws/sdk/v5/go/aws/ecs
github.com/pulumi/pulumi-awsx/sdk/go/awsx/ecs
To do this, we add a fileImporter type that tracks these imports.
It prefers unnamed imports for packages unless one of the following is
true:
- the name of the package has already been used by another import
- the name of the package does not match the last component
of the import path (e.g., `example.com/foo-go` with `package foo`).
If the name has already been used by another import,
it attempts the following in-order:
- Combine the last two path components of the import path
into an identifier and use that if available.
e.g., `awsxs3` from `sdk/go/awsx/s3`.
- Append a number to the package name and increment it
until an unused name is found.
e.g. `ecs2`, `ecs3`, and so on.
There's a change in how this information is tracked as well.
Previously, this was a pull approach: various calls returned
programImports objects which all got merged together.
This change switches to a push approach:
as code is generated and imports are requested,
they're submitted to the fileImporter which keeps track of them
until the next `Reset()` call.
The above also has a nice side effect of dropping a parameter.
Another change worth explicitly calling out:
Previously, getModOrAlias partially duplicated some of the logic
implemented in getPulumiImport, and used `mod`, `originalMod`
in a non-obvious way.
This generated incorrect imports like the following
(note the two `/aws` at the end):
github.com/pulumi/pulumi-aws/sdk/v5/go/aws/aws
This change replicates more of the logic of getPulumiImport
(now called addPulumiImport) into this function,
and addresses the discrepancy in codegen caused by `mod`/`originalMod`.
The result leaves most existing code unchanged,
except in a couple existing cases where the resulting changes make sense
given the logic for named imports outlined above.
Resolves#11176
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:
ioutil.Discard => io.Discard
ioutil.NopCloser => io.NopCloser
ioutil.ReadAll => io.ReadAll
ioutil.ReadFile => os.ReadFile
ioutil.TempDir => os.MkdirTemp
ioutil.TempFile => os.CreateTemp
ioutil.WriteFile => os.WriteFile
This change switches all of these entities
across the repository.
Following this change,
the only references to ioutil are in schema files:
% rg -l ioutil
pkg/codegen/testing/test/testdata/aws-4.26.0.json
pkg/codegen/testing/test/testdata/aws-4.36.0.json
pkg/codegen/testing/test/testdata/aws-4.37.1.json
pkg/codegen/testing/test/testdata/aws-5.4.0.json
pkg/codegen/testing/test/testdata/aws-5.16.2.json
The bulk of this change was generated automatically
with manual touch ups afterwards.
* Normalize providers merge
* Update CHANGELOG_PENDING.md
* Fix CHANGELOG_PENDING.md
* Add unit tests
* Ensure go lints
* Remove `Go Lint` as redundant
* Fix go.mod for sdk
* Fix some tests
* Correctly fetch the package name from a token
* Restore unnecessary changes
* Use the correct pkg for provider
As described in #8821, docs generated for helper functions can be incorrect because optional arguments to parameter objects are not correctly handled.
Previously, code would be generated like so:
```go
policyDocument, err := iam.GetPolicyDocument(ctx, &iam.GetPolicyDocumentArgs{
Statements: []iam.GetPolicyDocumentStatement{
iam.GetPolicyDocumentStatement{
Sid: "1",
//...
```
However "Sid" is of type `*string`.
This four helper conversion functions, to handle the primitive types we lower from, and modifies codegen to recursively apply these functions inside of an invoke call.
In the new code generation, the above is instead rendered as:
```go
policyDocument, err := iam.GetPolicyDocument(ctx, &iam.GetPolicyDocumentArgs{
Statements: []iam.GetPolicyDocumentStatement{
iam.GetPolicyDocumentStatement{
Sid: pulumi.StringRef("1"),
//...
```
* Move program tests into folders
* update package schema
* Enabled tests pass
* Fix lints and begin to update test cases
* Re-enable tests
* Update aws version to v4
* Refactor language specific parts
* Hook up dotnet and nodejs
* Update tests from master
* SSOT for schema/version
* Name blocking errors. Leave tests in valid state
* Give each language its own folder
* Remove SkipCompile for azure-sa (bug was fixed)
* Fix nits + changes asserts to require
* Remove unused import
* One last assert => require
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.
With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.