The version we currently have doesn't support Go 1.22 properly, so it
throws a bunch of warnings locally when trying to run it with the latest
Go version installed. Just running the latest version locally also
doesn't quite work, since it throws a bunch of errors from the
perfsprint linter, which seems to have gotten stricter.
Upgrade to the latest version of golangci-lint, and fix all the errors
we're getting from it. Mostly done via `perfsprint -fix`, with some
manual changes that `perfsprint -fix` wouldn't touch.
# Description
Fixes#15751
- In .NET, non-primitive types such as objects used in config variables
will be generated anyways under the `Types` namespace as plain-shaped
objects inside of the `Config` class. This is because the types from
`Outputs` or `Inputs` cannot be used directly from config. Also no
longer marking objects used from config variables as "outputs" (doesn't
make any sense)
- In Go, disable adding imports into the `config.go` based on used
shared types because these are not used/referenced in the config module
anywhere anyways.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#15478 where sdk-gen should no longer panic when encountering
overly nested arrays or maps of primitive types. The generated code
still fails to compile but validation can be added later to cover cases
where we are missing primitives in the core SDK to handle overly nested
type definitions.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
### Description
This PR fixes build issues for Go SDKs where enums import the `pulumix`
package without using it or sometimes using it with generated input
interfaces for the enums for non-generic sdks. The problem was using the
function `goPackageInfo` which reads the package info from the language
section of the schema but not initializing `Generics` field to `none`
when it is empty.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Turn on the golangci-lint exhaustive linter. This is the first step
towards catching more missing cases during development rather than
in tests, or in production.
This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.
I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
Fixes#14601
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Reverts pulumi/pulumi#14704
Fixes#15023 because the PR above changed how object types are
referenced when they are plain. It seems that we only support _shallow_
plain types (only container of nested objects can be plain, not the
object itself) which is why we confused the previous diffs to be
correct.
This reverts the PR previous to fix the breaking change at hand but will
reopen#14662 and make sure the fix for it is to generate `ObjectType`
_and_ `ObjectTypeArgs` regardless of whether the location where the
object type is referenced in the schema.
# Description
While covering more parts of go codegen, I've seen that config variables
are broken 😓 specifically when requiring config variables using
`RequireFloat` it should be `RequireFloat64` and `RequireBoolean` should
be `RequireBool`.
Moreover, it seems that `RequireObject` doesn't work at all since the
function signature doesn't match the way it was generated (see #14957)
C# has a similar issue with optional untyped objects as config
variables. For now have skipped compilation for those.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#13821 (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945
This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.
Most of these edits were made by running `perfsprint --fix`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#14734
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#14662 when choosing whether to use `ObjectName` (plain) or
`ObjectNameArgs` (non-plain) we have to check whether the non-plain
version was actually available in first place (sometimes it isn't when
the type isn't used anywhere as non-plain) so the fix here is to use the
package context that tracks the usages to determine whether the `Args`
suffix should be used.
Funny though, for other languages like C#, TypeScript and Python we
don't have a concept of a strictly plain type. Instead all types are
non-plain (accepting values for their properties being inputs/outputs)
which means that a plain element type on maps for these languages has no
effect
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
When using types in union cases (using `oneOf` in the schema) then Go
SDK-gen doesn't detect these types as being used as inputs and only
emits the plain version of a type. That is unless
`generateExtraInputTypes: true` is specified in which case Go SDK-gen
will emit the non-plain types as well.
In the case of azure-native modules, `generateExtraInputTypes` is
currently set to `false` (default) as of latest v2.19.0 and is missing a
few non-plain type variants (for example
`FirewallPolicyFilterRuleCollection` from [this
file](https://raw.githubusercontent.com/pulumi/pulumi-azure-native-sdk/master/network/pulumiTypes.go)).
There no single plain type in azure-native and yet it is missing these
non-plain variants.
This PR fixes that by also traversing the element types of schema union
definitions when determining plain-ness of types and how they are used.
I added an example schema with an array of unions (common case in
azure-native) and confirmed that the change actually results in the
non-plain types being generated when `generateExtraInputTypes` is set to
`false`
Fixes https://github.com/pulumi/pulumi-azure-native/issues/1922
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
After adding a test schema for aliases in #14594 @Frassle noticed it was
incorrect to write out the name and project of aliases into the SDKs.
This PR fixes this such that we now only write out the `type` of the
alias into generated code.
Supersedes #14594
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
While working on on #14585 I tried to add a test schema for assets and
archives that generate a `generics-only` go SDK but the the output
didn't compile. At first I thought the issue was specific to asset and
archive types but actually it was a broader issue where the
function`pkg.genPlainType(InputObjectType)` would always reduce
`Input[T]` and `Optional[T]` to just `T` on each property of
`InputObjectType`.
It is probably fine to reduce `Input[T]` because we are generating a
plain type after all. However, reducing `Optional[T]` to `T` is
incorrect because the generic variant of go sdks are more strict about
optionality of types.
> It probably works in non-generic SDKs today because it relies on a
runtime cast
Example type `TypeWithAssets` from schema that has a plain and optional
property called `plainAsset`:
❌ Before it was the following and it didn't compile for generic go sdks
```go
type TypeWithAssets struct {
PlainAsset pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
type TypeWithAssetsArgs struct {
PlainAsset pulumix.Input[*pulumi.AssetOrArchive] `pulumi:"plainAsset"`
}
func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) pulumi.AssetOrArchive { return v.PlainAsset })
}
```
✅ Now it generates:
```go
type TypeWithAssets struct {
PlainAsset *pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
type TypeWithAssetsArgs struct {
PlainAsset *pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) *pulumi.AssetOrArchive { return v.PlainAsset })
}
```
Which is correct and compiles
The behavior for current non-generic SDKs remains unchanged
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Support returning plain values from methods.
Implements Node, Python and Go support.
Remaining:
- [x] test receiving unknowns
- [x] acceptance tests written and passing locally for Node, Python, Go
clients against a Go server
- [x] acceptance tests passing in CI
- [x] tickets filed for remaining languages
- [x] https://github.com/pulumi/pulumi-yaml/issues/499
- [x] https://github.com/pulumi/pulumi-java/issues/1193
- [x] https://github.com/pulumi/pulumi-dotnet/issues/170
Known limitations:
- this is technically a breaking change in case there is code out there
that already uses methods that return Plain: true
- struct-wrapping limitation: the provider for the component resource
needs to still wrap the plain-returning Method response with a 1-arg
struct; by convention the field is named "res", and this is how it
travels through the plumbing
- resources cannot return plain values yet
- the provider for the component resource cannot have unknown
configuration, if it does, the methods will not be called
- Per Luke https://github.com/pulumi/pulumi/issues/11520 this might not
be supported/realizable yet
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/12709
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
PR #14492 removed `ToOutput(...)` methods for output types when
generating non-generic SDK variants. The fix there was added to only
include these when `side-by-side` is enabled, which implicitly included
`generics-only` except for a _single_ place where that wasn't the case 😓
(see comment below in `gen.go`)
This PR fixes issue and includes tests for go sdkgen where `generics`
setting is set to `generics-only`:
- `output-funcs-go-generics-only`
- `plain-and-default-go-generics-only`
- `secrets-go-generics-only`
- `simple-enum-schema-go-generics-only`
- `simple-plain-schema-go-generics-only`
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#14344 by removing generic `ToOutput*` methods on types when
generating providers in non-generic setting . This means that we will
only generate these methods when `generics` setting is `side-by-side`
since `generics-only` will not touch these anyways
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Currently for an array of objects, codegen for Go outputs a
`pulumi.AnyArray` type. However that does not exist. Make sure it always
outputs the correct `pulumi.Array` type.
While there give Maps the same treatment, as `pulumi.AnyMap` also
doesn't exist.
Fixes #https://github.com/pulumi/pulumi/issues/8072
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Partially addressing #12449 implements output-versioned invokes for
functions without inputs for go.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
This PR implements generating the generic variant of a go SDK from
Pulumi schemas. Currently the idea is to generate a directory `x` inside
the root directory of the go SDK which will contain the same SDK except
using generics and generating far less code than its current counter
part.
Also implements an enum option `$.language.go.generics` which can be set
to the following:
- `none` is the default which maintains the current behavior that
generates legacy types without generics
- `side-by-side` generates the generics sdk variant alongside the
current sdk under directory `x`
- `generics-only` generates only the new sdk with generics at the root
of the package
Still a bunch of things to do:
- [x] Generating `InvokeResult]Output` type from `Output[InvokeResult]`
and generating accessor methods for it
- [x] Generating default values for types and using the `pulumix`
subpackage to do so
- [x] Generating generic SDK variants for all test schemas we have and
making sure they compile (currently only testing
`simple-resource-schema` as shown below)
- [x] Account for plain inputs for components
- [x] Combine pulumix.Join with pulumix.Apply to generate resource
accessor methods
- [x] Problem with `GPtrOutput[T]` and `ArrayOutput[T]` being unwrapped
to `Output[*T]` and `Output[[]T]`
- [x] Remove excess untyped container types from generated enums
- [x] Fix default values for resource methods with lifted single return
value
- [x] Secret properties
Currently the following test schemas have opted for `generics:
"side-by-side"`:
- [x] output-funcs
- [x] simple-enum-schema
- [x] secrets
- [x] simple-plain-schema
- [x] plain-and-default
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
For generated types that impleemnt `pulumi.Output`,
or those that implement `pulumi.Input` with a means of converting
themselves to a `pulumi.Output`,
also generate `ToOutput(context.Context) pux.Output[..]`
to satisfy the `pux.Input[T]` interface.
This allows all these generated types to be used with `pux.Apply`
and other type-safe generic APIs per #13057.
Resolves#13587
# Description
This PR adds a schema option specific to go at
`$.language.go.internalModuleName` which, when set, overrides the name
used for the internal/utilities module and everywhere it is used (see
added test example below)
Fixes#13632
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
13136: Add explicit package versioning to Go codegen r=guineveresaenger a=guineveresaenger
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
This PR adds logic to explicitly set a Go SDK version as well as pluginDownloadURL in the SDK itself.
This will allow us to publish tags with explicit versions, ensure the engine references the correct plugin version, and will allow for the provider to use the correct version plugin much more reliably.
This PR adds a new `pulumiVersion` file and refactors the `pulumiUtilities` file to live alongside the new version file in an `internal` folder for utilities. This allows all resources in the provider to call on `internal.PkgVersion` without generating a mod-level utilities file alongside.
I have tested this against a flat structure SDK (pulumi-docker) and a modular SDK (pulumi-okta).
I have also edited the `schema.json` files in the unit tests to reflect the new utilities path.
The changes in the PR will allow us to:
1. Read a version into `pulumiVersion.go` during codegen
2. Commit and tag that commit as `sdk/v<Version>`
3. Have the exact same SDK that was built and tested be available on github
4. Clean up the CI step that explicitly tags the Go SDK.
Fixes#13012
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Guinevere Saenger <guinevere@pulumi.com>
Fixes https://github.com/pulumi/pulumi/issues/12738https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.
Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
All resources will need default options, so we reference the top
level module and export the resource options functions so they are
accessible in submodules.
If there's no alias defined and there's a root package name defined, then use that for the pkgName of the basePath instead of guessing it from the basePath.
Switches getEnvOrDefaul to os.LookupEnv,
which returns whether the environment variable was set,
even if it was set to blank.
This lets env-var-based defaults differentiate between
empty and unset.
Without this change, an empty string default is not allowed via
environment variables.
sdkgen has a discrepancy in behavior for default values.
There are three scenarios:
- default value
- default value and an environment variable fallback
- only an environment variable fallback
We handle the first two correctly:
the value is set only if there was something to set it to.
But in the third case, we set variables to their zero value
when the environment variable is empty.
// Current implementation effectively does:
if v := os.Getenv("whatever"); v != "" {
out.Field = pointerOf(v)
} else {
out.Field = pointerOf("")
}
To fix this, we generate code to set these values
only if the environment variable is non-empty.
Implementation notes:
Previosly, getDefaultValue generated an expression
holding the default value.
For fixed default values, it was just the value.
For default values with environment variable fallback,
it took the form:
getEnvOrDefault($default, $parser, $env1, $env2, ...).($type)
For example:
getEnvOrDefault("", nil, "ITEM_NAME").(string)
Given that getDefaultValue returned an expression,
its consumers were able to do something like:
if result == nil {
result = getEnvOrDefault(...).(string)
}
As part of this fix, we need to stop assinging to result
if the environment variable is unset,
so we want getDefaultValue to generate:
if result == nil {
if d := getEnvOrDefault(...); d != nil {
result = d
}
}
To accomplish this, we switch getDefaultValue to setDefaultValue.
Instead of returning an expression, it accepts a callback.
It generates the blocks it needs to, and invokes the callback
with an expression holding the default value if necessary.
Resolves#12971
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
Migrates uses of contract.{Assert, AssertNoError} in pkg/codegen/go
to use Assertf and AssertNoErrorf so that
we write more meaningful messages when these contracts are violated.
Incremental step towards deprecating the non-f variants.
Refs #12132
A couple places in pkg/ use Printf without any arguments.
These cases can use the input strings as-is instead.
Switch to either the Print variant, or io.WriteString.
Issues found by staticcheck:
```
backend/httpstate/backend.go:1627:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
backend/httpstate/backend.go:1644:5: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/go/gen.go:2260:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/nodejs/gen.go:1088:3: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/python/gen.go:1861:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/python/gen.go:436:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
```
Refs #11808
11819: Cleanup diag error for protected resources r=dixler a=iwahbe
Change the error message for deleting protected resources so it sounds less like we tried and failed to delete the resource, and more like we chose not to.
Fixes#11816
11834: Preallocate slices with a known capacity. r=dixler a=RobbieMcKinstry
Enable the prealloc linter, which identifies slices with a known capacity, but are not preallocated, which results in unnecessary allocations and memcpys.
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] N/A: enabling a linter. I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] **N/A: not a user-facing change.** I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
11842: Changelog and go.mod updates for v3.51.1 r=dixler a=justinvp
11845: Move SecretsProvider to pkg/secrets r=dixler a=Frassle
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Trying to push DefaultSecretsProvider up past the backends resulted in a module loop between pkg/resource/stack and pkg/resource/deploy. This places SecretProvider in the secrets module to avoid that.
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works - N/A Just moving interface declaration
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - N/A Internal code changes
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version - No
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: Robbie McKinstry <thesnowmancometh@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>