With https://github.com/pulumi/pulumi/pull/15957 we introduced the
schema setting `Languages.Python.InputTypes` to optionally generate
TypedDict based input types side-by-side with Args classes. This setting
defaulted to `classes`, meaning that only Args classes are generated. To
enable the TypedDict types, SDK authors had to explicitly opt in by
setting it to `classes-and-dicts`.
We now flip this setting to generating TypedDict input types
side-by-side with the Args classes, unless explicitly disabled by
setting it to `classes`.
Fixes https://github.com/pulumi/pulumi/issues/16375
The existing overlays (e.g. Chart v3 in Kubernetes, or CallbackFunction
in AWS) are not available in every language Pulumi supports. This often
confuses users because the generated docs include all languages Pulumi
supports (e.g. see
https://github.com/pulumi/pulumi-kubernetes/issues/2181).
To solve that problem, this change adds a new optional parameter to the
schema that allows configuring the languages an overlay (resource or
function) supports.
To support this in docsgen the existing Language Chooser
(`LangChooserLanguages`) of resources is made configurable and extended
to functions.
Note: This doesn't support resource methods right now. They'll need
extra handling because and overlay resource method might not support all
of the languages its resource supports. I'll tackle this in a follow up
PR.
Here's a screenshot of how this will look like for the Helm v3 chart for
example:
<img width="1046" alt="Screenshot 2024-07-01 at 16 11 23"
src="https://github.com/pulumi/pulumi/assets/2453580/b1a1365a-6dee-4099-829a-2859639a4c8c">
The PR contains the following commits. I'd recommend to look at the
first three ones and then check the regenerated golden files in the last
one:
- **Add schema parameter to constrain supported languages for overlays**
- **Update developer docs and changelog**
- **Refactor LanguageChooser and always pass supported languages**
- **Regenerate testdata**
relates to #13231
Epic: Improved Typing https://github.com/pulumi/pulumi/issues/12689
This PR adds the flag `Languages.Python.InputTypes` to the schema, which
can take the values `classes` or `classes-and-dicts`. In the first
iteration of the feature, this defaults to `classes`, which leaves code
generation as is and does not change input types.. If the flag is set to
`classes-and-dicts`, `TypedDict` based types are generated next to the
current `<Type>Args` classes. In the future this could be extended to
support `dicts` to generate only `TypedDict` types.
The generated types are conditional on the used type checker to work
around perf issues in MyPy and PyCharm, see
https://github.com/pulumi/pulumi/issues/12689#issuecomment-2117240276
```python
if not MYPY:
class DeploymentArgsDict(TypedDict):
api_version: NotRequired[Input[str]]
kind: NotRequired[Input[str]]
metadata: NotRequired[Input['ObjectMetaArgsDict']]
...
elif False:
DeploymentArgsDict: TypeAlias = Mapping[str, Any]
```
Removing the workaround is tracked in
https://github.com/pulumi/pulumi/issues/16408
---------
Co-authored-by: Anthony King <anthony@datapane.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
# Description
Fixes#15751
- In .NET, non-primitive types such as objects used in config variables
will be generated anyways under the `Types` namespace as plain-shaped
objects inside of the `Config` class. This is because the types from
`Outputs` or `Inputs` cannot be used directly from config. Also no
longer marking objects used from config variables as "outputs" (doesn't
make any sense)
- In Go, disable adding imports into the `config.go` based on used
shared types because these are not used/referenced in the config module
anywhere anyways.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#15478 where sdk-gen should no longer panic when encountering
overly nested arrays or maps of primitive types. The generated code
still fails to compile but validation can be added later to cover cases
where we are missing primitives in the core SDK to handle overly nested
type definitions.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This PR fixes a bug in the dotnet codegen where the wrong "args" class
name would be used, in the case that the resource has no constant
inputs. This is an edge case because most resources do have at least one
such input.
For example, a new resource definition in p/k produced this output:
```csharp
namespace Pulumi.Kubernetes.Yaml.V2
{
public ConfigGroup(string name, Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs? args = null, CustomResourceOptions? options = null)
: base("kubernetes:yaml/v2:ConfigGroup", name, args ?? new ConfigGroupArgs(), MakeResourceOptions(options, ""), remote: true)
{
}
}
```
Which doesn't compile because `ConfigGroupArgs` is in a separate
namespace.
Should be:
```csharp
public ConfigGroup(string name, Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs? args = null, CustomResourceOptions? options = null)
: base("kubernetes:yaml/v2:ConfigGroup", name, args ?? new Pulumi.Kubernetes.Types.Inputs.Yaml.V2.ConfigGroupArgs(), MakeResourceOptions(options, ""), remote: true)
{
}
```
[Here's
](https://github.com/pulumi/pulumi/pull/15488/files#diff-18b12fabab20d68398aced2890b1ca3073cc32081bb62a022b77a5090c209e3bR45)where
the fix manifests itself in the new test case.
## Testing
A new SDK test case was added to cover the whole `kubernetes20`
compatibility mode, based on a simplified schema from the
pulumi-kubernetes provider.
The schema contains a representative set of resources:
1. `kubernetes:core/v1:ConfigMap` - a non-overlay resource representing
a Kubernetes kind.
2. `kubernetes:core/v1:ConfigMapList` - a Kubernetes list kind
3. `kubernetes:helm.sh/v3:Release` - a non-overlay, non-Kubernetes
resource
4. `kubernetes:yaml:ConfigGroup` - an overlay component resource
An important detail is whether a resource has any input properties that
have a constant value, such as we see with `kind` and `apiVersion`. The
`Release` resource intentionally has no such constant inputs.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Long term I want to introduce a new snake_case based naming system to
schemas. There's an internal design doc about this at
https://docs.google.com/document/d/1ns07htpLjw0dJPn5p7TBPVULvwwYc1Pl7l1c1jdGXoY/edit.
As part of that we get the nice feature that _nearly_ all current names
are either valid and the same in both system (e.g. "foo"), or they look
different in the current system (e.g. "fooBar"). But there are a handful
of odd cases where we have providers using names that look like they
should be new style snake case names (e.g. kubernetes has a load of
"x_kubernetes_" properties).
To provide a way to ensure k8s can stay on the current name generation
(even though it gives pretty odd results) this change adds a test with a
schema demonstrating a load of snake_case names. When we do add the new
naming system this test _should not diff_, excepting possibly setting an
option if the new naming system is opt-out rather than opt-in.
Reverts pulumi/pulumi#14704
Fixes#15023 because the PR above changed how object types are
referenced when they are plain. It seems that we only support _shallow_
plain types (only container of nested objects can be plain, not the
object itself) which is why we confused the previous diffs to be
correct.
This reverts the PR previous to fix the breaking change at hand but will
reopen#14662 and make sure the fix for it is to generate `ObjectType`
_and_ `ObjectTypeArgs` regardless of whether the location where the
object type is referenced in the schema.
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
For python we were generating bad imports for 1) members of the same
module, and 2) for members imported from another module.
When importing from a different module, we want to always reference
members using a fully qualified name (`mymod.childmod.member.Member`) to
avoid any naming clashes in case multiple modules have members of the
same name. For these we need to import their top level python module
(`from ${relPath} import mymod`).
When importing members from the same module, we want to use relative
imports (`from .some_member import SomeMember`).
Fixes https://github.com/pulumi/pulumi/issues/12980
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
First step in addressing https://github.com/pulumi/pulumi/issues/14873
Note: I suspect I'll need to temporarily disable some codegen tests to
get this merged, and then once we release v3.98.0 and the next version
of esc, we can re-enable.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945
This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.
Most of these edits were made by running `perfsprint --fix`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Hunting down an issue related to sdk-gen with references to unions, so I
am adding a test schema with unions defined inline (in a property type
of a resource) in this PR and another one incoming for referenced union
types (which has the bug I believe)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#14734
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Fixes#14662 when choosing whether to use `ObjectName` (plain) or
`ObjectNameArgs` (non-plain) we have to check whether the non-plain
version was actually available in first place (sometimes it isn't when
the type isn't used anywhere as non-plain) so the fix here is to use the
package context that tracks the usages to determine whether the `Args`
suffix should be used.
Funny though, for other languages like C#, TypeScript and Python we
don't have a concept of a strictly plain type. Instead all types are
non-plain (accepting values for their properties being inputs/outputs)
which means that a plain element type on maps for these languages has no
effect
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
When using types in union cases (using `oneOf` in the schema) then Go
SDK-gen doesn't detect these types as being used as inputs and only
emits the plain version of a type. That is unless
`generateExtraInputTypes: true` is specified in which case Go SDK-gen
will emit the non-plain types as well.
In the case of azure-native modules, `generateExtraInputTypes` is
currently set to `false` (default) as of latest v2.19.0 and is missing a
few non-plain type variants (for example
`FirewallPolicyFilterRuleCollection` from [this
file](https://raw.githubusercontent.com/pulumi/pulumi-azure-native-sdk/master/network/pulumiTypes.go)).
There no single plain type in azure-native and yet it is missing these
non-plain variants.
This PR fixes that by also traversing the element types of schema union
definitions when determining plain-ness of types and how they are used.
I added an example schema with an array of unions (common case in
azure-native) and confirmed that the change actually results in the
non-plain types being generated when `generateExtraInputTypes` is set to
`false`
Fixes https://github.com/pulumi/pulumi-azure-native/issues/1922
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#14539
Adds regression tests to catch Python SDKgen Resource Argument
regressions.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14631.
This adds a couple of new tests to codegen. Firstly to check that if you
use "urn" or "id" as a resource output property we fail schema binding.
This is because these clash with the urn and id fields that every
resource already has to have.
Secondly a schema gen test that checks that urn and id _do_ work in
other places, i.e. resource inputs, nested types, etc.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
After adding a test schema for aliases in #14594 @Frassle noticed it was
incorrect to write out the name and project of aliases into the SDKs.
This PR fixes this such that we now only write out the `type` of the
alias into generated code.
Supersedes #14594
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
While working on on #14585 I tried to add a test schema for assets and
archives that generate a `generics-only` go SDK but the the output
didn't compile. At first I thought the issue was specific to asset and
archive types but actually it was a broader issue where the
function`pkg.genPlainType(InputObjectType)` would always reduce
`Input[T]` and `Optional[T]` to just `T` on each property of
`InputObjectType`.
It is probably fine to reduce `Input[T]` because we are generating a
plain type after all. However, reducing `Optional[T]` to `T` is
incorrect because the generic variant of go sdks are more strict about
optionality of types.
> It probably works in non-generic SDKs today because it relies on a
runtime cast
Example type `TypeWithAssets` from schema that has a plain and optional
property called `plainAsset`:
❌ Before it was the following and it didn't compile for generic go sdks
```go
type TypeWithAssets struct {
PlainAsset pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
type TypeWithAssetsArgs struct {
PlainAsset pulumix.Input[*pulumi.AssetOrArchive] `pulumi:"plainAsset"`
}
func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) pulumi.AssetOrArchive { return v.PlainAsset })
}
```
✅ Now it generates:
```go
type TypeWithAssets struct {
PlainAsset *pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
type TypeWithAssetsArgs struct {
PlainAsset *pulumi.AssetOrArchive `pulumi:"plainAsset"`
}
func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) *pulumi.AssetOrArchive { return v.PlainAsset })
}
```
Which is correct and compiles
The behavior for current non-generic SDKs remains unchanged
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
While checking out the code coverage report, noticed that lots of logic
that handles assets and archives isn't tested so I've added a schema
that uses assets and archives mainly to the SDK tests.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Support returning plain values from methods.
Implements Node, Python and Go support.
Remaining:
- [x] test receiving unknowns
- [x] acceptance tests written and passing locally for Node, Python, Go
clients against a Go server
- [x] acceptance tests passing in CI
- [x] tickets filed for remaining languages
- [x] https://github.com/pulumi/pulumi-yaml/issues/499
- [x] https://github.com/pulumi/pulumi-java/issues/1193
- [x] https://github.com/pulumi/pulumi-dotnet/issues/170
Known limitations:
- this is technically a breaking change in case there is code out there
that already uses methods that return Plain: true
- struct-wrapping limitation: the provider for the component resource
needs to still wrap the plain-returning Method response with a 1-arg
struct; by convention the field is named "res", and this is how it
travels through the plumbing
- resources cannot return plain values yet
- the provider for the component resource cannot have unknown
configuration, if it does, the methods will not be called
- Per Luke https://github.com/pulumi/pulumi/issues/11520 this might not
be supported/realizable yet
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/12709
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
PR #14492 removed `ToOutput(...)` methods for output types when
generating non-generic SDK variants. The fix there was added to only
include these when `side-by-side` is enabled, which implicitly included
`generics-only` except for a _single_ place where that wasn't the case 😓
(see comment below in `gen.go`)
This PR fixes issue and includes tests for go sdkgen where `generics`
setting is set to `generics-only`:
- `output-funcs-go-generics-only`
- `plain-and-default-go-generics-only`
- `secrets-go-generics-only`
- `simple-enum-schema-go-generics-only`
- `simple-plain-schema-go-generics-only`
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14418
Reopens https://github.com/pulumi/pulumi/issues/12546
This removes the `_configure()` ResourceArgs helper method as it has
caused a number of issues (linked below).
`_configure()` was added in order to support initializing default values
sdk side for python. This has led to the following PRs being merged to
address unexpected bugs.
## Overview of fixes:
It will be quite hard to demonstrate without examples, so I will give an
example of the code and describe its short comings:
---
- https://github.com/pulumi/pulumi/issues/14418
```python
args = MyArgs() # fails as required argument foo not provided
args.foo = "Hello, World!"
```
---
- https://github.com/pulumi/pulumi/pull/14235
Supporting `imageName` and `image_name`
```python
def _configure(...
image_name, # required
...):
...
# This should not error, but errors.
_configure(imageName="debian")
```
---
- https://github.com/pulumi/pulumi/pull/14281
```python
def _configure(...
image_name, # required
...
**kwargs):
...
# This should not fail, but fails as `image_name` is not provided
_configure(imageName="debian")
```
---
- https://github.com/pulumi/pulumi/pull/14014
```python
class Thing:
def __init__(self,
...
# Optional
certmanager=None,
...):
...
Thing._configure(
...
certmanager=None,
...)
...
def _configure(...):
...
# This block runs when certmanager = None, but should not.
if not isinstance(certmanager, ProviderCertmanagerArgs):
certmanager = certmanager or {}
def _setter(key, value):
certmanager[key] = value
ProviderCertmanagerArgs._configure(_setter, **certmanager)
...
Provider()
```
---
- https://github.com/pulumi/pulumi/pull/14321
```python
registry_info=accessToken.apply(get_registry_info)
# Build and publish the image.
image = Image(
'my-image',
build=DockerBuildArgs(
context='app',
),
image_name=image_name,
# Note that this is an Output.
registry=registry_info,
)
# registry is not None and it is not an instance of RegistryArgs, so we fall into the if, thinking it is a dict, but it is an Output.
if registry is not None and not isinstance(registry, RegistryArgs):
registry = registry or {}
def _setter(key, value):
registry[key] = value
RegistryArgs._configure(_setter, **registry)
__props__.__dict__["registry"] = registry
```
---
- https://github.com/pulumi/pulumi/pull/14318
```python
# foo.core.v1.PodArgs may be an external type and may not be upgraded to have _configure() and will fail, but should not.
if pod is not None and not isinstance(pod, foo.core.v1.PodArgs):
pod = pod or {}
def _setter(key, value):
pod[key] = value
pulumi_kubernetes.core.v1.PodArgs._configure(_setter, **pod)
```
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14274
Given a property of imageName, Python resource args can receive the
property as image_name or imageName. When provided imageName, the
required argument is provided as imageName, the required image_name
argument is not provided and the _configure call fails.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14296.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Python SDK generation's `_configure` method now supports setters of the
property name based on its original name.
Now supports:
```python
Foo("my-foo", propName=10)
```
Instead of just:
```python
Foo("my-foo", prop_name=10)
```
Fixes#14151Fixes#14239
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
We recently fixed an issue where defaults weren't set for nested objects
when the nested objects are passed as dicts (#13825). Unfortunately,
this introduced a regression when the nested object is optional, but it
itself has required fields, and the nested object is not specified. In
that case, an unintended error is raised.
Consider a `Provider` resource with an optional `certmanager:
ProviderCertmanagerArgs` argument, which itself has two required
properties: `mtls_cert_pem` and `mtls_key_pem`.
When creating a new `Provider` without specifying a `certmanager`, we
get an error:
```
TypeError: ProviderCertmanagerArgs._configure() missing 2 required positional arguments: 'mtls_cert_pem' and 'mtls_key_pem'
```
The source of the problem is this check in the generated `Provider`'s
constructor:
```python
if not isinstance(certmanager, ProviderCertmanagerArgs):
certmanager = certmanager or {}
def _setter(key, value):
certmanager[key] = value
ProviderCertmanagerArgs._configure(_setter, **certmanager)
```
When `certmanager` is not specified, its value is `None`, which is also
not an instance of `ProviderCertmanagerArgs`. So the code inside the
`if` executes. `ProviderCertmanagerArgs._configure` is called on an
empty dict, and the error is raised because there are two required
positional arguments to `ProviderCertmanagerArgs._configure`.
The fix is to add an additional check to ensure the value is not `None`.
Fixes#14012
# Description
This PR adds a schema option specific to go at
`$.language.go.internalModuleName` which, when set, overrides the name
used for the internal/utilities module and everywhere it is used (see
added test example below)
Fixes#13632
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
This commit introduces a new codegen test that checks the contents
of the pyproject.toml file. This allows us to demonstrate the file
is generated and matches against an expected manifest.
Adds a regression test for the issue reported in #12971.
The issue surfaces when a property has an env-var-based default,
but no constant default value.
The Python SDK generator wasn't correctly referencing types that have a
different package name in their type token than the name of the schema's
package (which is the case when embedding k8s CRD resources/types in a
package). This change fixes the generator to emit these local references
correctly.
name.
Docs URLs are case-insensitive so modules, functions and resources may have
name collisions. If a filename is taken by another class of document, it
will be prefixed with "mod", "res", "fn" and docs will point to this new
unique link.
The priority is as follows:
1. module
2. resource
3. function
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.