This adds support for replacement parameterised providers to Python and
a small integration test to check it works e2e.
When using parameterised providers we need to use the new (currently
unstable) RegisterPackage system, instead of sending
Version/DownloadURL/etc via RegisterResourceRequest. Once
RegisterPackage is stable the intention is to change _all_ packages to
use it and for normal packages to fall back to the
RegisterResourceRequest options, while parameterised packages will
error.
The actual parameter value is embedded in the python SDK as a base64
string that we decode before sending to the gRPC endpoint as bytes.
Noticed this issue while doing SDK gen for parameterised providers, but
figured it deserved its own conformance test. Check that if a provider
has a pre-release semver that the _exact_ version can be reported by the
generated SDK. This already just works for NodeJS, but Python needed a
fix to write the version to `_utilities.py` rather than trying to
unconvert the pypi version from the package.
Also needed to make the conformance test checks for
`GetProgramDependencies` even weaker (which is fine, they are just a
very basic sanity check) because the provider reports a version of
"3.0.0-alpha.1.internal" while the python version is "3.0.0a1+internal".
The latest version of the core Pulumi SDK contains a decorator,
`@deprecated`, that is used when generating SDK code in order to signify
deprecated properties in a way that can be recognised by other SDK code.
This is useful when writing generic Python code that e.g. traverses
class properties without triggering deprecation warnings for those not
explicitly mentioned in user code. The [original pull
request](https://github.com/pulumi/pulumi/pull/16400) has more details.
Alas, we can't rely on the fact that a user will upgrade _both_ a
particular (generated) provider SDK and the core Pulumi SDK at the same
time. Thus, it's entirely possible that a user bumps their version of
(say) `pulumi_aws`, whilst leaving their `pulumi` library at the same
(compatible, according to specified bounds) version. In doing so they'd
hit errors when the new SDK tried to import the `@deprecated` decorator,
which doesn't exist in the old core SDK.
This commit thus fixes this by altering code generation so that each SDK
receives its own inlined copy of the `@deprecated` decorator, which it
can reference regardless of the version of the core SDK. This decorator
applies the same `_pulumi_deprecated_callable` tag to functions it
decorates, which a sufficiently modern SDK will recognise to avoid
triggering e.g. https://github.com/pulumi/pulumi/issues/15894. Later on,
we can hopefully find a way to avoid doing this and use only a version
of `@deprecated` specified in the core SDK.
Codegen tests have been updated and the inlined decorator has manually
been tested using the AWS Classic SDK.
Addresses
https://github.com/pulumi/pulumi/pull/16400#discussion_r1646562455
Epic: Improved Typing https://github.com/pulumi/pulumi/issues/12689
This PR adds the flag `Languages.Python.InputTypes` to the schema, which
can take the values `classes` or `classes-and-dicts`. In the first
iteration of the feature, this defaults to `classes`, which leaves code
generation as is and does not change input types.. If the flag is set to
`classes-and-dicts`, `TypedDict` based types are generated next to the
current `<Type>Args` classes. In the future this could be extended to
support `dicts` to generate only `TypedDict` types.
The generated types are conditional on the used type checker to work
around perf issues in MyPy and PyCharm, see
https://github.com/pulumi/pulumi/issues/12689#issuecomment-2117240276
```python
if not MYPY:
class DeploymentArgsDict(TypedDict):
api_version: NotRequired[Input[str]]
kind: NotRequired[Input[str]]
metadata: NotRequired[Input['ObjectMetaArgsDict']]
...
elif False:
DeploymentArgsDict: TypeAlias = Mapping[str, Any]
```
Removing the workaround is tracked in
https://github.com/pulumi/pulumi/issues/16408
---------
Co-authored-by: Anthony King <anthony@datapane.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Presently, the Python SDK incorrectly emits deprecation warnings for
properties even if user code does not reference them. This is due to
various pieces of code in the SDK that enumerate properties e.g. for the
purpose of serialisation/RPC calls. In enumerating properties, their
getters are invoked, triggering the deprecation warnings.
There isn't currently a way for us to detect which properties may or may
not be deprecated and handle them appropriately, so this commit
introduces one. Deprecation messages are moved into a new decorator,
`@pulumi.deprecated`. This decorator accepts a `Callable` and wraps it,
making two changes:
* Before the decorated `Callable`'s execution, deprecation messages are
printed.
* The returned `Callable` is tagged with a new reserved property,
`_pulumi_deprecated_callable`, which references the wrapped `Callable`
(that is, the original `Callable` whose invocation will _not_ produce
deprecation warnings).
With this in place, we subsequently make the following two changes:
* We modify the SDK enumeration code (specifically that in
`input_type_to_dict` to check for `_pulumi_deprecated_callable`
properties and use them where appropriate to invoke getters without
triggering deprecation warnings.
* We modify Python code generation so that instead of emitting
statements to print deprecation warnings, we simply emit applications of
the `@pulumi.deprecated` decorator instead.
This commit adds some Python unit tests to exercise the decorator and
manual testing has been performed using the AWS Classic SDK.
Fixes#15894
The current Python code generator does not emit `__init__.py` files for
modules that only contain enumerations. This is problematic in cases
such as https://github.com/pulumi/pulumi/issues/16221, whereby
`pulumi-command` wishes to define an enumeration to be shared across
modules, in a module with no other types or resources. In such cases, an
SDK is generated that does not type check, since the enumeration types
are not exported appropriately. This commit addresses this by generating
`__init__.py` files in these cases as expected.
Fixes#16221
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new flag to the schema metadata to tell codegen to use the
new proposed style of SDKs where we fill in versions and write go.mods
etc.
I've reworked pack to operate on packages assuming they're in this new
style. That is pack no longer has the responsibility to fill in any
version information.
This updates python and node codegen to write out SDKs in this new
style, and fixes their core libraries to still be buildable via pack.
There are two approaches to fixing those, I've chosen option 1 below but
could pretty easily rework for option 2.
1) Write the version information directly to the SDKs at the same time
as we edit the .version file. To simplify this I've added a new
'set-version.py' script that takes a version string an writes it to all
the relevant places (.version, package.json, etc).
2) Write "pack" in the language host to search up the directory tree for
the ".version" file and then fill in the version information as we we're
doing before with envvar tricks and copying and editing package.json.
I think 1 is simpler long term, but does force some amount of cleanup in
unrelated bits of the system right now (release makefiles need a small
edit). 2 is much more localised but keeps this complexity that
sdk/nodejs sdk/python aren't actually valid source modules.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
When replaceOnChanges is set in the schema for a property with a
multi-word name, currently, Python SDK generation incorrectly puts its
Python name into replace_on_changes.
This is wrong, because these property names are passed to the engine and
they should be engine names (i.e. schema names). All other languages
(Node.js, .NET, Go) do this correctly.
This PR adds a test case and fixes the Python generation to use the same
naming as all other SDKs.
Fixes#15665
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Python 3.7 is unsupported and has been end-of-life since 6/27/2023. See
https://devguide.python.org/versions/
The core `pulumi` Python SDK package is being updated to indicate that
Python 3.8 or greater is required (see #15365).
This change updates the default minimum required Python version for
generated provider SDKs to Python 3.8 or greater as well.
Fixes#15367
### Description
Enabling python conformance tests.
Uses a virtual environment for the generated projects and activates it
when installing dependencies.
~Right now fails at runtime due to missing engine address but the rest
works(TM)~
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This addresses two issues:
1. `pkg_resource` is deprecated in favor of `importlib.resources` and
`importlib.metadata`
(https://setuptools.pypa.io/en/latest/pkg_resources.html)
2. Generated provider SDKs don't indicate that they have a dependency on
`setuptools` (which includes `pkg_resources`), which can cause problems
when installing the package in environments that don't have `setuptools`
installed. That's not often common in Pulumi projects, as the virtual
environment created by the CLI will include `setuptools`, however, if
creating the virtual environment manually with `python -m venv`,
`setuptools` is no longer included in the created virtual environment as
of Python 3.12.
Fixes#12414
Based on #14002, thanks @edgarrmondragon!
---------
Co-authored-by: Edgar Ramírez Mondragón <edgarrm358@gmail.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>
Turn on the golangci-lint exhaustive linter. This is the first step
towards catching more missing cases during development rather than
in tests, or in production.
This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.
I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
Fixes#14601
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
For python we were generating bad imports for 1) members of the same
module, and 2) for members imported from another module.
When importing from a different module, we want to always reference
members using a fully qualified name (`mymod.childmod.member.Member`) to
avoid any naming clashes in case multiple modules have members of the
same name. For these we need to import their top level python module
(`from ${relPath} import mymod`).
When importing members from the same module, we want to use relative
imports (`from .some_member import SomeMember`).
Fixes https://github.com/pulumi/pulumi/issues/12980
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Combination of a few cleanups.
1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945
This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.
Most of these edits were made by running `perfsprint --fix`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
After adding a test schema for aliases in #14594 @Frassle noticed it was
incorrect to write out the name and project of aliases into the SDKs.
This PR fixes this such that we now only write out the `type` of the
alias into generated code.
Supersedes #14594
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Support returning plain values from methods.
Implements Node, Python and Go support.
Remaining:
- [x] test receiving unknowns
- [x] acceptance tests written and passing locally for Node, Python, Go
clients against a Go server
- [x] acceptance tests passing in CI
- [x] tickets filed for remaining languages
- [x] https://github.com/pulumi/pulumi-yaml/issues/499
- [x] https://github.com/pulumi/pulumi-java/issues/1193
- [x] https://github.com/pulumi/pulumi-dotnet/issues/170
Known limitations:
- this is technically a breaking change in case there is code out there
that already uses methods that return Plain: true
- struct-wrapping limitation: the provider for the component resource
needs to still wrap the plain-returning Method response with a 1-arg
struct; by convention the field is named "res", and this is how it
travels through the plumbing
- resources cannot return plain values yet
- the provider for the component resource cannot have unknown
configuration, if it does, the methods will not be called
- Per Luke https://github.com/pulumi/pulumi/issues/11520 this might not
be supported/realizable yet
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/12709
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14418
Reopens https://github.com/pulumi/pulumi/issues/12546
This removes the `_configure()` ResourceArgs helper method as it has
caused a number of issues (linked below).
`_configure()` was added in order to support initializing default values
sdk side for python. This has led to the following PRs being merged to
address unexpected bugs.
## Overview of fixes:
It will be quite hard to demonstrate without examples, so I will give an
example of the code and describe its short comings:
---
- https://github.com/pulumi/pulumi/issues/14418
```python
args = MyArgs() # fails as required argument foo not provided
args.foo = "Hello, World!"
```
---
- https://github.com/pulumi/pulumi/pull/14235
Supporting `imageName` and `image_name`
```python
def _configure(...
image_name, # required
...):
...
# This should not error, but errors.
_configure(imageName="debian")
```
---
- https://github.com/pulumi/pulumi/pull/14281
```python
def _configure(...
image_name, # required
...
**kwargs):
...
# This should not fail, but fails as `image_name` is not provided
_configure(imageName="debian")
```
---
- https://github.com/pulumi/pulumi/pull/14014
```python
class Thing:
def __init__(self,
...
# Optional
certmanager=None,
...):
...
Thing._configure(
...
certmanager=None,
...)
...
def _configure(...):
...
# This block runs when certmanager = None, but should not.
if not isinstance(certmanager, ProviderCertmanagerArgs):
certmanager = certmanager or {}
def _setter(key, value):
certmanager[key] = value
ProviderCertmanagerArgs._configure(_setter, **certmanager)
...
Provider()
```
---
- https://github.com/pulumi/pulumi/pull/14321
```python
registry_info=accessToken.apply(get_registry_info)
# Build and publish the image.
image = Image(
'my-image',
build=DockerBuildArgs(
context='app',
),
image_name=image_name,
# Note that this is an Output.
registry=registry_info,
)
# registry is not None and it is not an instance of RegistryArgs, so we fall into the if, thinking it is a dict, but it is an Output.
if registry is not None and not isinstance(registry, RegistryArgs):
registry = registry or {}
def _setter(key, value):
registry[key] = value
RegistryArgs._configure(_setter, **registry)
__props__.__dict__["registry"] = registry
```
---
- https://github.com/pulumi/pulumi/pull/14318
```python
# foo.core.v1.PodArgs may be an external type and may not be upgraded to have _configure() and will fail, but should not.
if pod is not None and not isinstance(pod, foo.core.v1.PodArgs):
pod = pod or {}
def _setter(key, value):
pod[key] = value
pulumi_kubernetes.core.v1.PodArgs._configure(_setter, **pod)
```
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
We recently added the ability to assign default values for nested types
in generated Python SDKs, when the nested types are passed as dicts. We
do so by calling a new static `_configure` method on the input type, to
assign default values.
This wasn't working correctly if the value was an `Output[T]` or
`Awaitable[T]`. This change fixes the code to call `_configure` inside
an `apply`, if needed.
Since the boilerplate code was getting bigger, moved it into a
`configure` helper method in `_utilities.py`.
Fixes#14320
We recently added the ability to assign default values for nested types
in generated Python SDKs, when the nested types are passed as dicts. We
do so by calling a new static `_configure` method on the input type, to
assign default values.
The problem is that we're emitting calls to `_configure` for external
types, and such types may not have been upgraded yet to the latest
codegen, so they may not have a static `_configure` method yet.
This change makes it so that `_configure` is only called for external
types that have a `_configure`, via a runtime `hasattr` check.
Fixes#14316
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14274
Given a property of imageName, Python resource args can receive the
property as image_name or imageName. When provided imageName, the
required argument is provided as imageName, the required image_name
argument is not provided and the _configure call fails.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Python SDK generation's `_configure` method now supports setters of the
property name based on its original name.
Now supports:
```python
Foo("my-foo", propName=10)
```
Instead of just:
```python
Foo("my-foo", prop_name=10)
```
Fixes#14151Fixes#14239
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
Partially addressing https://github.com/pulumi/pulumi/issues/12449
implements output-versioned invokes for functions without inputs for
python SDKs
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
We recently fixed an issue where defaults weren't set for nested objects
when the nested objects are passed as dicts (#13825). Unfortunately,
this introduced a regression when the nested object is optional, but it
itself has required fields, and the nested object is not specified. In
that case, an unintended error is raised.
Consider a `Provider` resource with an optional `certmanager:
ProviderCertmanagerArgs` argument, which itself has two required
properties: `mtls_cert_pem` and `mtls_key_pem`.
When creating a new `Provider` without specifying a `certmanager`, we
get an error:
```
TypeError: ProviderCertmanagerArgs._configure() missing 2 required positional arguments: 'mtls_cert_pem' and 'mtls_key_pem'
```
The source of the problem is this check in the generated `Provider`'s
constructor:
```python
if not isinstance(certmanager, ProviderCertmanagerArgs):
certmanager = certmanager or {}
def _setter(key, value):
certmanager[key] = value
ProviderCertmanagerArgs._configure(_setter, **certmanager)
```
When `certmanager` is not specified, its value is `None`, which is also
not an instance of `ProviderCertmanagerArgs`. So the code inside the
`if` executes. `ProviderCertmanagerArgs._configure` is called on an
empty dict, and the error is raised because there are two required
positional arguments to `ProviderCertmanagerArgs._configure`.
The fix is to add an additional check to ensure the value is not `None`.
Fixes#14012
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#13810
Adds a `_configure` helper function to configure `ResourceArgs` and
reuse the code on `dict`s (or resources that can `__getitem__`
`__setitem__` to set the default value.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
Removes Resource constructor deprecation checks in favor of deprecation
checks in the ResourceArgs constructor.
Checks in the ResourceArgs constructor which will be changed to
`ResourceArgs.__configure__` in #13825 and compatible with dicts and
objects that implement `__getitem__` `__setitem__`.
Merge https://github.com/pulumi/pulumi/pull/13825 after and update
`__internal_init__` logic to use the `ResourceArgs.__configure__`
Fixes#12546
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This change affects projects that use Python SDK generator with the
following option enabled:
"pyproject": {"enable": true"}
Before this change, pyproject.toml was generated alongsie setup.py for
such projects.
After the change, setup.py is no longer generated, by pyproject.toml is
extended to opt into setuptools backend and ensure that building this
project embeds metadata files:
py.typed # to support mypy and pyright users
pulumi-plugin.json # to support Pulumi interop with the generated
provider SDK
The generated code can then be built with the `build` module to produce
source and/or wheel distributions. See also:
https://pypa-build.readthedocs.io/en/latest/
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#13709 - this takes Option 2 (drop setup.py, make pyproject.toml
self-sufficient)
See also:
https://setuptools.pypa.io/en/latest/userguide/datafiles.html#package-data
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/13754.
Fixes https://github.com/pulumi/pulumi/issues/13709.
This removes the install scripts that we generated as part of NodeJS and
Python packages. As we move to Python wheels we'll no longer have an way
to run these scripts (see #13709). That would leave NodeJS as the only
language which supports this feature.
So for consistency and simplicity just remove it from everywhere.
Users who still want to ensure plugins are installed upfront (before
`up` or similar runs) can run `pulumi plugin install` to install all the
current project plugins.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Fixes https://github.com/pulumi/pulumi/issues/12738https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.
Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
This change causes deprecation messages on Result objects from Function
invokes to only be shown when using getters on the result object instead
of on instantiation.
SDK functions return Result objects and they may have deprecated fields.
Instantiation of these objects causes deprecation messages to appear
even if a user is not using the field.
Fixes#11739
Python codegen set the input to the default value and then checked if it
was used to determine if the deprecation message should be printed.
This caused the deprecation message to always be printed if a default
value was set.
This fixes this and is split into 2 commits for easier reviewing.
This commit extracts the logic from setup.py to calculate
dependencies and reuses it in pyproject.toml. It creates a
deterministic list of deps and their associated version constraints.
This commit introduces a new codegen test that checks the contents
of the pyproject.toml file. This allows us to demonstrate the file
is generated and matches against an expected manifest.
This commit adds the pyproject.toml schema in Go,
and provides a hook to marshal the data in codegen.
Lastly, it adds a toggle to the provider schema to
enable pyproject.toml generation.
The Python SDK generator wasn't correctly referencing types that have a
different package name in their type token than the name of the schema's
package (which is the case when embedding k8s CRD resources/types in a
package). This change fixes the generator to emit these local references
correctly.
Prefer a top-level function over a method for minPythonVersion,
because minPythonVersion is a validation step, and thus should be
colocated with validation code. It's not a concern of the PackageInfo.
This commit introduces a log line to remind users they
should be setting the Python minimum version in their schema.
This refactor makes accessing the min version a checked error,
increasing safety.
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
12267: chore(all): strings.Replace(..., -1) => strings.Replace(...) r=abhinav a=abhinav
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
12268: chore: WriteString(Sprintf(..)) => Fprintf(..) r=abhinav a=abhinav
Replace `buffer.WriteString(fmt.Sprintf(..))` calls,
where buffer is one of `bytes.Buffer`, `strings.Builder`, or `bufio.Writer`,
with equivalent `fmt.Fprintf` calls -- all those types are io.Writers.
12337: Freeze v3.56.0 r=dixler a=dixler
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: Kyle Dixler <kyle@pulumi.com>
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
Incremental step towards #12132
Migrates uses of contract.{Assert, AssertNoError, Require}
to `*f` variants so that we're required to provide more error context.
Refs #12132
A couple places in pkg/ use Printf without any arguments.
These cases can use the input strings as-is instead.
Switch to either the Print variant, or io.WriteString.
Issues found by staticcheck:
```
backend/httpstate/backend.go:1627:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
backend/httpstate/backend.go:1644:5: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/go/gen.go:2260:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/nodejs/gen.go:1088:3: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/python/gen.go:1861:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
codegen/python/gen.go:436:2: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
```
Refs #11808