These were found by running `python -m pylint ./lib/test
--rcfile=.pylintrc` in the python SDK folder. Currently we don't
consistently run pylint (or mypy for that matter) against the test
folder and it's built up a number of issues before we can turn it on.
This PR fixes some of the smaller less common ones, a baby step towards
getting the tests fully linted and type checked. Each issue is resolved
in a single commit named for the pylint issue it fixed.
Addressing https://github.com/pulumi/pulumi/issues/8237 for Python.
Since `import` is a reserved keyword in python, I went with
`import_resources` as the function name. It's a bit unfortunate that it
is not the same name as other SDKs but I wasn't sure which name to use
here. Suggestions are welcome 🙏
By default, `pulumi destroy` removes all resources within a stack but
leaves the stack and its configuration intact. If one passes the
`--remove` option to `destroy`, however, the stack and its configuration
will also be removed once the resources within the stack have been
deleted. This commit updates the work of @Moon1706 in #11080 to add
`remove` as an option to the Go, NodeJS and Python Automation API SDKs'
`destroy` methods, which then perform an analogous clean-up.
Closes#11080
---------
Co-authored-by: Nikita Sharaev <n.p.sharaev@tinkoff.ru>
The Python Automation API SDK serializes project settings (the contents
of `Pulumi.yaml` files) using Python's `pyyaml` package. Project
settings in the Python Automation API SDK are represented as instances
of the `ProjectSettings` class. By default, `pyyaml` will serialize
class instances as YAML objects "tagged" with a string that indicates
their class. This is so that, upon deserialization, it can construct
objects of the appropriate class (as opposed to, just dictionaries). As
an example, the following Python program:
```python
class Person:
def __init(self, name: str, age: int) -> None:
self.name = name
self.age = age
will = Person("will", 37)
yaml.dump(will)
```
will produce the following YAML:
```yaml
!!python/object:__main__.Person
age: 37
name: will
```
The string `!!python/object:__main__.Person` is the _tag_ indicating the
class that Python will need to instantiate if e.g. this YAML is
deserialized with `yaml.load` or similar.
Outside of the various Automation APIs, `Pulumi.yaml` files are
"plain-old YAML files" -- language- or library-specific concepts such as
class tags are nowhere to be seen. We thus don't really want this
behaviour when we serialize project settings to YAML. Fortunately, there
is a relatively simple workaround -- instead of passing instances of
`ProjectSettings` to `yaml.dump`, we can just pass vanilla dictionaries
containing the same data. These will be rendered as YAML objects with no
tags, which is what we want.
<em>Un</em>fortunately, we must turn _all_ objects in a hierarchy into
plain dictionaries, or tags will appear at some point. Presently, this
is not the case in the Python SDK, which just uses
`ProjectSettings.__dict__` to get the dictionary for the top-level
object. If there are nested objects in this dictionary (such as e.g.
`ProjectBackend` or `ProjectRuntimeInfo` objects), these are _not_
converted into dictionaries and `pyyaml` writes them as tagged objects,
which will later fail to deserialize.
This commit fixes this issue, adding explicit `to_dict` and `from_dict`
methods to settings classes for the purposes of recursively converting
objects to and from dictionaries. It also:
* adds a test that confirms serialization/deserialization of
configurations containing nested objects now works.
* in order to write this test, exports some types (`ProjectTemplate` and
friends) that appear to be part of the public API but have until now not
been exported.
This commit rebases @Maradonna90's PR #14066 to add support for
`destroy`'s `--exclude-protected` argument to the Python automation SDK.
This addresses the Python part of #12733.
Closes#14066
Co-authored-by: Jendryczko, Marco <marco.jendryczko@hermesworld.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This PR adds an optional `opts` argument to the NodeJS SDK
`Workspace::removeStack` method, which has `force` and `preserveConfig`
optional booleans. Setting these bools to true will add their
corresponding CLI flag to the command.
Fixes#16332
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Many of our tests test JavaScript/TypeScript behaviour and thus rely on
`package.json` files specifying dependencies and `package-lock.json`
files pinning those dependencies. While the dependencies we use in our
tests are minimal and their bounds haven't changed (nor maybe have much
reason to change), the transitive dependency set is potentially large
and worth keeping up-to-date (e.g. for security reasons, or just keeping
Dependabot happy). This commit thus regenerates all the
`package-lock.json` files associated with tests that Dependabot has
previously recommended we bump.
* Closes#13478
* Closes#13473
* Closes#13472
* Closes#13471
* Closes#13470
* Closes#13469
* Closes#13468
* Closes#13467
* Closes#13424
* Closes#13419
* Closes#13418
* Closes#13414
* Closes#13412
* Closes#13411
* Closes#13408
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
Adds Go/Nodejs/Python automation API support for pulumi stack ls --all.
Fixes: [#14226](https://github.com/pulumi/pulumi/issues/14226)
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
Test code should be formatted and linted the same as library code. This
is the first step of that, simply including ./lib/test to the folder
that the black formatter runs on.
When tailing the event log in automation API we currently have nothing
that makes sure we read only complete lines. This means if the OS
happens to flush an incomplete line for whatever reason (or the Go JSON
encoder does, which we're using to write these lines), we might read a
line that is incompletely written, and thus will fail to JSON decode it.
Since the JSON encoder always writes a newline at the end of each
string, we can also make sure that the line we read ends with a newline
and otherwise wait for the rest of the line to be written.
The library we use in Go provides a convenient setting for this, while
in python and nodejs we need to add some code to do this ourselves.
Fixes https://github.com/pulumi/pulumi/issues/15235
Fixes https://github.com/pulumi/pulumi/issues/15652
Fixes https://github.com/pulumi/pulumi/issues/9269 (This is closed
already, but never had a proper resolution afaics)
Fixes https://github.com/pulumi/pulumi/issues/6768
It would be nice to add a typescript test here as well, but I'm not sure
how to do that without marking the readLines function non-private. But I
don't know typescript well, so any hints of how to do that would be
appreciated!
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This updates the typing annotations and semantics of Python Automation
API inline programs. It's now defined to be a callable that returns an
optional awaitable of `None`. This supports both existing synchronous
functions (should just return `None`) and now also supports async
functions that return an `Awaitable` that eventually resolves to `None`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
---------
Co-authored-by: Julien P <julien@caffeine.lu>
# Description
Provide a way for the Automation API to install the Pulumi CLI so that
Automation API can be used in a more standalone manner.
https://github.com/pulumi/pulumi/issues/14987
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
In preparation for supporting Python 3.12...
The test was installing a `requirements.txt` file that depends on the
released version of the `pulumi` package, but that package depends on
the old `grpcio` package that does not work Python 3.12. Instead of
depending on the public `pulumi` package, install the locally built
Python SDK.
Also, fix the test to be able to run on Windows, while making changes
here.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Dependabot updated some references to this in
https://github.com/pulumi/pulumi/pull/15131. But missed a lot,
importantly it didn't update pkg or sdk which are the most important
modules in this repo.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Automation API support for `pulumi config env ls`
Fixes https://github.com/pulumi/pulumi/issues/14797
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) to 0.17.0.
Replaces all the dependabot PRs in the repo with this single PR.
Also bumped `github.com/pulumi/pulumi/sdk/v3` in
`tests/integration/transformations/go/simple/go.mod` from v3.97.0 to
v3.98.0 to use esc v0.6.1, and avoid the appdash issue.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Add python automation API support for adding and removing environments
for stack configuration.
Fixes https://github.com/pulumi/pulumi/issues/14793
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Currently if no project settings are provided this is the default value
for `main`, but if the caller providers a project_settings object
without this set it will cause a module load issue if the inline program
depends on other local python files. The current work-around is to put
all the code in one file or set `work_dir`.
Fixes#7960
## Checklist
- [X] I have run `make tidy` to update any new dependencies
- [X] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [X] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Co-authored-by: Paul Roberts <proberts@pulumi.com>
This PR replaces all the dependabot PRs with a single commit that
updates all relevant go.mod files.
This resolves 3 Dependabot alerts on golang.org/x/net including a
moderate severity alert.
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.
This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(https://github.com/googleapis/go-genproto/issues/1015)
As a result of this, some users may see the error:
```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```
Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.
This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).
The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
12374: whoami: Add --json flag, expose everything in Auto API r=abhinav a=mrod-io
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
* Adds `--json` flag to `pulumi whoami` (CLI)
* Adds `url` and `organizations` to WhoAmIResult in NodeJS, Go and Python (automation API)
I sanity tested the auto api in the 3 runtimes and the cli output looks like:
```bash
$ pulumi whoami -j
{
"user": "me",
"organizations": [
"me",
"company"
],
"url": "https://app.pulumi.com/me"
}
```
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Fixes # https://github.com/pulumi/pulumi/issues/10356
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
12397: Changelog and go.mod updates for v3.57.1 r=abhinav a=pulumi-bot
bors merge
Co-authored-by: Matthew Rodrigues <matthew.rodrigues@starburstdata.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: github-actions <github-actions@github.com>
12275: Add stack tag support for python automation api sdk r=justinvp a=mrod-io
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
Adds support for tagging stacks with the Python automation API.
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Fixes # (issue)
* https://github.com/pulumi/pulumi/issues/11936 (Python support)
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Matthew Rodrigues <matthew.rodrigues@starburstdata.com>
Upgrades all go.mod files to v0.7.0 of golang.org/x/net.
This will take care of the disparate dependabot updates we're receiving
for these files.
See also https://github.com/pulumi/pulumi/security/dependabot/151
Refs CVE-2022-41723
11543: Update Pulumi codegen to net6.0 r=Frassle a=Frassle
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
.NET Core 3.1 is out of support on December 13th:
https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-core
We've already stopped testing on .NET Core 3.1, and program gen has been targeting only 6.0 for a while now as well.
This updates sdkgen to only 6.0 as well.
We'll need to ensure the version number in pkg/codegen/dotnet/gen.go matches the version number that https://github.com/pulumi/pulumi-dotnet/pull/10 release as.
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Fraser Waters <fraser@pulumi.com>