Replace pylint and black with Ruff. Ruff is a lot faster and combines
both linting and formatting. It also has [very good editor
integration](https://docs.astral.sh/ruff/editors/setup/) via
https://github.com/astral-sh/ruff-lsp.
The formatting is mostly compatible with black, and only introduces some
minor changes.
A lot of the removed `pylint: XXX` comments where noops when linting
with pylint, most likely to due to updates to pylint catching more false
positives.
---------
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Some of the linters we're using have an option to fix some of the lint
errors they are finding. This obviously only works for the simple cases,
and nothing too complex, but can still be useful.
Since there are a bunch of different tools we're using, it's not always
easy to remember which option needs to be passed to which tool. Add
`lint_fix` Makefile targets that pass these options to our linters where
available, to help fix these issues more easily.
Updates the type for the `outputs` property of the `StackReference`
class to be a `dict[str, Any]` instead of a `dict` (with unknown keys
and values). This change fixes the type error in the referenced issue.
Fixes#16955
Co-authored-by: Fraser Waters <fraser@pulumi.com>
Test code should be formatted and linted the same as library code. This
is the first step of that, simply including ./lib/test to the folder
that the black formatter runs on.
# Description
Provide a way for the Automation API to install the Pulumi CLI so that
Automation API can be used in a more standalone manner.
https://github.com/pulumi/pulumi/issues/14987
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Some `make` targets broke
when we turned pulumi-language-python into
an independent Go module.
It also has a different import path than it did before.
This updates the Makefile to match the new path
and uses `-C` to cd into that directory for relevant Go commands.
This commit modifies `setup.py` to use a Python variable as the source of the package version instead of a placeholder string and adds a fallback for the README definition. This makes the package installable via the `-e` flag directly from its source directory (`sdk/python/lib`), rather than having a build step and having to `-e` install the built directory (`sdk/python/env/src`).
* allow to override makefile variables
When packaging pulumi-python for www.nixos.org, there is no /bin/bash
available so allow for an alternative path to the shell.
Same for versions, PYPI version can be set via the environment variable
PULUMI_VERSION.
* Update sdk/nodejs/Makefile
Co-authored-by: Fraser Waters <frassle@gmail.com>
* Update sdk/go/Makefile
Co-authored-by: Fraser Waters <frassle@gmail.com>
Co-authored-by: Fraser Waters <frassle@gmail.com>
* `pip` & the `wheel` package are dependencies before lint/build.
* Fix a .py file committed without the execute bit
Used these commands with `watchexec` to make a variety of changes and
then backed them out to find the minimum viable patch.
`watchexec "git clean -xfd; make lint"`
`watchexec "git clean -xfd; make build_package"`
And finally:
`watchexec "git clean -xfd; make test_all"`
Initially I thought the issue was that my distro only has
`/usr/bin/python3` as I was getting errors on running `python`, but it
looks like the issue was primarily activating the environment and making
sure dependencies were installed correctly.
There was an idempotency issue as well, where the commands would fail on
first run, but on second the `bdist_wheel` package would be set up and
they would succeed.
* Add coverage folder
* Adjust GHA to run Test target on Windows
* Extend error message on packages not found
* Try with path normalizer
* Edit Makefiles
* Skip SDK codegen tests on Windows
* Skip program gen tests on Windows
* Skip tests that started running on Windows and failing
* Fix non-compiling test
* Skip failing Windows HCL2 tests
* Fix lint
* Merged constants
* Skip one more test failing on Windows
* Disable cov-enabled builds on Windows
* Fix TestDeterminePulumiPackages on Windows
* Fix TestInstallCleansOldFiles test on Windows
* Fix TestCreatingProjectWithPulumiBackendURL on Windows
* Weaken TestAbbreviateFilePath to pass on Windows
* Fix TestDepRootCalc on Windows
* Fix LocalUrl() to be sensible on Windows
* Fix Go lint issue
* Windows fix for TestComments
* Cross-ref skip tracking issue
* Cross-ref issue to fix asset_test skips
* More cross-ref issues
* Use choco not chocolatey
* Use yarn run to ensure the right tools are selected
* Revert python3->python change in common.mk
* In CI context, use python not python3
* More randomness in temp folder names to aoid Windows collisions
* Go lint
We've had a few issues in the recent past related to pipenv oddities in
CI which lead us to temporarily globally install the Python SDK in CI.
This change removes the use of pipenv in favor of Python's built-in
venv and avoids globally installing the Python SDK.
These changes add support for gathering code coverage data during tests.
For tests that do not involve the Pulumi CLI, this is straightforward: all of
the ecosystems we target already support gathering coverage data, and we follow
the rules accordingly. Support for each language is broken out into its own
commit.
For tests that do involve the Pulumi CLI, the picture is a bit more complicated.
Go does not make it trivial to perform a coverage-instrumented build (go build
does not have a -cover flag, for example). In lieu of official support, we abuse
go test -c and TestMain to produce a build of the CLI that supports collecting
and reporting coverage data.
* Experiment with gotestsum and test timings
* Fix to locating the helper script
* Fix the code for installing gotestsum
* Try alternative installation method
* Use go to compute test stats; Python fails parsing time values
* Try version without v
* Try with fixed gorelaser config
* Fix test time correlation
* Try a stable test stat sort finally
* Use more accurate test duration aggregation
* Include python and auto-api tests in the Go timing counts
* Bring back TESTPARALLELISM
* Fix test compilation
* Only top 100 slow tests
* Try to fracture build matrix to fan out tests
* Do not run Publish Test Results on unsuppored Mac
* Auto-create test-results-dir
* Fix new flaky test by polling for logs
* Try to move native tests to their own config
* Actually skip
* Do not fail on empty test-results folder
* Try again
* Try once more
* Integration test config is the crit path - make it smaller
* Squash underutilized test configurations
* Remove the test result summary box from PR - counts now incorrec
* Remove debugging step
Fixes:#6565
As part of #6460, the logic for determing the version of the build was
moved to be a dependency on pulumictl.
Unfortunately, the homebrew installs use the "make dist" command to
build + install Pulumi to the user maching and as that would have a
dependency on pulumictl and it not existing on the user machine, it
would pass an empty version to the ldflag
This then manifested to the user as:
```
▶ pulumi version
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.22.0', run
$ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
```
We are able to mitigate this behaviour by bringing back the get-version
script and using that script as part of the make brew installation
We can see that the versions are the same between the 2 different
installation techniques
```
make dist <------- uses pulumict
DIST:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=2.24.0-alpha.1616029310+787eb70a" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
DIST:
BUILD:
```
```
make brew <----- uses the legacy script
▶ make brew
BREW:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=v2.24.0-alpha.1616029310+g787eb70a2" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
BREW:
```
A full post mortem will be carried out to ensure we mitigate these
types of errors going forward and that we are able to better test
these types of situations
Implement GetRequiredPlugins for Python, which determines the plugins
required by the program.
Also, if the `virtualenv` runtime option is set, and the specified
virtual directory is missing or empty, automatically create it and
install dependencies into it.
The previous attempt to allow this didn't actually allow it, so this is
take two. As part of the previous attempt, I thought after tweaking the
test I had observed the test failing, and then succeeding after making
the product changes, but I must have been mistaken.
It turns out that our existing mocks tests weren't running at all
because of a missing `__init__.py` file. Once the missing `__init__.py`
is added, the tests run, but other tests ("test mode" tests) fail
because the code that creates the mocks and resources will run during
test discovery, and setting the mocks modifies global state.
To address the test issue, I've moved the mocks tests into their own
`test_with_mocks` package that can be run separately from other tests.
And addressed the original issue, by creating a root Stack resource if
one isn't already present when the mocks are set.