We're in the process of renaming the main branch in our repos to `main`
from `master`. I'm planning to do homebrew-tap next, and this `git push`
will need to be updated to keep that working.
This is meant to be merged right after the branch is renamed in that
repo.
These changes contain some minor refactorings to conditionally disable
the use of packages that are cannot be built for `GOOS=js GOARCH=wasm`.
With these edits, `pkg/display` can be built targeting WASM.
These changes act as a safeguard to ensure that we are not adding
additional code that will _prevent_ building `pkg/display` for WASM
targets. They are not sufficient to produce a version of the display
renderer that is appropriate for actual use in a WASM environment:
- The current renderer API is not well-suited for use outside the
context of the CLI
- The current event stream format has no versioning data
- Actually building this code into a WASM module results in an
unpleasantly large file (70M uncompressed, 13M gzipped)
These changes also add a size gate for the built WASM module. The gate
is set to the 110% of the size of the WASM module as of this commit. Our
goal is to lower the size of the WASM module over time; as we do so we
will tighten this gate.
Part of #13258.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This reverts commit 75340dd942.
Fixes https://github.com/pulumi/pulumi/issues/16018.
This re-enables the locking and race detection. The locking is more
finely scoped to not be held over provider methods like Read/Update.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
A couple of our tests now generate multi-line `requirements.txt` whose
order is dependent on that in which we enumerate packages when testing.
This PR changes this so that we always sort them, meaning we can
consistently check against snapshotted results (as opposed to having
flaky tests that only pass when the generated order matches the one we
happened to snapshot last).
Fixes https://github.com/pulumi/pulumi/issues/16113
Currently the sign workflow only depends `build-release`. However it
also needs the SDKs being built, so add a dependency on that.
This usually works because the release builds takes longer, so it just
works, but we should make the dependencies explicit.
Fixes https://github.com/pulumi/pulumi/issues/16016
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This reverts the sdk and pkg changes from commit
655b76d030.
It also disables race detection from all builds and tests.
Fixes https://github.com/pulumi/pulumi/issues/15991.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
The version we currently have doesn't support Go 1.22 properly, so it
throws a bunch of warnings locally when trying to run it with the latest
Go version installed. Just running the latest version locally also
doesn't quite work, since it throws a bunch of errors from the
perfsprint linter, which seems to have gotten stricter.
Upgrade to the latest version of golangci-lint, and fix all the errors
we're getting from it. Mostly done via `perfsprint -fix`, with some
manual changes that `perfsprint -fix` wouldn't touch.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Combining #15120 and #15124 and including fixes to allow tests to pass.
Enable race detection in the binary we're using for integration tests.
This will allow us to catch more data races before they get into any
release. This does mean the binary we're using for integration tests is
slightly different from the binary we're releasing, however that's
already the case as we're running a binary with coverage enabled for
them. Later we rebuild the binary we're actually releasing.
This requires us to fix the race between snapshot code, display code,
and the step executor. I've done that by adding a lock to the State
struct. This does not feel great, but it's a quick way to fix this and
get race detection running (and unblocks #15871 which was also hitting
the race detector because it started pulling snapshot code into unit
tests as well). There's probably a more principled overhaul that doesn't
require locking at this level.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
In the on-push job, we are already on `master`. Therefore diff
`master...HEAD` will never result in any changes. Also we need to
deepen the fetch here so we have an actual previous commit to compare
with.
In addition to that, it looks like this actually results in a string
output not a boolean, so we need to check against that, or the
condition will always evaluate to true.
I'm actually still not quite sure this is doing the right thing, but
I'll keep an eye on it.
Currently we run update-homebrew-tap in parallel to the other jobs. That
means it could complete before we updated the blobs on S3, and before
the version number on pulumi.com/pulumi-version is updated.
That in turn can make the CI fail, raising a P1.
Since all this happens within a pretty sort time, the easiest way to
work around this is to just do the homebrew tap update last, after
pulumi.com/pulumi-version is updated. This delays the homebrew tap
update, but only by a short amount of time.
Fixes https://github.com/pulumi/homebrew-tap/issues/24
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This helps the providers team not go get P1 in our inbox when the
downstream test fail by distinguishing tests from "production" runs such
as upgrading to a known-good version of the bridge .
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Before every CI job we currently check whether the Go cache is primed,
and if not compile all the tests to prime it. However that check is
currently wrong. `steps.setup-go.outputs.cache-hit` is a boolean
variable, however we compare it against a string 'true'. Since we are
comparing for inequality, this check is always true, so we are always
priming the cache, even if it is already primed.
Fix this check, so we can only prime the cache when it needs to be and
don't waste extra CI time to prime the cache when it already might be.
h/t again to @iamricard for noticing that we are taking a long time to
prime the cache in some cases.
Currently we do our dev releases while running CI for the merge queue,
which adds to the time it takes for things to get through the merge
queue, slowing our development process down.
However there's no really good reason to do dev releases pre-merge. The
pre-merge checks are there to make sure the main branch is always green.
We already know the binaries will be able to be built successfully since
we do that for integration tests. And if the network fails for uploading
the binaries we can always just retry the CI job, which we can do for a
on-push CI job, but isn't useful for the merge CI job since the PR will
already have been taken out of the queue.
h/t to @iamricard who suggested this during a conversation this weekend.
I really don't see a reason not to do this. We should probably do the
same for the release builds, but it's probably best to start with dev
releases since those are less important, and this should be hopefully
not very controversial.
/xref https://github.com/pulumi/pulumi/issues/15416
We disabled this because the way we were tagging the version was
incorrect as we were tagging a version outside of the main branch, but
the way we fixed it the first time around (trying to push the commits
directly to master) didn't work because of branch protections.
Fix this now by tagging pkg during merge, using the version we get from
the commit message.
I think we can actually test this one by creating a fake "Changelog and
go.mod updates for v0.0.9999" PR, merging that, and deleting that tag
again afterwards. It's going to be after the fact testing, but at least
we should be able to do it before the release.
---------
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR adds a prefix to the titles of PRs created in provider repos for
downstream tests. It'd make it easier for provider maintainers to
recognise test PRs from real upgrades and help with not automatically
raising P1 issues on the PR CI failing.
We added a similar change to the bridge:
https://github.com/pulumi/pulumi-terraform-bridge/pull/1726
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes # (issue)
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This commit contains two changes:
1. Temporarily comment out jobs that have already successfully run in
the `release.yml` workflow. See
https://github.com/pulumi/pulumi/actions/runs/8117091017/job/22188577398
2. Add a way to manually dispatch a release via
`on-release-manual-dispatch.yml` workflow. The new file is a copy of
`on-release.yml`, but setup to allow manual trigger with input
variables, rather than being triggered by a release.
This change will be reverted after v3.108.1 is fully released.
This PR reverts the change from
https://github.com/pulumi/pulumi/pull/15515 because it's currently
broken, and even if we fix the problem where it is broken, I'm not sure
we can make it work without unchecking the "Do not allow bypassing the
above settings" in `master`'s branch protection rules, which I think we
do want to keep checked (some more details in #15554).
An additional commit also temporarily disables tagging `pkg/<version>`
in the workflow. Instead, that will have to be done manually as part of
the release process when the change lands in `master`. As a fast follow,
I'd like to see if there's a way we can automate adding the tag after
the change lands in master, perhaps if the PR has a certain label or is
named a certain way?
I mainly want to do this because it looks like we may have a Node.js SDK
regression and will need to do a quick v3.108.1 release, and I want to
make sure we can get that out without a broken workflow and needing to
go through a bunch of manual temporary workarounds to unblock that
release.
@tgummerer, what do you think?
Fixes#15554
(If we merge this, we should re-open
https://github.com/pulumi/pulumi/issues/15458 to track fixing this)
This commit contains two changes:
1. Temporarily comment out jobs that have already successfully run in
the `release.yml` workflow. See
https://github.com/pulumi/pulumi/actions/runs/8106099187
2. Add a way to manually dispatch a release via
`on-release-manual-dispatch.yml` workflow. The new file is a copy of
`on-release.yml`, but setup to allow manual trigger with input
variables, rather than being triggered by a release.
This change will be reverted after v3.108.0 is fully released.
Post release, we need to update the changelog (clear out all the pending
entries), and update the go.mod in pkg, and also tag the latest version
of that. Currently we create a new PR for that. However since we are
using squash merges, and need to do the tagging while creating the PR,
we're tagging a commit that doesn't end up on the main branch. This
means the Go tooling doesn't work properly with pkg. See also
https://github.com/pulumi/pulumi/issues/15458.
There is nothing really we are doing with that PR other than merging it
(in fact queue-merge is set to true, so it should be queued and merged
automatically, but that doesn't seem to work).
Because of that, we can just tag the release, and push it straight to
the main branch instead. This removes one human step from the release
process and makes the Go tooling work correctly.
The only potential downsides of this change are:
- a small race condition, if a commit gets merged just before we do the
rebase we might end up failing that and the release workflow fails. This
is very unlikely, since the release process should be relatively quick.
- we need to adjust the branch protection rules for pulumi-bot so it's
able to push to the main branch directly. This should not be a problem,
since it's an automated tool, so all of its potential activity has been
reviewed by humans beforehand.
Curious especially if @justinvp has any thoughts on this, since he's
mostly taking care of the release process.
Fixes https://github.com/pulumi/pulumi/issues/15458
Currently we release a new dev SDK when nothing changed because the
condition here is set incorrectly. We want to create a new release when
the exit code is non-zero, so we need to check for `if ! git diff`.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
Initial support pnpm. Note that this does not support pnpm workspaces
yet.
This also does not handle passing the package manager through from
`pulumi new`. Once a user manually runs pnpm, creating a pnpm-lock.yaml,
we'll detect that and pnpm.
Fixes https://github.com/pulumi/pulumi/issues/15455
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
For testing purposes we released a new dev SDK on every merge. However
this is unnecessary once dev releases are working, and just clogs up the
release page/uses extra storage space for the package hosts.
Only publish new dev releases when something actually changed in the
SDK, otherwise we can skip publishing.
In https://github.com/pulumi/pulumi/pull/15299 we changed the versioning
scheme of dev releases slightly, as npm and python have slightly
different requirements. However this also means that names of CLI dev
releases changed, and thus downloads failed.
Fix that by updating the latest-dev-version file to the new versioning
scheme.
Fixes https://github.com/pulumi/pulumi/issues/15443
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Publish dev versions of the python and node SDKs, so we can consume them
pre-release and make our releases more stable.
Also clean up the publish_npm.sh script a little bit, removing now
unused cruft.
resourcedocsgen has been remved from the pulumi/docs repo in
https://github.com/pulumi/docs/pull/10009, so now the related CI build
fails. Get it from the pulumi/registry repository instead, where it
lives now.
Note that this requires https://github.com/pulumi/docs/pull/10053 to be
merged first to update the script to no longer try to build the tool.
I'm not sure why the CI job didn't break earlier, the first time I
noticed it being broken was in
https://github.com/pulumi/pulumi/pull/14156.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
This TODO has been there since this
6ef7ef64d6 was merged to the main branch,
but the flag was always true, and the comment also tells us to set it to
true. Setting it to true is still the right thing, so let's just get rid
of the confusing comment.
Just a minor cleanup I noticed while working on some CI stuff.
I was watching the merge queue quite closely today, and it's continually
stuck waiting to run jobs on MacOS runners. However when the jobs are
running on them they tend to be faster than e.g. Windows runners, often
running the tests assigned to them in 2 min, vs. 8 min on windows.
In addition we're spending quite a bit of time to setup the actual
running of tests, and by spreading the tests out over multiple runners
that time multiplies. Given the macos runners are the most expensive
ones that probably also adds up.
Given all that, we can reduce the number of MacOS runners we're using,
while not trading that off against longer overall CI times, as those are
limited by slower jobs.
Unfortunately this results in some duplication in the ci job, but I
think the tradeoff here might be worth it.
(Also happy to hear if we think this isn't worth the extra messiness in
the GitHub actions files, as usually the merge queue isn't as bad as it
is today, so this isn't that much of an issue)
---------
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
In preparation for supporting Python 3.12...
As of Python 3.12, `setuptools` is no longer installed by default into a
virtual environment created with `python -m venv`
(https://github.com/python/cpython/issues/95299). It must be explicitly
installed via `python -m pip install setuptools`.