When the user specifies a provider with a specific path in the
`Pulumi.yaml`, we later in the program assert that the path that was
passed in also matches with the path where we found the plugin. We do
this by using a string comparison, however that doesn't work if the path
the user passes is not clean, e.g. has a trailing slash, or has a double
slash, or some such.
Fix this by using `filepath.Clean` on the user supplied path, which
these things up.
I was also briefly wondering if this works properly if the user passes
in a path that is a symlink (it does), and wrote a test for that,
checking that behaviour.
Fixes https://github.com/pulumi/pulumi/issues/17130
Before this fix GetPluginInfo would error because stat on the expected
`pulumi-resource-exe` file would fail (because it didn't exist). This
fixes it to fallback to looking at the folder instead.
Adds the flag `--use-language-version-tools` to `pulumi install`. When
passed, and `pyenv` is installed, and a `.python-version` file is in the
project directory or any of its parent directories, Pulumi will install
the requested python version if it is not already installed.
`LanguageRuntime.InstallDependencies` now takes a struct
`InstallDependenciesRequest` as argument to make it easier to extend
this call with options.
See https://github.com/pulumi/pulumi-docker-containers/pull/232
---------
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
### Motivation
Pulumi plugin binaries can be downloaded by the CLI from multiple
sources. By default, it's downloaded from Pulumi's GitHub releases or
get.pulumi.com, but plugins can also specify their binary sources via
the `PluginDownloadURL` schema option. They can point to custom GitHub,
Gitlab, or HTTP locations.
Enterprise customers ask for a way to isolate the CLI from downloads
from random locations and to configure the CLI to go to their internal
pre-approved artefact location instead. This way, Pulumi can run in
"air-gapped" environments (which still have access to Cloud APIs, of
course).
Related issues:
- https://github.com/pulumi/pulumi/issues/14459
- https://github.com/pulumi/pulumi/issues/16240
Currently, there is a basic mechanism to do so via the variable
`pluginDownloadURLOverrides`, but it has two major limitations:
- The variable value is set via a compile-time flag, so it requires a
custom build of the CLI
- The overrides are based on the plugin name, so the rules must be
defined without access to the original URL, which makes it hard to
provide universal rules and still distinguish between first-party,
public third-party, or private in-house plugins
- We ignore overrides for all plugins that have `PluginDownloadURL` set
- Overrides can set a plugin replacement redirect only to HTTP(s)
addresses
### Proposal
This PR makes two sets of changes:
1. It allows passing overrides via the
`PULUMI_PLUGIN_DOWNLOAD_URL_OVERRIDES` environment variable. The
compile-time flag is still supported, but the env var takes priority.
More configuration levers could be supported, but it not clear if we
have good ones until [Support .pulumirc file for global
config](https://github.com/pulumi/pulumi/issues/13484) is implemented. I
don't expect users to want to set this via their stack configs, but I'm
curious what others think. In any case, more sources can be added later.
2. The overrides now apply based on the original download URL, not just
on plugin names. Actually, it's the base URL of a download source that
is passed to the regexp matcher. Examples of possible options are:
- `github://api.github.com/pulumi/pulumi-xyz` for a first-party plugin
(note that we don't pass `get.pulumi.com`
- `github://api.github.com/pulumiverse/pulumi-grafana` for a community
plugin that sets `PluginDownloadURL`
- `gitlab://gitlab-host/proj-name` for a community plugin hosted on
Gitlab
- `https://example.com/downloads/` for HTTP sources
So, the override
`^github://api.github.com/pulumi/pulumi-xyz=https://example.com/downloads/pulumi-xyz/`
will override the single provider URL from our GitHub releases to the
given HTTP location.
On top of that, regular expressions may contain name groups to capture
and use templated values. For example,
`^github://api.github.com/(?P<org>[^/]+)/(?P<repo>[^/]+)=https://example.com/downloads/${org}/${repo}`
captures any GitHub plugin and redirects it to its corresponding HTTP
location. Group indices are also supported: the above override can also
be written as
`^github://api.github.com/(?P<org>[^/]+)/(?P<repo>[^/]+)=https://example.com/downloads/$1/$2`,
with `$0` meaning the full match.
The override URLs have the same semantics as `PluginDownloadURL`, so
they can point to GitHub, Gitlab, HTTP, or anything we introduce in the
future.
### Impact
Technically, this is a breaking change, because name-based overrides
will stop working. However, we are fairly certain that we have a single
customer using the existing compile-time approach, and they indicated
that they don't need the name-based overrides if they have URL-based
overrides. I reviewed this PR with them and made sure they can migrate
immediately after the change is released.
Backwards compatibility is slightly tricky, because we'd need to keep
name-based override _and_ not applying them to third-party plugins. But
we can do it if necessary.
Resolve#16240
https://github.com/pulumi/pulumi/pull/16346 introduced the capability to
query the language runtime for additional prompts. We use this to let
the user pick a package manager among npm, yarn and pnpm during `pulumi
new` when using the nodejs runtime.
When there is no explicitly configured package manager, we re-use the
previous behaviour for determining the package manager (check
`PULUMI_PREFER_YARN` env variable, look for lock files).
Defaults to `npm` when running `new` in non-interactive mode.
# Description
When trying to install a plugin of type `tool` from GitHub, the Pulumi
convention is to have these plugins available in repositories named
`pulumi-tool-<name>` so that we can install them via the CLI as follows:
```
pulumi plugin install tool <name>
```
However, today this fails because we don't prefix the repository name
correctly with `"pulumi-tool-"`. This PR fixes that. Tested against
[Zaid-Ajaj/pulumi-tool-importer](https://github.com/Zaid-Ajaj/pulumi-tool-importer)
Also removes hardcoded plugin download URL for known converter plugins
of mine. These are moved to the `pulumi` organisation and no longer
require a separate URL.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
# Description
This PR refactors the existing Python dependency installation and
command running code to use the `Toolchain` interface. This will make it
possible to swap out the default Pip based toolchain for a Poetry based
toolchain.
Fixes https://github.com/pulumi/pulumi/issues/16285
Ref https://github.com/pulumi/pulumi/issues/15937
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
Overriding plugin download URLs with compilation flags was originally
added in #8798. Its intent was allowing our customers to override
download locations for all plugins, so that only trusted pre-approved
plugins could be downloaded.
Since then, we've added `PluginDownloadURL` for a package, which is the
default URL for that package's binary if it's shipped outside our Pulumi
org. Currently, `PluginDownloadURL` takes precedence over
`pluginDownloadURLOverrides`, which means it's impossible to override
third-party package binary locations.
This PR changes plugin source resolution to flip the priority of those
two. If an override matches regex, its URL will take priority over the
default `PluginDownloadURL` specified in the package.
I have added tests to verify `pluginDownloadURLOverrides` with and
without `PluginDownloadURL`. The second one fails before my change.
Resolves#16058
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This PR moves PluginKind to apitype to prevent circular dependencies
when adding apitype as a dependency of the workspace module.
It also re-exports PluginKind to keep backward compatibility
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes#15150
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Mostly for providers to experiment with, currently hidden behind
"PULUMI_DEV".
We are using our terminal in raw mode, which means we're not getting CRs
automatically added. Add one after a progress bar has finished to
improve the output slightly.
The final output after this looks like:
```
$ pulumi preview
Please choose a stack, or create a new one: dev
Previewing update (dev):
Downloading plugin: 18.39 MiB / 18.39 MiB [========================] 100.00% 15s
[resource plugin docker-3.6.1] installing
Downloading plugin: 20.19 MiB / 20.19 MiB [========================] 100.00% 15s
[resource plugin awsx-1.0.5] installing
Downloading plugin: 164.18 MiB / 164.18 MiB [======================] 100.00% 32s
[resource plugin aws-5.42.0] installing
[...]
```
Which seems slightly better, but not a huge improvement, and the
progress bar is unfortunately also still jumpy. I'll write down the rest
of my learnings here in https://github.com/pulumi/pulumi/issues/14250.
I'm not entirely sure it's even worth merging this, but I'm putting this
up as a PR for further discussion and to show where the investigation
led.
Combination of a few cleanups.
1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945
This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.
Most of these edits were made by running `perfsprint --fix`.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Fixes https://github.com/pulumi/pulumi/issues/14680.
Updated the plugin logic (which we still use when no explict version is
given) to prefer selecting a stable version over a pre-release version
when no explict requested version is given.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Allows `${NAME}` in the download url template string.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Also fix an issue where if a platform was missing a checksum it would
error with "invalid checksum, expected , actual 01234".
None of the language runtimes yet return anything for this, but it's a
simple plumbing to expose it for the future.
We'll _probably_ start adding checksums to the pulumi-plugin.json files,
and then GetRequiredPlugins can simply return that data.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.
While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.
This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.
Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.
Re-instates https://github.com/pulumi/pulumi/pull/13607 with a fix for
symlinks included.
This commit adds a check to `pulumi plugin install` to confirm
that the plugin is not bundled with Pulumi before performing the download.
Without this check, the CLI will produce a 404 error because the language
plugins aren't distributable independently of the CLI executable.
If PULUMI_DEV is set we'll skip the error and try to download anyway.
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.
While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.
This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.
Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.
Turns out this wasn't tested and regressed in
51cc6461df
when we started URL parsing the input string and so ended up with a text
string with "%7B" instead of "{" to replace.
This fixes the interpolation and changes a couple of the http tests to
actually use this feature so it's tested.
Maybe fixes https://github.com/pulumi/pulumi-terraform-bridge/issues/1209
Fixes https://github.com/pulumi/pulumi-terraform-bridge/issues/1200
When looking up schemas we would call into `defaultHost.InstallPlugin` to
install any missing plugins, and we would generally _not_ have version
information for that. Unfortunately `defaultHost.InstallPlugin` didn't
handle the case of version not being set, double unfortunately it also
didn't error.
This fixes that install path to call `GetLatestVersion` if version isn't
set.
The retry logic wants to make at most 5 attempts
(that's what the "limit" says)
but it actually makes 6 requests.
This fixes the off-by-one error
by switching to the re-usable util/retry package
instead of re-implementing retries here.
As with util/retry.Retryer,
pluginDownloader allows overriding time.After from tests,
so that we don't sleep in tests.
The pluginDownloader retries downloads with specialized logic,
and it relies on getHTTPResponse to make the HTTP requests.
However, getHTTPResponse retries the requests *it* makes as well.
This changes getHTTPResponse to not perform any retries,
and makes a copy of it, getHTTPResponseWithRetry, that does.
The pluginDownloader uses getHTTPResponse,
and other prior uses of getHTTPResponse have switched to the new one.
This resolves the multiplied retries for pluginDownloader,
however there's still an off-by-one error.
All state for DownloadToFile was in a bunch of anonymous functions
that called each other.
The only way to inject new options was via new positional parameters.
This turns the core logic of DownloadToFile into
a pluginDownloader struct,
with retry and wrapper injected as fields rather than parameters.
It further breaks out all anonymous functions in DownloadToFile
into methods on that struct.
Their order has been retained to make this easy to review.
This change contains no logic changes.
Migrates all remaining usages of
`contract.Assert*` and `contract.Require*` to the f variants,
which require adding meaningful error messages.
There were a couple cases where a `testing.T` or `testing.B`
was already available.
For those, this uses t.FailNow or require.NoError.
Refs #12132
Per team discussion, switching to gofumpt.
[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.
[1]: https://github.com/mvdan/gofumpt
See the full list of [Added rules][2], but it includes:
- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
`) {` on a separate line.
[2]: https://github.com/mvdan/gofumpt#Added-rules
gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.
gofumpt support is built into the tooling we use for development
so this won't change development workflows.
- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
(see [installation instrutions][3])
[3]: https://github.com/mvdan/gofumpt#installation
This change was generated by running:
```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```
The following files were manually tweaked afterwards:
- pkg/cmd/pulumi/stack_change_secrets_provider.go:
one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
`var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
long line because of error message
- pkg/backend/snapshot_test.go:
long line trying to assign three variables in the same assignment
I have included mention of gofumpt in the CONTRIBUTING.md.
12190: Rename http source r=Frassle a=Frassle
<!---
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->
Just a little tidy up some things I noticed while working on the gitlab source.
1. Rename pluginURLSource to httpSource. A plugin URL is _anything_ that is given as a pluginDownloadURL. This type only handles the downloads from http/https locations.
2. Make GetSource error for unrecognized source schemes. If we add new schemes in the future (fairly likely) if someone tries to use one of the new schemes on an old CLI they'll get a fairly comprehensible error that the scheme wasn't recognized rather than some cryptic error from however the http library fails trying to parse the strange (to it) url.
## Checklist
<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - Not really user facing.
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
<!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->
Co-authored-by: Fraser Waters <fraser@pulumi.com>
12180: sdk/go: Prefer contract.Assertf over Assert r=abhinav a=abhinav
Migrates all uses of contract.{Assert, AssertNoError, Require} in sdk/
to the `*f` variants that are required to provide more error context.
Step towards deprecating non-f variants entirely.
For context, `contract.Require` is similar to `contract.Assert`,
except it has a required parameter name as an argument:
func Require(cond bool, param string)
func Requiref(cond bool, param string, msg string, args ...any)
It includes the parameter name in the error message by default,
so the `msg` and `args` should only describe the constraint
without naming the parameter.
Refs #12132
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Migrates all uses of contract.{Assert, AssertNoError, Require} in sdk/
to the `*f` variants that are required to provide more error context.
Step towards deprecating non-f variants entirely.
For context, `contract.Require` is similar to `contract.Assert`,
except it has a required parameter name as an argument:
func Require(cond bool, param string)
func Requiref(cond bool, param string, msg string, args ...any)
It includes the parameter name in the error message by default,
so the `msg` and `args` should only describe the constraint
without naming the parameter.
Refs #12132
Enable the prealloc linter, which identifies slices
with a known capacity, but are not preallocated, which
results in unnecessary allocations and memcpys.
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:
ioutil.Discard => io.Discard
ioutil.NopCloser => io.NopCloser
ioutil.ReadAll => io.ReadAll
ioutil.ReadFile => os.ReadFile
ioutil.TempDir => os.MkdirTemp
ioutil.TempFile => os.CreateTemp
ioutil.WriteFile => os.WriteFile
This change switches all of these entities
across the repository.
Following this change,
the only references to ioutil are in schema files:
% rg -l ioutil
pkg/codegen/testing/test/testdata/aws-4.26.0.json
pkg/codegen/testing/test/testdata/aws-4.36.0.json
pkg/codegen/testing/test/testdata/aws-4.37.1.json
pkg/codegen/testing/test/testdata/aws-5.4.0.json
pkg/codegen/testing/test/testdata/aws-5.16.2.json
The bulk of this change was generated automatically
with manual touch ups afterwards.
11362: Library for environmental variable access r=iwahbe a=iwahbe
Add a library to encapsulate environmental variable declaration, lookup and parsing.
### Usage
Variables strongly typed and declared in the module scope like so:
```go
// Automatically prefixed with PULUMI_ by default
var Experimental = env.Bool("EXPERIMENTAL", "Enable experimental options and commands")
```
Values are accessed by calling `.Value()` on the module level variable:
```go
if Experimental.Value() { // .Value() returns a bool here
// do the new exiting thing
} else {
// do the well tested thing
}
```
Environmental variables can be predicated on other boolean type variables:
```go
var NewThingParam = env.String("NEW_THING", "pass this to the new thing",
env.Needs(Expiremental))
```
Predicated variables will only show up as set if there predicate is `true`:
```go
if v := NewThingParam.Value(); v != "" {
do_thing(v)
} else {
// do the well tested thing
}
```
The above `if` check is equivalent to
```go
if v = os.Getenv("PULUMI_NEW_THING"); v != "" && cmduilt.IsTruthy(os.Getenv("PULUMI_EXPERIMENTAL"))
```
This makes marking and unmarking a variable as experimental a 1 line change.
---
All declared variables can then be iterated on for documentation with `env.Variables()`. This is how we can use this lib to build documentation for our environmental variables.
### Assumptions
#### Immutability
The library assumes that environmental variables are inherited from the environment. Values are captured during program startup, changes to the environment are not observed. We are not resilient to environmental variables changing out from under us, so this complies with current usage. This assumption can be changed without breaking the API.
#### Known values
The library assumes that the names of environmental variables are known at compile time. If the variable to be looked up is derived from user input, a different mechanism should be used.
### Adoption
Adopting this system over our pulumi/pkg codebase will take a while. My plan is to merge in this PR without moving over our existing env vars from `os.Getenv`. We can then gradually move over existing env vars piecemeal in subsequent PRs. Any new env vars added will use the new system.
Prerequisite for https://github.com/pulumi/pulumi/issues/10747.
### Other Options - Viper
Viper looks like an excellent config retrieval package, but doesn't support at-site documentation. It also has less type safety then I would prefer as a top level api. If it would be helpful, we could switch the underlying mechanism from `os.Lookup` to `viper.Get*` without changing the API this library exposes. I don't think viper is necessary right now, but adopting this library doesn't preclude viper.
Co-authored-by: Ian Wahbe <ian@wahbe.com>