Bumps google.golang.org/protobuf,
[golang.org/x/crypto](https://github.com/golang/crypto) and
[github.com/moby/moby](https://github.com/moby/moby).
Replaces #15688 -- running the acceptance tests on that PR is having
problems
Fixes#15674
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
This adds a new experimental feature to the Go SDK to register remote
transform functions. These are currently all prefixed 'X' to show
they're experimental (they can't be in their own package because of
circular dependencies).
These transform functions will run even for resources created inside
MLCs.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
github.com/golang/protobuf is marked deprecated and I was getting
increasingly triggered by the inconsistency of importing the `Empty`
type from "github.com/golang/protobuf/ptypes/empty" or
"google.golang.org/protobuf/types/known/emptypb" as "pbempty" or "empty"
or "emptypb". Similar for the struct type.
So this replaces all the Protobufs imports with ones from
"google.golang.org/protobuf", normalises the import name to always just
be the module name (emptypb), and adds the depguard linter to ensure we
don't use the deprecated package anymore.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Dependabot updated some references to this in
https://github.com/pulumi/pulumi/pull/15131. But missed a lot,
importantly it didn't update pkg or sdk which are the most important
modules in this repo.
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) to 0.17.0.
Replaces all the dependabot PRs in the repo with this single PR.
Also bumped `github.com/pulumi/pulumi/sdk/v3` in
`tests/integration/transformations/go/simple/go.mod` from v3.97.0 to
v3.98.0 to use esc v0.6.1, and avoid the appdash issue.
First step in addressing https://github.com/pulumi/pulumi/issues/14873
Note: I suspect I'll need to temporarily disable some codegen tests to
get this merged, and then once we release v3.98.0 and the next version
of esc, we can re-enable.
These changes add two commands for managing a stack's environments:
- `pulumi config env add`, which adds environments to a stack's import
list
- `pulumi config env rm`, which removes an environment from a stack's
import list
As implied by their paths, these commands hang off of a new sub-command
of `pulumi config`, `pulumi config env`.
From the usage:
* `pulumi config env add`
Adds environments to the end of a stack's import list. Imported
environments are merged in order per the ESC merge rules. The list of
stacks behaves as if it were the import list in an anonymous
environment.
* `pulumi config env rm`
Removes an environment from a stack's import list.
Each of these commands previews the new stack environment and shows the
environment definition. These commands print a warning if the stack's
environment does not define any of the `environmentVariables`, `files`,
or `pulumiConfig` properties.
<!---
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->
# Description
This exposes a GitRepo option to enable shallow cloning repositories.
This is helpful in cases where the repo has a large history.
## Checklist
- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
- [ ] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
These changes add support for ESC environments to the Pulumi CLI. This
involves two major changes:
- Support for the `env` subcommand
- Support for the `environment` stanza in stack config files
The former reuses the command from `esc` itself with a little
rebranding.
The latter adds support to stack config files for an `environment`
property of the form:
```yaml
environment:
- list
- of
- environment
- names
```
If this property is present in a stack's config file, the CLI will open
the and merge the listed environments during `pulumi up` et. al. If an
object-valued `pulumiConfig` property is present in the opened
environment, its values will be merged on top of the stack's config
prior to whatever operation is to be performed. If an object-valued
`environmentVariables` property is present inthe opened environment, its
values will be published as environment variables prior to the Pulumi
operation. Any values in the open environment's `pulumiConfig` or
`environmentVariables` that are marked as secret will be encrypted in
the resulting config and will be filtered from the command's logs.
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
These changes replace the idiosyncratic implementation of some of the
config Map and Value APIs with (hopefully) more straightforward code.
The fundamental representation of a config.Value remains a (value,
secure, object) tuple, where value is either a plain, possible-encrypted
string value or the JSON encoding of an object value. All operations on
values that need to observe the object value itself still decode the
JSON representation into a richer representation. This richer
representation, however, is no longer composed of `any` values: instead,
it is composed of `object` values. These values contain a restricted set
of types and directly track whether or not their contents are a secure
string value. The object-based representation allows for much clearer
implementations of the marshaling and traversal code without breaking
compatibility.
In addition to the new implementation for config.Value, these changes
add a config.Plaintext type that represents a plaintext config value. A
Plaintext value can be created manually or by decrypting a Value, and
can be encrypted and converted to a Value. This allows for more natural
creation and manipulation of config values.
Update to a newer version to avoid
[CVE-2022-32149](https://github.com/advisories/GHSA-69ch-w2m2-3vjp) in
versions < v0.3.8.
See #8667 on why we originally pinned the version. Looks like unpinning
altogether still pulls in older versions, so keeping it pinned for now.
Switch the cmdutil.ReadConsole and cmdutil.ReadConsoleNoEcho functions
to use the bubbletea library to render the prompt,
using the textinput widget provided by the accompanying bubbles library.
The resulting input widgets support arrow keys, back space,
and some basic readline-style bindings including Ctrl-A, Alt-B, etc.
I went through all uses of ReadConsole or ReadConsoleNoEcho.
Only the one in new.go had a non-compliant prompt that I had to adjust.
Note: One divergence in behavior I opted for was that
password prompts will echo '*' characters as the user is typing
and then no echo once they've accepted or canceled the value.
Previously, the prompt did not echo anything in either case.
<details>
<summary>
Introduction if you're unfamiliar with bubbletea
</summary>
bubbletea operates by modeling the widget state as
an immutable data structure that receives messages for events.
On receiving a message (key press, e.g.) the model's Update method
returns a new model instance representing its new state.
Update may also optionally return additional commands for the program,
e.g. stop running, or print something and move on.
The model's View method returns what should be drawn in the terminal
based on the model's current state.
This programming model makes it reasonably straightforward to unit test
some of the core functionality of independent widgets
as demonstrated in this PR.
</details>
Resolves#1565
---
Demos:
<details>
<summary>Plain text</summary>
![prompt-plain](https://github.com/pulumi/pulumi/assets/41730/66258fc8-f772-4d01-bc7c-1f7b116aebaa)
</details>
<details>
<summary>Secret</summary>
![prompt-secret](https://github.com/pulumi/pulumi/assets/41730/372f862e-9186-4d47-ba7d-0107c47f52f6)
</details>
<details>
<summary>Secret prompt with padding</summary>
![prompt-secret-2](https://github.com/pulumi/pulumi/assets/41730/e9b7c253-4c9d-4235-9fa6-197aa0522033)
</details>
This adds support for SSH-style Git URLs, enabling folks to use
private repos for their templates.
For instance,
$ pulumi new git@github.com:acmecorp/templates/website
will now work as intended.
The `ssh_config` library handles finding the relevant SSH key for the
given host.
If the SSH key is protected by a password, the user will be prompted to
supply the password on-demand. (It is memoized to avoid asking multiple
times, as the template workflow requires using it more than once.) To
avoid prompting, the `PULUMI_GITSSH_PASSPHRASE` env var can be set.
Fixes#4872 and #5007.
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.
This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(https://github.com/googleapis/go-genproto/issues/1015)
As a result of this, some users may see the error:
```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```
Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.
This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).
The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
The Node SDK creates a tmp file when packing programs, to house
the packed tarball. We previously used uuid to create our own
tmpfile with a random name. This commit uses os.tempfile instead.
The stdlib function os.tempfile is race safe, collision-proof, and
defaults to a tmp directory, which the OS is free to automatically
clean up in case Pulumi fails to.
This changes codegen to be invoked via gRPC from pkg, rather than
invoking pkg/codegen directly.
Consider it a proof-of-concept for moving codegen to a gRPC interface
without the worries of forwards-backwards compatability (because we ship
language plugins at a fixed version side-by-side to users).
The Go language host cannot resolve dependencies or plugins if a Pulumi
program vendors its dependencies.
BACKGROUND
The GetRequiredPlugins and GetProgramDependencies methods of the Go
language host rely on the following two commands:
go list -m -mod=mod all
go list -m -mod=mod ...
# '...' means current module and its descendants
GetRequiredPlugins additionally searches the source directories for each
returned module for pulumi-plugin.json files at a pre-determined paths.
$module/pulumi-plugin.json
$module/go/pulumi-plugin.json
$module/go/*/pulumi-plugin.json
This works for most Pulumi programs, except those that vendor private
dependencies with 'go mod vendor'.
For those programs, the above commands fail because -mod=mod forces them
to run in module mode, and their private dependencies are not accessible
in module mode (because they are not exposed publicly).
We use the -mod=mod flag to force 'go list' to run in module mode
because otherwise, it will automatically use vendor mode if a vendor
directory is present. However, in vendor mode, the two 'go list'
commands above are not supported.
The following links add more context on why, but in short:
vendor does not have enough information for the general 'go list'.
- https://stackoverflow.com/a/60660593,
- https://github.com/golang/go/issues/35589#issuecomment-554488544
In short,
- list all with -mod=mod fails because the dependency is private
- list without -mod=mod will use vendor mode
- vendor mode doesn't support the listing all
SOLUTION
Drop the -mod=mod flag so that 'go list' can decide whether to run in
module mode or vendor mode.
However, instead of running it with 'all' or '...',
pass in a list of dependencies extracted from the go.mod.
go list -m import/path1 import/path2 # ...
This operation is completely offline in vendor mode
so it can list information about private dependencies too.
This alone isn't enough though because in vendor mode,
the JSON output does not include the module root directory.
E.g.
% go list -mod=vendor -json -m github.com/pulumi/pulumi/sdk/v3
{
"Path": "github.com/pulumi/pulumi/sdk/v3",
"Version": "v3.55.0",
"GoVersion": "1.18"
}
# Versus
% go list -mod=mod -json -m github.com/pulumi/pulumi/sdk/v3
{
"Path": "github.com/pulumi/pulumi/sdk/v3",
"Version": "v3.55.0",
"Time": "2023-02-14T11:04:22Z",
"Dir": "[...]/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.55.0",
"GoMod": "[...]/go/pkg/mod/cache/download/github.com/pulumi/pulumi/sdk/v3/@v/v3.55.0.mod",
"GoVersion": "1.18"
}
Therefore, we have to manually calculate the path for each module root.
That's easy enough: vendor/$importPath.
Lastly, since GetProgramDependencies only needs a dependency list,
it now extracts information from the go.mod without calling 'go list'.
TESTING
Adds a variant of the test added in #12715 that verifies the
functionality with vendoring. It removes the sources for the
dependencies to simulate private dependencies. The new test fails
without the accompanying change.
The fix was further manually verified against the reproduction included
in #12526.
% cd go-output
% pulumi plugin rm -a -y
% pulumi preview
Previewing update (abhinav):
Downloading plugin: 15.19 MiB / 15.19 MiB [=========================] 100.00% 0s
[resource plugin random-4.8.2] installing
Type Name Plan
+ pulumi:pulumi:Stack go-output-abhinav create
+ └─ random:index:RandomId rrr create
Resources:
+ 2 to create
% pulumi plugin ls
NAME KIND VERSION SIZE INSTALLED LAST USED
random resource 4.8.2 33 MB 26 seconds ago 26 seconds ago
TOTAL plugin cache size: 33 MB
Note that the version of random (4.8.2) is what's specified in the
go.mod, not the latest release (v4.12.1).
% grep pulumi-random go.mod
github.com/pulumi/pulumi-random/sdk/v4 v4.8.2
With the plugin downloaded, I ran this again without an internet
connection.
% pulumi preview
Previewing update (abhinav):
Type Name Plan
+ pulumi:pulumi:Stack go-output-abhinav create
+ └─ random:index:RandomId rrr create
Resources:
+ 2 to create
This means that if the dependencies are vendored, and the plugin is
already available, we won't make additional network requests, which also
addresses #7089.
Resolves#12526Resolves#7089
12197: deps: Upgrade to pgregory.net/rapid v0.5 r=abhinav a=abhinav
The 0.5 release of rapid exposes a generics-based API
instead of `interface{}` and casting everywhere.
This makes for much cleaner usage.
There are a handful of cases where strongly typed generators,
e.g. `Generator[bool]`, need to be turned into `interface{}` (`any`),
which is doable with `AsAny()`.
API changes:
The only non-test changes to the SDK package are in
go/common/resource/testing, which contains testing utiltiies.
Functions that previously returned the old, untyped `Generator`
now return a strongly typed `Generator[T]`.
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Upgrades all go.mod files to v0.7.0 of golang.org/x/net.
This will take care of the disparate dependabot updates we're receiving
for these files.
See also https://github.com/pulumi/pulumi/security/dependabot/151
Refs CVE-2022-41723
The 0.5 release of rapid exposes a generics-based API
instead of `interface{}` and casting everywhere.
This makes for much cleaner usage.
There are a handful of cases where strongly typed generators,
e.g. `Generator[bool]`, need to be turned into `interface{}` (`any`),
which is doable with `AsAny()`.
API changes:
The only non-test changes to the SDK package are in
go/common/resource/testing, which contains testing utiltiies.
Functions that previously returned the old, untyped `Generator`
now return a strongly typed `Generator[T]`.
Raise the 'go' directive for top-level Go modules to 1.18.
This is the minimum version of Go required by Pulumi.
Raising this directive allows us to use language features like generics
in the codbase.
Resolves#11798
In general, go-git can't clone from Azure DevOps, because the latter
requires the capabilities multi_ack and multi_ack_detailed, which aren't
implemented. However, there's now a workaround, which boils down to
this: pretend, for the initial clone, that those capabilities _are_
supported, and expect them not to be used.
(See https://github.com/go-git/go-git/pull/613 for more on this
workaround.)
Signed-off-by: Michael Bridgen <mbridgen@pulumi.com>
* Add NewUniqueName
NewUniqueHexV2 tried to generate deterministic names via sequence
numbers, but this was insufficent to generate names that worked for many
scenarios. We're going to try to generate names based on more sources of
data, but there's no need to keep updating providers and provider code
each time we change the data set. Instead we're going to pass the byte
slice to Check and use that as a random source, the engine will then be
free to change how those random bytes get generated (including possibly
just generating them completly randomly)
This adds a new function for generating unique names that will work with
that idea. It takes a byte slice and uses that to generate a random
number sequence to fill in the random part of the name. This new method
also takes a "charset" option as currently there's a number of resources
spread across different providers that have their own name generation
code due to them needing a different charset to [a-f0-9].
At a later point we can probably remove NewUniqueHexV2. While it's
technically part of our exported public API nothing should be using it
(and a search across github doesn't show any hits).
* Add to CHANGELOG
* Use frand