Commit Graph

86 Commits

Author SHA1 Message Date
Julien P a59b694515
Query language runtime for options during “pulumi new” ()
# Description

Fixes https://github.com/pulumi/pulumi/issues/16309

During `pulumi new` we query the language runtime using the new
`RuntimeOptionsPrompts` RPC call to get additional prompts to ask the
user.

<img width="900" alt="Screenshot 2024-06-07 at 14 28 58"
src="https://github.com/pulumi/pulumi/assets/387068/e68ef702-978b-47f7-9d4b-afdf10409ed8">

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!-- av pr metadata
This information is embedded by the av CLI when creating PRs to track
the status of stacks when using Aviator. Please do not delete or edit
this section of the PR.
```
{"parent":"master","parentHead":"","trunk":"master"}
```
-->

---------

Co-authored-by: Will Jones <will@sacharissa.co.uk>
Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>
2024-06-17 17:10:55 +00:00
Pat Gavlin c376f9c728
[chore] Update esc to v0.9.1 ()
Bring in the latest additions to the CLI.
2024-06-05 06:22:01 +00:00
dependabot[bot] c9b5c90c84
Bump the go_modules group across 2 directories with 1 update ()
Bumps the go_modules group with 1 update in the /pkg directory:
[golang.org/x/net](https://github.com/golang/net).
Bumps the go_modules group with 1 update in the /sdk directory:
[golang.org/x/net](https://github.com/golang/net).

Updates `golang.org/x/net` from 0.23.0 to 0.25.0
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d27919b57f"><code>d27919b</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="e0324fcdb5"><code>e0324fc</code></a>
http2: use net.ErrClosed</li>
<li><a
href="b20cd5933a"><code>b20cd59</code></a>
quic: initiate key rotation earlier in connections</li>
<li><a
href="f95a3b3a48"><code>f95a3b3</code></a>
html: fix typo in package doc</li>
<li><a
href="0a24555f5c"><code>0a24555</code></a>
http/httpguts: speed up ValidHeaderFieldName</li>
<li><a
href="ec05fdcd71"><code>ec05fdc</code></a>
http2: don't retry the first request on a connection on GOAWAY
error</li>
<li><a
href="b67a0f0535"><code>b67a0f0</code></a>
http2: send correct LastStreamID in stream-caused GOAWAY</li>
<li><a
href="a130fcc1c1"><code>a130fcc</code></a>
quic: don't consider goroutines running when tests start as leaked</li>
<li><a
href="7bbe32058a"><code>7bbe320</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.23.0...v0.25.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `golang.org/x/net` from 0.23.0 to 0.25.0
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d27919b57f"><code>d27919b</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="e0324fcdb5"><code>e0324fc</code></a>
http2: use net.ErrClosed</li>
<li><a
href="b20cd5933a"><code>b20cd59</code></a>
quic: initiate key rotation earlier in connections</li>
<li><a
href="f95a3b3a48"><code>f95a3b3</code></a>
html: fix typo in package doc</li>
<li><a
href="0a24555f5c"><code>0a24555</code></a>
http/httpguts: speed up ValidHeaderFieldName</li>
<li><a
href="ec05fdcd71"><code>ec05fdc</code></a>
http2: don't retry the first request on a connection on GOAWAY
error</li>
<li><a
href="b67a0f0535"><code>b67a0f0</code></a>
http2: send correct LastStreamID in stream-caused GOAWAY</li>
<li><a
href="a130fcc1c1"><code>a130fcc</code></a>
quic: don't consider goroutines running when tests start as leaked</li>
<li><a
href="7bbe32058a"><code>7bbe320</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.23.0...v0.25.0">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts page](https://github.com/pulumi/pulumi/network/alerts).

</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2024-05-21 05:23:56 +00:00
Justin Van Patten 61c5b03dc7
Bump go modules ()
This is a replacement of https://github.com/pulumi/pulumi/pull/16043,
with an additional commit that includes the changes after running `make
tidy`.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-25 14:30:00 +00:00
Thomas Gummerer 1339f96833
automation: only read complete lines before trying to deserialize ()
When tailing the event log in automation API we currently have nothing
that makes sure we read only complete lines. This means if the OS
happens to flush an incomplete line for whatever reason (or the Go JSON
encoder does, which we're using to write these lines), we might read a
line that is incompletely written, and thus will fail to JSON decode it.

Since the JSON encoder always writes a newline at the end of each
string, we can also make sure that the line we read ends with a newline
and otherwise wait for the rest of the line to be written.

The library we use in Go provides a convenient setting for this, while
in python and nodejs we need to add some code to do this ourselves.

Fixes https://github.com/pulumi/pulumi/issues/15235
Fixes https://github.com/pulumi/pulumi/issues/15652
Fixes https://github.com/pulumi/pulumi/issues/9269 (This is closed
already, but never had a proper resolution afaics)
Fixes https://github.com/pulumi/pulumi/issues/6768

It would be nice to add a typescript test here as well, but I'm not sure
how to do that without marking the readLines function non-private. But I
don't know typescript well, so any hints of how to do that would be
appreciated!

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-26 14:32:56 +00:00
Justin Van Patten 14babed82b
Bump google.golang.org/protobuf, golang.org/x/crypto, and github.com/moby/moby ()
Bumps google.golang.org/protobuf,
[golang.org/x/crypto](https://github.com/golang/crypto) and
[github.com/moby/moby](https://github.com/moby/moby).

Replaces  -- running the acceptance tests on that PR is having
problems

Fixes 

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-17 22:20:32 +00:00
Fraser Waters ae161d6758
Go SDK transform support ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->


This adds a new experimental feature to the Go SDK to register remote
transform functions. These are currently all prefixed 'X' to show
they're experimental (they can't be in their own package because of
circular dependencies).

These transform functions will run even for resources created inside
MLCs.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-02-27 13:00:45 +00:00
Fraser Waters 6162d16eb2
Move goversion out of sdk/common to the go language host ()
Another small part we can pull out of sdk/go/common.
2024-01-17 14:56:18 +00:00
Fraser Waters 72bddd809f
Update github.com/cloudflare/circl to v1.3.7 ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Dependabot updated some references to this in
https://github.com/pulumi/pulumi/pull/15131. But missed a lot,
importantly it didn't update pkg or sdk which are the most important
modules in this repo.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-16 08:59:57 +00:00
Justin Van Patten 37e6ad44d0
Upgrade go-git to v5.11.0 ()
Bumps github.com/go-git/go-git/v5 to 5.11.0 to address
https://github.com/go-git/go-git/security/advisories/GHSA-mw99-9chc-xw7r

Co-authored-by: Roy Reznik <roy@wiz.io>
2024-01-02 18:41:06 +00:00
Justin Van Patten 53244f09ae
Bump golang.org/x/crypto to 0.17.0 ()
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) to 0.17.0.

Replaces all the dependabot PRs in the repo with this single PR.

Also bumped `github.com/pulumi/pulumi/sdk/v3` in
`tests/integration/transformations/go/simple/go.mod` from v3.97.0 to
v3.98.0 to use esc v0.6.1, and avoid the appdash issue.
2023-12-20 09:14:29 +00:00
Justin Van Patten 9853228bc9
Update esc to v0.6.2 ()
Update to the latest version of esc.
2023-12-20 04:15:34 +00:00
Justin Van Patten b0c276ece0
Update esc to v0.6.1 ()
First step in addressing https://github.com/pulumi/pulumi/issues/14873

Note: I suspect I'll need to temporarily disable some codegen tests to
get this merged, and then once we release v3.98.0 and the next version
of esc, we can re-enable.
2023-12-19 08:05:16 +00:00
Fraser Waters 075c024f26
Update go.mod for pulumi/appdash ()
Fixes https://github.com/pulumi/pulumi/issues/14725.
2023-12-05 19:53:45 +00:00
Fraser Waters d078735823
Reimport appdash from our mirror ()
Fixes https://github.com/pulumi/pulumi/issues/14646.
2023-11-30 14:21:35 +00:00
Pat Gavlin 064fb93587
[esc] Add commands for managing stack environments ()
These changes add two commands for managing a stack's environments:

- `pulumi config env add`, which adds environments to a stack's import
list
- `pulumi config env rm`, which removes an environment from a stack's
import list

As implied by their paths, these commands hang off of a new sub-command
of `pulumi config`, `pulumi config env`.

From the usage:

* `pulumi config env add`

Adds environments to the end of a stack's import list. Imported
environments are merged in order per the ESC merge rules. The list of
stacks behaves as if it were the import list in an anonymous
environment.

* `pulumi config env rm`

Removes an environment from a stack's import list.

Each of these commands previews the new stack environment and shows the
environment definition. These commands print a warning if the stack's
environment does not define any of the `environmentVariables`, `files`,
or `pulumiConfig` properties.
2023-11-22 05:04:14 +00:00
Bryce Lampe cbcad3277e
Allow shallow clones for local workspaces ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This exposes a GitRepo option to enable shallow cloning repositories.
This is helpful in cases where the repo has a large history.


## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-01 17:21:52 +00:00
Justin Van Patten 7f2555444d
bump google.golang.org/grpc from 1.57.0 to 1.57.1 ()
This PR replaces all the dependabot PRs with a single commit that
updates all relevant go.mod files.

This resolves a high severity dependabot alert.
2023-10-28 15:56:28 +00:00
Justin Van Patten 9d653491f6
[chore] update esc ()
Pin to v0.5.6 to pick up the CLI doc generation fix, and other
improvements.

Fixes 
2023-10-19 19:42:54 +00:00
Pat Gavlin 7e189e3a78
[chore] update esc ()
Pin to v0.5.2 to pick up a couple of bug fixes and a new command-line
option for `env init`.
2023-10-11 16:57:30 +00:00
Pat Gavlin 4d3b82cb9f
[cli] Add support for environments ()
These changes add support for ESC environments to the Pulumi CLI. This
involves two major changes:

- Support for the `env` subcommand
- Support for the `environment` stanza in stack config files

The former reuses the command from `esc` itself with a little
rebranding.

The latter adds support to stack config files for an `environment`
property of the form:

```yaml
environment:
  - list
  - of
  - environment
  - names
```

If this property is present in a stack's config file, the CLI will open
the and merge the listed environments during `pulumi up` et. al. If an
object-valued `pulumiConfig` property is present in the opened
environment, its values will be merged on top of the stack's config
prior to whatever operation is to be performed. If an object-valued
`environmentVariables` property is present inthe opened environment, its
values will be published as environment variables prior to the Pulumi
operation. Any values in the open environment's `pulumiConfig` or
`environmentVariables` that are marked as secret will be encrypted in
the resulting config and will be filtered from the command's logs.
2023-10-10 01:35:39 +00:00
Joe Duffy 96a9a77167
Policy remediations feature ()
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
2023-10-09 18:31:17 +00:00
Pat Gavlin 6756c12fd0
[config] Clean up implementation ()
These changes replace the idiosyncratic implementation of some of the
config Map and Value APIs with (hopefully) more straightforward code.

The fundamental representation of a config.Value remains a (value,
secure, object) tuple, where value is either a plain, possible-encrypted
string value or the JSON encoding of an object value. All operations on
values that need to observe the object value itself still decode the
JSON representation into a richer representation. This richer
representation, however, is no longer composed of `any` values: instead,
it is composed of `object` values. These values contain a restricted set
of types and directly track whether or not their contents are a secure
string value. The object-based representation allows for much clearer
implementations of the marshaling and traversal code without breaking
compatibility.

In addition to the new implementation for config.Value, these changes
add a config.Plaintext type that represents a plaintext config value. A
Plaintext value can be created manually or by decrypting a Value, and
can be encrypted and converted to a Value. This allows for more natural
creation and manipulation of config values.
2023-10-09 04:51:21 +00:00
Justin Van Patten 03def5365e
[sdk/go] Update pinned version of golang.org/x/text ()
Update to a newer version to avoid
[CVE-2022-32149](https://github.com/advisories/GHSA-69ch-w2m2-3vjp) in
versions < v0.3.8.

See  on why we originally pinned the version. Looks like unpinning
altogether still pulls in older versions, so keeping it pinned for now.
2023-09-19 23:02:51 +00:00
Abhinav Gupta b51caa6ab4
cmdutil.ReadConsole[NoEcho]: Use bubbletea ()
Switch the cmdutil.ReadConsole and cmdutil.ReadConsoleNoEcho functions
to use the bubbletea library to render the prompt,
using the textinput widget provided by the accompanying bubbles library.
The resulting input widgets support arrow keys, back space,
and some basic readline-style bindings including Ctrl-A, Alt-B, etc.

I went through all uses of ReadConsole or ReadConsoleNoEcho.
Only the one in new.go had a non-compliant prompt that I had to adjust.

Note: One divergence in behavior I opted for was that
password prompts will echo '*' characters as the user is typing
and then no echo once they've accepted or canceled the value.
Previously, the prompt did not echo anything in either case.

<details>

  <summary>
  Introduction if you're unfamiliar with bubbletea
  </summary>

  bubbletea operates by modeling the widget state as
  an immutable data structure that receives messages for events.
  On receiving a message (key press, e.g.) the model's Update method
  returns a new model instance representing its new state.
  Update may also optionally return additional commands for the program,
  e.g. stop running, or print something and move on.
  The model's View method returns what should be drawn in the terminal
  based on the model's current state.
This programming model makes it reasonably straightforward to unit test
  some of the core functionality of independent widgets
  as demonstrated in this PR.

</details>

Resolves 

---

Demos:

<details>
  <summary>Plain text</summary>
  

![prompt-plain](https://github.com/pulumi/pulumi/assets/41730/66258fc8-f772-4d01-bc7c-1f7b116aebaa)
</details>

<details>
  <summary>Secret</summary>


![prompt-secret](https://github.com/pulumi/pulumi/assets/41730/372f862e-9186-4d47-ba7d-0107c47f52f6)
</details>

<details>
  <summary>Secret prompt with padding</summary>


![prompt-secret-2](https://github.com/pulumi/pulumi/assets/41730/e9b7c253-4c9d-4235-9fa6-197aa0522033)
</details>
2023-08-30 17:08:44 +00:00
Abhinav Gupta 91a079851b
deps: Upgrade google.golang.org/{genproto, grpc}
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(https://github.com/googleapis/go-genproto/issues/1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker ().

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
2023-07-27 16:24:33 -07:00
Abhinav Gupta f59ab49fc5
deps(go): Upgrade to grpc 1.56.1
Upgrades to gRPC Go 1.56.1 to resolve the influx of dependabot PRs.
Supersedes all dependabot PRs created for this.

Fixes CVE-2023-32731
2023-07-06 09:04:16 -07:00
Robbie McKinstry 73ff332f5e
Use os.tmpfile instead of creating our own.
The Node SDK creates a tmp file when packing programs, to house
the packed tarball. We previously used uuid to create our own
tmpfile with a random name. This commit uses os.tempfile instead.
The stdlib function os.tempfile is race safe, collision-proof, and
defaults to a tmp directory, which the OS is free to automatically
clean up in case Pulumi fails to.
2023-05-30 13:58:02 -04:00
Pat Gavlin a8f41f031b [sdk] Update uniseg
The latest version of this module dramatically improves its allocation
volume and offers a friendlier API for measuring string width.
2023-05-25 22:24:13 -07:00
Fraser Waters 7e485f5d3d Plumb codegen rpc into nodejs 2023-05-25 14:56:45 +01:00
Abhinav Gupta 52a47d6295
all: cloudflare/circl 1.1.0 => 1.3.3
Upgrade version of cloudflare/circl to pick up important fixes
and supersede a bunch of dependabot PRs.

Addresses CVE-2023-1732
2023-05-11 13:51:01 -07:00
Fraser Waters 1a38eadc69 gRPC for GenerateProject/Program/Package
This changes codegen to be invoked via gRPC from pkg, rather than
invoking pkg/codegen directly.

Consider it a proof-of-concept for moving codegen to a gRPC interface
without the worries of forwards-backwards compatability (because we ship
language plugins at a fixed version side-by-side to users).
2023-05-06 13:14:59 +01:00
Abhinav Gupta 33b5ad6527
feat(go/host): Support vendored dependencies
The Go language host cannot resolve dependencies or plugins if a Pulumi
program vendors its dependencies.

BACKGROUND

The GetRequiredPlugins and GetProgramDependencies methods of the Go
language host rely on the following two commands:

    go list -m -mod=mod all
    go list -m -mod=mod ...
    # '...' means current module and its descendants

GetRequiredPlugins additionally searches the source directories for each
returned module for pulumi-plugin.json files at a pre-determined paths.

    $module/pulumi-plugin.json
    $module/go/pulumi-plugin.json
    $module/go/*/pulumi-plugin.json

This works for most Pulumi programs, except those that vendor private
dependencies with 'go mod vendor'.
For those programs, the above commands fail because -mod=mod forces them
to run in module mode, and their private dependencies are not accessible
in module mode (because they are not exposed publicly).

We use the -mod=mod flag to force 'go list' to run in module mode
because otherwise, it will automatically use vendor mode if a vendor
directory is present. However, in vendor mode, the two 'go list'
commands above are not supported.
The following links add more context on why, but in short:
vendor does not have enough information for the general 'go list'.

- https://stackoverflow.com/a/60660593,
- https://github.com/golang/go/issues/35589#issuecomment-554488544

In short,

- list all with -mod=mod fails because the dependency is private
- list without -mod=mod will use vendor mode
- vendor mode doesn't support the listing all

SOLUTION

Drop the -mod=mod flag so that 'go list' can decide whether to run in
module mode or vendor mode.
However, instead of running it with 'all' or '...',
pass in a list of dependencies extracted from the go.mod.

    go list -m import/path1 import/path2 # ...

This operation is completely offline in vendor mode
so it can list information about private dependencies too.

This alone isn't enough though because in vendor mode,
the JSON output does not include the module root directory.
E.g.

    % go list -mod=vendor -json -m github.com/pulumi/pulumi/sdk/v3
    {
            "Path": "github.com/pulumi/pulumi/sdk/v3",
            "Version": "v3.55.0",
            "GoVersion": "1.18"
    }

    # Versus

    % go list -mod=mod -json -m github.com/pulumi/pulumi/sdk/v3
    {
            "Path": "github.com/pulumi/pulumi/sdk/v3",
            "Version": "v3.55.0",
            "Time": "2023-02-14T11:04:22Z",
            "Dir": "[...]/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.55.0",
            "GoMod": "[...]/go/pkg/mod/cache/download/github.com/pulumi/pulumi/sdk/v3/@v/v3.55.0.mod",
            "GoVersion": "1.18"
    }

Therefore, we have to manually calculate the path for each module root.
That's easy enough: vendor/$importPath.

Lastly, since GetProgramDependencies only needs a dependency list,
it now extracts information from the go.mod without calling 'go list'.

TESTING

Adds a variant of the test added in  that verifies the
functionality with vendoring. It removes the sources for the
dependencies to simulate private dependencies. The new test fails
without the accompanying change.

The fix was further manually verified against the reproduction included
in .

    % cd go-output
    % pulumi plugin rm -a -y
    % pulumi preview
    Previewing update (abhinav):
    Downloading plugin: 15.19 MiB / 15.19 MiB [=========================] 100.00% 0s
                                                                                    [resource plugin random-4.8.2] installing
         Type                      Name               Plan
     +   pulumi:pulumi:Stack       go-output-abhinav  create
     +   └─ random:index:RandomId  rrr                create

    Resources:
        + 2 to create

    % pulumi plugin ls
    NAME    KIND      VERSION  SIZE   INSTALLED       LAST USED
    random  resource  4.8.2    33 MB  26 seconds ago  26 seconds ago

    TOTAL plugin cache size: 33 MB

Note that the version of random (4.8.2) is what's specified in the
go.mod, not the latest release (v4.12.1).

    % grep pulumi-random go.mod
            github.com/pulumi/pulumi-random/sdk/v4 v4.8.2

With the plugin downloaded, I ran this again without an internet
connection.

    % pulumi preview
    Previewing update (abhinav):
         Type                      Name               Plan
     +   pulumi:pulumi:Stack       go-output-abhinav  create
     +   └─ random:index:RandomId  rrr                create

    Resources:
        + 2 to create

This means that if the dependencies are vendored, and the plugin is
already available, we won't make additional network requests, which also
addresses .

Resolves 
Resolves 
2023-04-24 09:49:16 -07:00
Kyle Dixler 3af78f9ca7
Bump go-git to v5.6.0 to remove cgo dependency fixing
pulumi-docker-containers builds.
2023-02-28 16:01:31 -08:00
bors[bot] f7ad50317f
Merge
12197: deps: Upgrade to pgregory.net/rapid v0.5 r=abhinav a=abhinav

The 0.5 release of rapid exposes a generics-based API
instead of `interface{}` and casting everywhere.
This makes for much cleaner usage.

There are a handful of cases where strongly typed generators,
e.g. `Generator[bool]`, need to be turned into `interface{}` (`any`),
which is doable with `AsAny()`.

API changes:
The only non-test changes to the SDK package are in
go/common/resource/testing, which contains testing utiltiies.
Functions that previously returned the old, untyped `Generator`
now return a strongly typed `Generator[T]`.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
2023-02-25 04:38:10 +00:00
Abhinav Gupta 8614885326
all(go.mod): Upgrade golang.org/x/net to v0.7.0
Upgrades all go.mod files to v0.7.0 of golang.org/x/net.
This will take care of the disparate dependabot updates we're receiving
for these files.

See also https://github.com/pulumi/pulumi/security/dependabot/151

Refs CVE-2022-41723
2023-02-17 11:06:15 -08:00
Abhinav Gupta 5e346dfac9
deps: Upgrade to pgregory.net/rapid v0.5
The 0.5 release of rapid exposes a generics-based API
instead of `interface{}` and casting everywhere.
This makes for much cleaner usage.

There are a handful of cases where strongly typed generators,
e.g. `Generator[bool]`, need to be turned into `interface{}` (`any`),
which is doable with `AsAny()`.

API changes:
The only non-test changes to the SDK package are in
go/common/resource/testing, which contains testing utiltiies.
Functions that previously returned the old, untyped `Generator`
now return a strongly typed `Generator[T]`.
2023-02-16 12:04:07 -08:00
Guillaume Truchot de868c8be3
chore: update `net` package to fix CVE-2022-27664
Upgrades golang.org/x/net to v0.5.0.
This addresses CVE-2022-27664
and switches to semver-ed releases of the package.
2023-02-08 12:32:32 -08:00
Michael Bridgen c581c95996 Tidy go mods after go-git upgrade
./scripts/tidy.sh

Signed-off-by: Michael Bridgen <mbridgen@pulumi.com>
2023-02-02 16:56:51 +00:00
Abhinav Gupta b199f6f4d3
go mod tidy
Run tidy.sh to update all go.mod/go.sum files.
2023-01-31 10:07:54 -08:00
Robbie McKinstry 68a91dfeff
Attempting to merge config for linting operations. 2023-01-30 17:30:15 -05:00
Aaron Friel 9f9989af97 Update Go gRPC, require impl to embed UnimplementedXServer 2023-01-05 18:00:16 -08:00
Aaron Friel 7d8e2f6e90 chore: Update dependencies 2022-12-10 14:26:14 -08:00
Aaron Friel 05a182f6eb Update YAML to 1.0.4 2022-12-07 13:25:30 -08:00
Fraser Waters 5ff65320e2 Use project schema to validate projects we load 2022-09-02 10:09:24 +01:00
Aaron Friel 0fc18cbafb
[cli] Improve Windows reliability with dependency update to ssh-agent ()
* [cli] Improve Windows reliability with dependency update to ssh-agent

* chore: changelog
2022-08-25 09:19:12 -07:00
Aaron Friel 89b08f3f1f
fix: Sporadic initialization panics in ssh-agent library () 2022-08-22 11:48:55 -07:00
Fraser Waters c9094f2ceb
Update go-git to v5 ()
* Update go-git to v5

* Add to CHANGELOG
2022-08-09 12:46:28 +01:00
Aaron Friel 8d83206498
[windows] replace ssh-agent with one that lazily loads dlls () 2022-07-22 11:50:38 -07:00
Anton Tayanovskyy eeac88294f
Update java to 0.5.0 ()
* Update java to 0.5.0

* Run make tidy
2022-07-14 17:01:37 -04:00