Commit Graph

54 Commits

Author SHA1 Message Date
Thomas Gummerer c5889864c7
plugins: don't panic when the given path is not clean ()
When the user specifies a provider with a specific path in the
`Pulumi.yaml`, we later in the program assert that the path that was
passed in also matches with the path where we found the plugin. We do
this by using a string comparison, however that doesn't work if the path
the user passes is not clean, e.g. has a trailing slash, or has a double
slash, or some such.

Fix this by using `filepath.Clean` on the user supplied path, which
these things up.

I was also briefly wondering if this works properly if the user passes
in a path that is a symlink (it does), and wrote a test for that,
checking that behaviour.

Fixes https://github.com/pulumi/pulumi/issues/17130
2024-09-04 10:08:44 +00:00
Fraser Waters f8d05644e3
Fix GetPluginInfo with shimless project plugins ()
Before this fix GetPluginInfo would error because stat on the expected
`pulumi-resource-exe` file would fail (because it didn't exist). This
fixes it to fallback to looking at the folder instead.
2024-08-30 15:51:15 +00:00
Mikhail Shilkov d4f1cf5c87
URL-based plugin source overrides via env var ()
### Motivation

Pulumi plugin binaries can be downloaded by the CLI from multiple
sources. By default, it's downloaded from Pulumi's GitHub releases or
get.pulumi.com, but plugins can also specify their binary sources via
the `PluginDownloadURL` schema option. They can point to custom GitHub,
Gitlab, or HTTP locations.

Enterprise customers ask for a way to isolate the CLI from downloads
from random locations and to configure the CLI to go to their internal
pre-approved artefact location instead. This way, Pulumi can run in
"air-gapped" environments (which still have access to Cloud APIs, of
course).

Related issues:
- https://github.com/pulumi/pulumi/issues/14459
- https://github.com/pulumi/pulumi/issues/16240

Currently, there is a basic mechanism to do so via the variable
`pluginDownloadURLOverrides`, but it has two major limitations:
- The variable value is set via a compile-time flag, so it requires a
custom build of the CLI
- The overrides are based on the plugin name, so the rules must be
defined without access to the original URL, which makes it hard to
provide universal rules and still distinguish between first-party,
public third-party, or private in-house plugins
- We ignore overrides for all plugins that have `PluginDownloadURL` set
- Overrides can set a plugin replacement redirect only to HTTP(s)
addresses

### Proposal

This PR makes two sets of changes:

1. It allows passing overrides via the
`PULUMI_PLUGIN_DOWNLOAD_URL_OVERRIDES` environment variable. The
compile-time flag is still supported, but the env var takes priority.

More configuration levers could be supported, but it not clear if we
have good ones until [Support .pulumirc file for global
config](https://github.com/pulumi/pulumi/issues/13484) is implemented. I
don't expect users to want to set this via their stack configs, but I'm
curious what others think. In any case, more sources can be added later.

2. The overrides now apply based on the original download URL, not just
on plugin names. Actually, it's the base URL of a download source that
is passed to the regexp matcher. Examples of possible options are:

- `github://api.github.com/pulumi/pulumi-xyz` for a first-party plugin
(note that we don't pass `get.pulumi.com`
- `github://api.github.com/pulumiverse/pulumi-grafana` for a community
plugin that sets `PluginDownloadURL`
- `gitlab://gitlab-host/proj-name` for a community plugin hosted on
Gitlab
    - `https://example.com/downloads/` for HTTP sources

So, the override
`^github://api.github.com/pulumi/pulumi-xyz=https://example.com/downloads/pulumi-xyz/`
will override the single provider URL from our GitHub releases to the
given HTTP location.

On top of that, regular expressions may contain name groups to capture
and use templated values. For example,
`^github://api.github.com/(?P<org>[^/]+)/(?P<repo>[^/]+)=https://example.com/downloads/${org}/${repo}`
captures any GitHub plugin and redirects it to its corresponding HTTP
location. Group indices are also supported: the above override can also
be written as
`^github://api.github.com/(?P<org>[^/]+)/(?P<repo>[^/]+)=https://example.com/downloads/$1/$2`,
with `$0` meaning the full match.

The override URLs have the same semantics as `PluginDownloadURL`, so
they can point to GitHub, Gitlab, HTTP, or anything we introduce in the
future.

### Impact

Technically, this is a breaking change, because name-based overrides
will stop working. However, we are fairly certain that we have a single
customer using the existing compile-time approach, and they indicated
that they don't need the name-based overrides if they have URL-based
overrides. I reviewed this PR with them and made sure they can migrate
immediately after the change is released.

Backwards compatibility is slightly tricky, because we'd need to keep
name-based override _and_ not applying them to third-party plugins. But
we can do it if necessary.

Resolve 
2024-07-26 10:37:09 +00:00
Mikhail Shilkov 642cb5b5c7
Revert "Prefer pluginDownloadURLOverrides over PluginDownloadURL specified in the package" ()
Reverts 
Resolves https://github.com/pulumi/pulumi/issues/16316
2024-06-04 17:37:34 +00:00
Mikhail Shilkov 3e0aedeee2
Prefer pluginDownloadURLOverrides over PluginDownloadURL specified in the package ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

Overriding plugin download URLs with compilation flags was originally
added in . Its intent was allowing our customers to override
download locations for all plugins, so that only trusted pre-approved
plugins could be downloaded.

Since then, we've added `PluginDownloadURL` for a package, which is the
default URL for that package's binary if it's shipped outside our Pulumi
org. Currently, `PluginDownloadURL` takes precedence over
`pluginDownloadURLOverrides`, which means it's impossible to override
third-party package binary locations.

This PR changes plugin source resolution to flip the priority of those
two. If an override matches regex, its URL will take priority over the
default `PluginDownloadURL` specified in the package.

I have added tests to verify `pluginDownloadURLOverrides` with and
without `PluginDownloadURL`. The second one fails before my change.

Resolves 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-13 14:35:44 +00:00
Germán Lena d7f24dfcfb
Refactor: move plugin kind to apitype ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

This PR moves PluginKind to apitype to prevent circular dependencies
when adding apitype as a dependency of the workspace module.
It also re-exports PluginKind to keep backward compatibility

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-25 17:30:30 +00:00
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Fraser Waters 0f4ddc2ccf
Use EqualError/ErrorContains instead of Error ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This is a pass over all of /sdk to replace asserts that just checked we
had an error with asserts for what the error value is.

Just checking for an error is a weak test that can result in error paths
being broken and tests not detecting it.
2023-12-08 06:40:14 +00:00
Fraser Waters 1f28042b2d
Prefer stable plugin release to pre-releases ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/14680.

Updated the plugin logic (which we still use when no explict version is
given) to prefer selecting a stable version over a pre-release version
when no explict requested version is given.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-03 09:15:07 +00:00
Fraser Waters a6050592f1
Support {NAME} in plugin download url ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Allows `${NAME}` in the download url template string. 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-30 09:29:12 +00:00
Joe Duffy 96a9a77167
Policy remediations feature ()
This PR implements the new policy transforms feature, which allows
policy packs to not only issue warnings and errors in response to policy
violations, but actually fix them by rewriting resource property state.
This can be used, for instance, to auto-tag resources, remove Internet
access on the fly, or apply encryption to storage, among other use
cases.
2023-10-09 18:31:17 +00:00
Abhinav Gupta 79ffe88b6d
ci(lint): Upgrade to latest golangci-lint ()
Upgrade to latest version of golangci-lint
and fix or opt-out the issues it caught.

The false positives are:

```
sdk/go/common/workspace/plugins_test.go:512:3: G101: Potential hardcoded credentials (gosec)
pkg/resource/deploy/builtins.go:180:2: G101: Potential hardcoded credentials (gosec)
```

The fixed issues are:

```
pkg/resource/deploy/deploytest/pluginhost.go:440:16: G601: Implicit memory aliasing in for loop. (gosec)
                        Version:    &v.version,
                                    ^
pkg/engine/lifecycletest/alias_test.go:58:25: G601: Implicit memory aliasing in for loop. (gosec)
                        DeleteBeforeReplace: &r.deleteBeforeReplace,
                                             ^
```
2023-08-31 18:26:04 +00:00
Fraser Waters 0ade454f11
Allow language plugins to return plugin checksums ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Also fix an issue where if a platform was missing a checksum it would
error with "invalid checksum, expected , actual 01234".

None of the language runtimes yet return anything for this, but it's a
simple plumbing to expose it for the future.

We'll _probably_ start adding checksums to the pulumi-plugin.json files,
and then GetRequiredPlugins can simply return that data.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-08-25 15:26:25 +00:00
Fraser Waters a691975202 Warn about ambient plugins loaded from $PATH
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.

While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.

This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.

Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.

Re-instates https://github.com/pulumi/pulumi/pull/13607 with a fix for
symlinks included.
2023-08-08 13:11:34 +01:00
Robbie McKinstry c8d331ca37 Check if Plugin is bundled before installing
This commit adds a check to `pulumi plugin install` to confirm
that the plugin is not bundled with Pulumi before performing the download.
Without this check, the CLI will produce a 404 error because the language
plugins aren't distributable independently of the CLI executable.

If PULUMI_DEV is set we'll skip the error and try to download anyway.
2023-08-07 15:33:12 +01:00
Kyle Dixler 86ebe1bbd3 Revert "Warn about ambient plugins loaded from $PATH" 2023-08-04 16:54:16 -07:00
Fraser Waters a5b1590499 Warn about ambient plugins loaded from $PATH
By default Pulumi will load ambient plugins from $PATH before looking in
the plugins directory or at bundled plugins.

While this is very useful for development it often causes confusion when
people have forgotten that they have plugins left on $PATH.

This makes the use of these $PATH plugins a diagnostic warning to try
and make that failure mode a little less silent.

Normal users shouldn't ever have plugins on $PATH and so won't see this
new warning.
2023-07-27 17:59:44 +01:00
Fraser Waters 753a21daa2 Fix lookup of side-by-side binaries when PULUMI_IGNORE_AMBIENT_PLUGINS is set 2023-07-19 11:47:00 +01:00
Fraser Waters 1949f3a920 Fix http source URL interpolation
Turns out this wasn't tested and regressed in
51cc6461df
when we started URL parsing the input string and so ended up with a text
string with "%7B" instead of "{" to replace.

This fixes the interpolation and changes a couple of the http tests to
actually use this feature so it's tested.
2023-07-10 11:05:30 +01:00
Lee Briggs f42268be21 add rate limited url to http requests 2023-04-24 16:16:35 +01:00
Abhinav Gupta f9458a3510
sdk/workspace/pluginDownloader: Re-use util/retry
The retry logic wants to make at most 5 attempts
(that's what the "limit" says)
but it actually makes 6 requests.

This fixes the off-by-one error
by switching to the re-usable util/retry package
instead of re-implementing retries here.

As with util/retry.Retryer,
pluginDownloader allows overriding time.After from tests,
so that we don't sleep in tests.
2023-03-24 18:11:40 -07:00
Abhinav Gupta e5aafcd382
refactor(sdk/workspace/DownloadToFile): Pull state into a struct
All state for DownloadToFile was in a bunch of anonymous functions
that called each other.
The only way to inject new options was via new positional parameters.

This turns the core logic of DownloadToFile into
a pluginDownloader struct,
with retry and wrapper injected as fields rather than parameters.
It further breaks out all anonymous functions in DownloadToFile
into methods on that struct.
Their order has been retained to make this easy to review.

This change contains no logic changes.
2023-03-24 18:07:42 -07:00
Abhinav Gupta 52f2bec40a
sdk/workspace: Add failing test case for DownloadToFile retries
Adds a failing test case for 
that validates the number of attempts we make
when downloading a plugin.

```
--- FAIL: TestDownloadToFile_retries (9.85s)
    plugins_test.go:961:
                Error Trace:    [..]/pulumi/sdk/go/common/workspace/plugins_test.go:961
                Error:          Not equal:
                                expected: 30
                                actual  : 5
                Test:           TestDownloadToFile_retries
                Messages:       number of attempts does not match number of requests
    plugins_test.go:927:
                Error Trace:    [..]/pulumi/sdk/go/common/workspace/plugins_test.go:927
                                                        [..]/pulumi/sdk/go/common/workspace/plugins_test.go:963
                Error:          Not equal:
                                expected: 5
                                actual  : 30
                Test:           TestDownloadToFile_retries
                Messages:       server received more requests than expected
```
2023-03-24 18:07:42 -07:00
Fraser Waters b0ab245078 Move automatic plugin-install to engine 2023-03-20 23:41:07 +00:00
Fraser Waters 67f26e5a51 Don't send empty token/Bearer headers 2023-03-09 09:43:04 +00:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Fraser Waters c4083d3821 Nicer github rate limit error 2023-03-02 21:33:38 +00:00
Fraser Waters 4ed7fb78bf Rename http source 2023-02-16 19:51:37 +00:00
Fraser Waters 99ebe52db6 Add gitlab source 2023-02-15 10:18:44 +00:00
Abhinav Gupta c97d9401f2
getDiffInfo: Fix slice length check
Checking if the slice is non-nil instead of checking length
causes breakage when we pre-allocate diffs based on `len(md.Diffs)`.
2023-01-11 21:53:05 -08:00
Robbie McKinstry 4959522a53
Repair tests expecting nil slices 2023-01-11 21:53:04 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Eng Zer Jun 48eff4676a
test: use T.TempDir to create temporary test directory
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-12-03 15:17:08 +08:00
Fraser Waters 6bfb453fcb Clean up tests 2022-10-02 08:34:01 +01:00
Zaid Ajaj 04a910088d Bring back auto-install plugins 2022-09-13 16:40:09 +02:00
Aaron Friel bc68b921e8
Revert "Install missing plugins when encountering them ()" ()
This reverts commit 0c753103a0.
2022-09-07 21:41:23 -07:00
Zaid Ajaj 0c753103a0
Install missing plugins when encountering them ()
* Install missing plugins when encountering them

* changelog entry

* Update CHANGELOG_PENDING.md

Co-authored-by: Fraser Waters <fraser@pulumi.com>

* lint

* Move auto install logic to getPluginInfoAndPath

* clean up missing plugin error and its corresponding tests

* Introduce PluginInstallError which give users more context and action they could perform in case auto plugin install fails

* lint

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2022-08-31 13:23:32 +02:00
Fraser Waters 1163c8b68c
Add support for SHA256 checksums in PluginSpec ()
* Add support for SHA256 checksums in PluginSpec

* Update sdk/go/common/workspace/plugins.go

Co-authored-by: Ian Wahbe <ian@wahbe.com>

* lint

Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-08-30 10:44:56 +01:00
Fraser Waters 6b496b0d18
Split PluginInfo in Info and Spec ()
Retry of https://github.com/pulumi/pulumi/pull/10492.

This time with a fixed and improved testDeletePlugin function.

This reverts commit 603d859126.
2022-08-26 15:51:14 +01:00
Anton Tayanovskyy 603d859126
Revert "Split PluginInfo in Info and Spec ()" ()
This reverts commit b81207f98c.
2022-08-25 14:56:23 -04:00
Fraser Waters b81207f98c
Split PluginInfo in Info and Spec ()
PluginSpec is used to specifiy a plugin, and is what is passed to things
like "Install". PluginInfo is used to refer to an installed plugin, and
so has extra data like file sizes, and time stamps, but does not include
things like plugin download url.
2022-08-25 12:27:28 +01:00
Eng Zer Jun 19d84ef1f3
test: use `T.Setenv` to set env vars in tests ()
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-24 10:41:44 +01:00
Harry bb84532fe6
Plugin Link ()
* demo

* modifications for serialization

* Provisionally changed plugins from map to array

* warnings for duplicate

* avoid breaking change

* avoid null pointer dereference

* added test

* Delete Pulumi.yaml

* ensurePluginsAreInstalled

* lint

* reworked NewContext and added kind

* auto-detect current project for YAML

* lint

* removed debug statement

* automatically modify local paths

* typo

* First return value of GetPluginPath was never used

* Always use the path returned from getPluginInfoAndPath in GetPluginPath

Also assert that Path is the correct directory for PluginInfo.

* address comments

* added language, analyzers

* path tweaks and cosmetic changes

* changelog + tweaks

* changed NewContextWithRoot to accept plugins instead of project

* Fix TestUnmarshalProjectWithProviderList

* Fix NewContext

* Fix comment

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2022-07-22 14:17:43 +01:00
Fraser Waters a80fe65912
Add github:// download url ()
* Add github:// download url

This will allow pulumiverse plugins (or others) to use the github releases api:
```
% pulumi plugin install resource astra --server github://api.github.com/pulumiverse
[resource plugin astra-1.0.25] installing
Downloading plugin: 15.12 MiB / 15.12 MiB [=========================] 100.00% 5s
```

* Add to CHANGELOG

* ws

* lint
2022-06-29 20:15:01 +01:00
Komal 4f1403e489
Improve the missing plugin error to include the full name of the binary ()
* Improve the plugin error to include the full name of the binary

* update changelog

* fix second error message

* PR feedback updates

* lint

* GetPluginPath shouldn't return empty path and no error

Co-authored-by: komal <komal@pulumi.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2022-03-17 12:22:56 +00:00
Fraser Waters 874d127c83
Support plugins from private Pulumi repos ()
* Support plugins from private Pulumi repos

This uses the same environment vars that we we're using for experimental private repos. That is GITHUB_TOKEN or GITHUB_ACTOR and GITHUB_PERSONAL_ACCESS_TOKEN.

This allows us to develop plugins in private repos but continue to use our standard plugin flow.

* Add test

* Add to CHANGELOG

* Only use GITHUB_TOKEN

As described in https://docs.github.com/en/rest/overview/resources-in-the-rest-api#authentication we should be using the Authorization header.

* Fix tests

* Improve download tests

* lint

* Unauth tests need to make sure GITHUB_TOKEN is clear

* Update CHANGELOG

* Log a warning about GITHUB_PERSONAL_ACCESS_TOKEN
2022-03-14 21:14:36 +00:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Fraser Waters f9e03d1af5
Support looking up plugin version from GitHub ()
* Support looking up plugin version from GitHub

* PluginSource and testing

* Linting

* test GetLatestVersion

* Add to CHANGELOG

* Fix some logs

* lint
2022-03-03 09:22:21 +00:00
Patrizio Bruno fd25e56d8c
Add support for GitHub private releases as plugin source ()
* Add support for GitHub private releases as plugin source

* update CHANGELOG_PENDING.md

* fix: invert token and tokentype parms in call to authenticatedGetPluginResponse

* Update CHANGELOG_PENDING.md

Co-authored-by: Fraser Waters <frassle@gmail.com>

* refactor: more reusable/testable private github private release download

* lint: fix errors

* syntax: add missing newline

Co-authored-by: Fraser Waters <frassle@gmail.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2022-02-17 21:20:29 +00:00