Commit Graph

34 Commits

Author SHA1 Message Date
Fraser Waters 3560333ae6
Clean up uses of .Error() ()
Combination of a few cleanups.

1. Don't call .Error() on errors that are being passed to "%s" format
functions. Format will call `Error()` itself.
2. Don't call assert.Error then assert.Equal/Contains, just use
assert.ErrorEqual/ErrorContains instead.
3. Use "%w" if appropriate, instead of "%v"/"%s".
2023-12-20 15:54:06 +00:00
Kyle Pitzen f82b7358f1
Adds remote .zip archive support to `pulumi new` ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi.ai/issues/363

This change introduces a new accepted suffix to `pulumi new` HTTP
arguments - `.zip`.

When a URL of the form `http[s]://<url>.zip` is provided to `pulumi
new`, we send a GET request to `<url>` (without the .zip suffix) with
the `accept` header set to `application/zip`. It is then up to whatever
is hosted at that location to correctly return a binary zip archive in
its response.

We then parse that response in memory and write each file to a tempdir
as we do with current templates, which can then be picked up by the
standard `pulumi new` templating logic.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-31 22:05:39 +00:00
Fraser Waters e4ea1ec548
Use fallback project name rather than "pulum" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/13946.

This also moves some of the logic from sdk/go/common to pkg/ as it's
only used be `new` and `up.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-18 18:01:37 +00:00
Bryce Lampe 9a8a677a39 [cli/new] Support SSH-style Git URLs
This adds support for SSH-style Git URLs, enabling folks to use
private repos for their templates.

For instance,

    $ pulumi new git@github.com:acmecorp/templates/website

will now work as intended.

The `ssh_config` library handles finding the relevant SSH key for the
given host.

If the SSH key is protected by a password, the user will be prompted to
supply the password on-demand. (It is memoized to avoid asking multiple
times, as the template workflow requires using it more than once.) To
avoid prompting, the `PULUMI_GITSSH_PASSPHRASE` env var can be set.

Fixes  and .
2023-08-02 09:29:34 -07:00
Kyle Dixler af16d45009
`pulumi new` now supports numeric stack names.
`pulumi new` now correctly serializes a stack name of 123456 as a string
in the `Pulumi.yaml` file. `123456` is interpreted as a number in YAML
by default.
2023-06-20 12:55:21 -07:00
Abhinav Gupta e7aa7c484f
sdk/go: Add tokens.ValidateProjectName
As pointed out in
https://github.com/pulumi/pulumi/pull/13078#discussion_r1227854722,
we repreatedly reimplement the project name validation logic.

This moves it to a central place: `tokens.ValidateProjectName`,
and re-uses it in existing places where copies were found.
2023-06-13 10:08:31 -07:00
Bryce Lampe aacf262c1f cli/new: Escape special characters in project description 2023-06-07 14:54:02 -07:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta cc32691bc1
chore(all): strings.Replace(..., -1) => strings.Replace(...)
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
2023-03-01 13:22:33 -08:00
Abhinav Gupta 7bddec255d
sdk/go: Prefer contract.Assertf over Assert
Migrates all uses of contract.{Assert, AssertNoError, Require} in sdk/
to the `*f` variants that are required to provide more error context.

Step towards deprecating non-f variants entirely.

For context, `contract.Require` is similar to `contract.Assert`,
except it has a required parameter name as an argument:

    func Require(cond bool, param string)
    func Requiref(cond bool, param string, msg string, args ...any)

It includes the parameter name in the error message by default,
so the `msg` and `args` should only describe the constraint
without naming the parameter.

Refs 
2023-02-15 10:22:43 -08:00
Fraser Waters eb4565936f Replace pkg/errors in sdk/go/common/workspace 2023-01-19 20:22:54 +00:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
aq17 183ee2eda7 Do not fail pulumi policy new on invalid template(s) 2022-11-22 17:14:35 -08:00
aq17 da17e3f357 Add broken templates to end of options 2022-11-22 15:06:41 -08:00
杨成锴 7b09ad142a chore: Update doc comments, coding style, fix lint
* about error string: error string should not be capitalized or end with punctuation mark
* apply suggestions from code review
* lint: fix format error and simplify the code

Co-authored-by: Fraser Waters <frassle@gmail.com>
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-10-13 13:50:49 -07:00
Fraser Waters 800e6b4b53 Wrap project load errors with the path that failed 2022-10-09 12:08:05 +01:00
Fraser Waters 3b97bf5788 Fix cloning from ADO repos
Small path bug, "git clone" by default clones to a new folder based on
the last part of the repo name. So running something like `pulumi new
https://fraser0275@dev.azure.com/fraser0275/fraser/_git/fraser` resulted
in us making a folder like /tmp/pulumi-templates-1234/fraser with all
the templates in it. That last part of the path was unexpected, because
when we use gogit it clones directly into the target directory we give
it.

Simple fix to just add `.` to the git clone command. I also added some
logging while I was about here, given it helped me work out this issue.
2022-10-07 08:27:46 +01:00
Eng Zer Jun 57b25b45d1
refactor: use `os.ReadDir` for lightweight directory reading
`os.ReadDir` was added in Go 1.16 as part of the deprecation of `ioutil`
package. It is a more efficient implementation than `ioutil.ReadDir`.

Reference: https://pkg.go.dev/io/ioutil#ReadDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-09-24 15:20:43 +08:00
avlllo 966508f6cd
Add token auth to git go module ()
* Add username/password support into git urls

* Add auth parser

* Add pending changelog

Co-authored-by: user <a.v.olshanskiy@tinkoff.ru>
Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-09-13 19:24:22 +02:00
Fraser Waters c9094f2ceb
Update go-git to v5 ()
* Update go-git to v5

* Add to CHANGELOG
2022-08-09 12:46:28 +01:00
Zaid Ajaj 868f22f4fc
Revert "Clone and test pulumi/templates from the current working branch ()" ()
This reverts commit d163a1de25.
2022-06-10 11:39:54 +01:00
Zaid Ajaj d163a1de25
Clone and test pulumi/templates from the current working branch ()
* Clone and test templates from the current working branch

* simplify the fix

* Remove redundant check

* fix formatting
2022-06-09 22:35:36 +02:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Ian Wahbe 7222e5570a
[cli] check main after master ()
* Allow specifying a branch with url#branch

* Probe for master and main

* Update CHANGELOG

* Fix linter errors

* Remove unnecessary feature

* Fix lint

* Update changelog to reflect new limited scope.

We only talk about the master -> main probing, because that is all the
PR does. It used to duplicate another feature.
2021-11-19 13:49:59 -08:00
Anton Tayanovskyy 3e1dc52851
Fixes 6608: unusable venv after template copy, python not executable ()
* Fixes 6608: unusable venv after template copy, python not executable

* Add changelog entry
2021-06-04 10:03:04 -04:00
svangordon-fruit 5e495e85e5
Clean the template dir if the remote has changed () 2021-04-16 13:51:42 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
James Nugent 172fdb2e0d
Make template repository URL and ref modifiable () 2021-03-18 16:31:37 +00:00
Joe Duffy 12e5e46c73
Handle blank project names in 'new' ()
The logic for validating prompted values in 'new' wasn't quite right,
leading to the possibility of creating Pulumi.yaml files with blank
project names.

This manifests in various ways and I've hit it a number of times
over the past few months because of the way we handle project/stack
name conflicts in 'new' -- which itself is a bit annoying too:

https://github.com/pulumi/pulumi/blob/master/pkg/cmd/pulumi/new.go#L206-L207

Because we substitue a default value of "", and because the prompting
logic assumed default values are always valid, we would skip validation
and therefore accept a blank Pulumi.yaml file.

This generates an invalid project which causes errors elsewhere, such as

    error: failed to load Pulumi project located at ".../Pulumi.yaml":
        project is missing a 'name' attribute

I hit this all the time with our getting started guide because I've
gone through it so many times and have leftover stacks from prior
run-throughs. I wouldn't be surprised if a lot of people hit this.

The solution here validates all values, including the default.

Note also that we failed to validate the value used by 'new --yes'
which meant you could bypass all validation by passing --yes, leading
to similar outcomes.

I've added a couple new tests for these cases. There is a risk we
depend on illegal default values somewhere which will now be rejected,
but that would seem strange, and assuming the tests pass, I would
assume that's not true. Let me know if that's wrong.

Fixes .
2020-10-05 13:40:24 -07:00
Levi Blackstone 8961f5b0ca
Rename RetrieveTemplate function to RetrieveGitFolder ()
The new name better reflects the functionality for external
consumers of this package.
2020-06-26 15:01:51 -06:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls ()

* Switch away from native grpc impl. ()

* Remove usage of the 'deasync' library from @pulumi/pulumi. ()

* Only retry as long as we get unavailable back.  Anything else continues. ()

* Handle all errors for now. ()


* Do not assume --yes was present when using pulumi in non-interactive mode ()

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes ()

Adjust C# generation

* Replace IDeployment with a sealed class ()

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty ()

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen ()
2020-04-14 09:30:25 +01:00
evanboyle 0e9c5989bb move pkg/util/gitutil -> sdk/go/common/util/gitutil 2020-03-18 15:52:09 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle 8fb3f428b0 move pkg/workspace -> sdk/go/common/workspace 2020-03-18 14:35:53 -07:00