pulumi/pkg/util/gitutil/git_test.go

227 lines
7.1 KiB
Go
Raw Normal View History

Initial support for passing URLs to `new` and `up` (#1727) * Initial support for passing URLs to `new` and `up` This PR adds initial support for `pulumi new` using Git under the covers to manage Pulumi templates, providing the same experience as before. You can now also optionally pass a URL to a Git repository, e.g. `pulumi new [<url>]`, including subdirectories within the repository, and arbitrary branches, tags, or commits. The following commands result in the same behavior from the user's perspective: - `pulumi new javascript` - `pulumi new https://github.com/pulumi/templates/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/master/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates/javascript` To specify an arbitrary branch, tag, or commit: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates/javascript` Branches and tags can include '/' separators, and `pulumi` will still find the right subdirectory. URLs to Gists are also supported, e.g.: `pulumi new https://gist.github.com/justinvp/6673959ceb9d2ac5a14c6d536cb871a6` If the specified subdirectory in the repository does not contain a `Pulumi.yaml`, it will look for subdirectories within containing `Pulumi.yaml` files, and prompt the user to choose a template, along the lines of how `pulumi new` behaves when no template is specified. The following commands result in the CLI prompting to choose a template: - `pulumi new` - `pulumi new https://github.com/pulumi/templates/templates` - `pulumi new https://github.com/pulumi/templates/tree/master/templates` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates` Of course, arbitrary branches, tags, or commits can be specified as well: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates` This PR also includes initial support for passing URLs to `pulumi up`, providing a streamlined way to deploy installable cloud applications with Pulumi, without having to manage source code locally before doing a deployment. For example, `pulumi up https://github.com/justinvp/aws` can be used to deploy a sample AWS app. The stack can be updated with different versions, e.g. `pulumi up https://github.com/justinvp/aws/tree/v2 -s <stack-to-update>` Config values can optionally be passed via command line flags, e.g. `pulumi up https://github.com/justinvp/aws -c aws:region=us-west-2 -c foo:bar=blah` Gists can also be used, e.g. `pulumi up https://gist.github.com/justinvp/62fde0463f243fcb49f5a7222e51bc76` * Fix panic when hitting ^C from "choose template" prompt * Add description to templates When running `pulumi new` without specifying a template, include the template description along with the name in the "choose template" display. ``` $ pulumi new Please choose a template: aws-go A minimal AWS Go program aws-javascript A minimal AWS JavaScript program aws-python A minimal AWS Python program aws-typescript A minimal AWS TypeScript program > go A minimal Go program hello-aws-javascript A simple AWS serverless JavaScript program javascript A minimal JavaScript program python A minimal Python program typescript A minimal TypeScript program ``` * React to changes to the pulumi/templates repo. We restructured the `pulumi/templates` repo to have all the templates in the root instead of in a `templates` subdirectory, so make the change here to no longer look for templates in `templates`. This also fixes an issue around using `Depth: 1` that I found while testing this. When a named template is used, we attempt to clone or pull from the `pulumi/templates` repo to `~/.pulumi/templates`. Having it go in this well-known directory allows us to maintain previous behavior around allowing offline use of templates. If we use `Depth: 1` for the initial clone, it will fail when attempting to pull when there are updates to the remote repository. Unfortunately, there's no built-in `--unshallow` support in `go-git` and setting a larger `Depth` doesn't appear to help. There may be a workaround, but for now, if we're cloning the pulumi templates directory to `~/.pulumi/templates`, we won't use `Depth: 1`. For template URLs, we will continue to use `Depth: 1` as we clone those to a temp directory (which gets deleted) that we'll never try to update. * List available templates in help text * Address PR Feedback * Don't show "Installing dependencies" message for `up` * Fix secrets handling When prompting for config, if the existing stack value is a secret, keep it a secret and mask the prompt. If the template says it should be secret, make it a secret. * Fix ${PROJECT} and ${DESCRIPTION} handling for `up` Templates used with `up` should already have a filled-in project name and description, but if it's a `new`-style template, that has `${PROJECT}` and/or `${DESCRIPTION}`, be helpful and just replace these with better values. * Fix stack handling Add a bool `setCurrent` param to `requireStack` to control whether the current stack should be saved in workspace settings. For the `up <url>` case, we don't want to save. Also, split the `up` code into two separate functions: one for the `up <url>` case and another for the normal `up` case where you have workspace in your current directory. While we may be able to combine them back into a single function, right now it's a bit cleaner being separate, even with some small amount of duplication. * Fix panic due to nil crypter Lazily get the crypter only if needed inside `promptForConfig`. * Embellish comment * Harden isPreconfiguredEmptyStack check Fix the code to check to make sure the URL specified on the command line matches the URL stored in the `pulumi:template` config value, and that the rest of the config from the stack satisfies the config requirements of the template.
2018-08-11 01:08:16 +00:00
// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gitutil
import (
"io/ioutil"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
ptesting "github.com/pulumi/pulumi/pkg/testing"
)
func TestParseGitRepoURL(t *testing.T) {
test := func(expectedURL string, expectedURLPath string, rawurl string) {
actualURL, actualURLPath, err := ParseGitRepoURL(rawurl)
assert.NoError(t, err)
assert.Equal(t, expectedURL, actualURL)
assert.Equal(t, expectedURLPath, actualURLPath)
}
// GitHub.
pre := "https://github.com/pulumi/templates"
exp := pre + ".git"
test(exp, "", pre+".git")
test(exp, "", pre+"")
test(exp, "", pre+"/")
test(exp, "templates", pre+"/templates")
test(exp, "templates", pre+"/templates/")
test(exp, "templates/javascript", pre+"/templates/javascript")
test(exp, "templates/javascript", pre+"/templates/javascript/")
test(exp, "tree/master/templates", pre+"/tree/master/templates")
test(exp, "tree/master/templates/python", pre+"/tree/master/templates/python")
test(exp, "tree/929b6e4c5c39196ae2482b318f145e0d765e9608/templates",
pre+"/tree/929b6e4c5c39196ae2482b318f145e0d765e9608/templates")
test(exp, "tree/929b6e4c5c39196ae2482b318f145e0d765e9608/templates/python",
pre+"/tree/929b6e4c5c39196ae2482b318f145e0d765e9608/templates/python")
// Gists.
pre = "https://gist.github.com/user/1c8c6e43daf20924287c0d476e17de9a"
exp = "https://gist.github.com/1c8c6e43daf20924287c0d476e17de9a.git"
test(exp, "", pre+"")
test(exp, "", pre+"/")
testError := func(rawurl string) {
_, _, err := ParseGitRepoURL(rawurl)
assert.Error(t, err)
}
// No owner.
testError("https://github.com")
testError("https://github.com/")
// No repo.
testError("https://github.com/pulumi")
testError("https://github.com/pulumi/")
// Not HTTPS.
testError("http://github.com/pulumi/templates.git")
testError("http://github.com/pulumi/templates")
}
func TestGetGitReferenceNameOrHashAndSubDirectory(t *testing.T) {
e := ptesting.NewEnvironment(t)
defer deleteIfNotFailed(e)
// Create local test repository.
repoPath := filepath.Join(e.RootPath, "repo")
err := os.MkdirAll(repoPath, os.ModePerm)
assert.NoError(e, err, "making repo dir %s", repoPath)
e.CWD = repoPath
createTestRepo(e)
// Create temp directory to clone to.
cloneDir := filepath.Join(e.RootPath, "temp")
err = os.MkdirAll(cloneDir, os.ModePerm)
assert.NoError(e, err, "making clone dir %s", cloneDir)
test := func(expectedHashOrBranch string, expectedSubDirectory string, urlPath string) {
ref, hash, subDirectory, err := GetGitReferenceNameOrHashAndSubDirectory(repoPath, urlPath)
assert.NoError(t, err)
if ref != "" {
assert.True(t, hash.IsZero())
shortNameWithoutOrigin := strings.TrimPrefix(ref.Short(), "origin/")
assert.Equal(t, expectedHashOrBranch, shortNameWithoutOrigin)
} else {
assert.False(t, hash.IsZero())
assert.Equal(t, expectedHashOrBranch, hash.String())
}
assert.Equal(t, expectedSubDirectory, subDirectory)
}
// No ref or path.
test("HEAD", "", "")
test("HEAD", "", "/")
// No "tree" path component.
test("HEAD", "foo", "foo")
test("HEAD", "foo", "foo/")
test("HEAD", "content/foo", "content/foo")
test("HEAD", "content/foo", "content/foo/")
// master.
test("master", "", "tree/master")
test("master", "", "tree/master/")
test("master", "foo", "tree/master/foo")
test("master", "foo", "tree/master/foo/")
test("master", "content/foo", "tree/master/content/foo")
test("master", "content/foo", "tree/master/content/foo/")
// HEAD.
test("HEAD", "", "tree/HEAD")
test("HEAD", "", "tree/HEAD/")
test("HEAD", "foo", "tree/HEAD/foo")
test("HEAD", "foo", "tree/HEAD/foo/")
test("HEAD", "content/foo", "tree/HEAD/content/foo")
test("HEAD", "content/foo", "tree/HEAD/content/foo/")
// Tag.
test("my", "", "tree/my")
test("my", "", "tree/my/")
test("my", "foo", "tree/my/foo")
test("my", "foo", "tree/my/foo/")
// Commit SHA.
test("2ba6921f3163493809bcbb0ec7283a0446048076", "",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076")
test("2ba6921f3163493809bcbb0ec7283a0446048076", "",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076/")
test("2ba6921f3163493809bcbb0ec7283a0446048076", "foo",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076/foo")
test("2ba6921f3163493809bcbb0ec7283a0446048076", "foo",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076/foo/")
test("2ba6921f3163493809bcbb0ec7283a0446048076", "content/foo",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076/content/foo")
test("2ba6921f3163493809bcbb0ec7283a0446048076", "content/foo",
"tree/2ba6921f3163493809bcbb0ec7283a0446048076/content/foo/")
// The longest ref is matched, so we should get "my/content" as the expected ref
// and "foo" as the path (instead of "my" and "content/foo").
test("my/content", "foo", "tree/my/content/foo")
test("my/content", "foo", "tree/my/content/foo/")
testError := func(urlPath string) {
_, _, _, err := GetGitReferenceNameOrHashAndSubDirectory(repoPath, urlPath)
assert.Error(t, err)
}
// No ref specified.
testError("tree")
testError("tree/")
// Invalid casing.
testError("tree/Master")
testError("tree/head")
testError("tree/My")
// Path components cannot contain "." or "..".
testError(".")
testError("./")
testError("..")
testError("../")
testError("foo/.")
testError("foo/./")
testError("foo/..")
testError("foo/../")
testError("content/./foo")
testError("content/./foo/")
testError("content/../foo")
testError("content/../foo/")
}
func createTestRepo(e *ptesting.Environment) {
e.RunCommand("git", "init")
writeTestFile(e, "README.md", "test repo")
e.RunCommand("git", "add", "*")
e.RunCommand("git", "commit", "-m", "'Initial commit'")
writeTestFile(e, "foo/bar.md", "foo-bar.md")
e.RunCommand("git", "add", "*")
e.RunCommand("git", "commit", "-m", "'foo dir'")
writeTestFile(e, "content/foo/bar.md", "content-foo-bar.md")
e.RunCommand("git", "add", "*")
e.RunCommand("git", "commit", "-m", "'content-foo dir'")
e.RunCommand("git", "branch", "my/content")
e.RunCommand("git", "tag", "my")
}
func writeTestFile(e *ptesting.Environment, filename string, contents string) {
filename = filepath.Join(e.CWD, filename)
dir := filepath.Dir(filename)
err := os.MkdirAll(dir, os.ModePerm)
assert.NoError(e, err, "making all directories %s", dir)
err = ioutil.WriteFile(filename, []byte(contents), os.ModePerm)
assert.NoError(e, err, "writing %s file", filename)
}
// deleteIfNotFailed deletes the files in the testing environment if the testcase has
// not failed. (Otherwise they are left to aid debugging.)
func deleteIfNotFailed(e *ptesting.Environment) {
if !e.T.Failed() {
e.DeleteEnvironment()
}
}