Stop cloning pulumi/home. This doesn't work in Travis because public
repositories can not have private SSH keys, which we'd need to clone
this repository. All the scripts we consume from there are now in
pulumi/scripts and so we'll just consume them from there.
All scripts that are generally useful across all builds have been moved
into `pulumi/scripts`. These changes clone that repository and retarget
the various scripts to their new location.
* Introduce a simple repetition operator to match expected error messages against actual ones
* Convert required and optional objects to use a Map (node v9 compat), improve the error formatting for failed tests
* Test node v6, v8, and v9 in CI
* Get rid of PULUMI_API env in .travis.yml, it's set from the Travis console now
Tests now target managed stacks instead of local stacks.
The existing logged in user and target backend API are used unless PULUMI_ACCES_TOKEN is defined, in which case tests are run under that access token and against the PULUMI_API backend.
For developer machines, we will now need to be logged in to Pulumi to run tests, and whichever default API backend is logged in (the one listed as current in ~/.pulumi/credentials.json) will be used. If you need to override these, provide PULUMI_ACCESS_TOKEN and possibly PULUMI_API.
For Travis, we currently target the staging service using the Pulumi Bot user.
We have decided to run tests in the pulumi organization. This can be overridden for local testing (or in Travis in the future) by defining PULUMI_API_OWNER_ORGANIZATION and using an access token with access to that organization.
Part of pulumi/home#195.
Also, rename/cleanup a bunch of serialization code.
Also, generate better environment names in the serialized closure code. Thsi code should be much easier to make sense of as hte names will better track to the original names in the user code.
Also, dedupe simple non-capturing functions. This helps ensure we don't spit out N copies of __awaiter (one per file it is declared in).
Make many fixes to closure serialization
Primary things that i've done as part of this change:
Added support for cyclic objects.
Properly serialize objects that are shared across different function. previously you would get multiple copies, now you properly reference the same copy.
Remove the usages of 'hashes' for functions. Because we track identity of objects, we no longer need them.
Serialize properties of functions (if they have any).
Handle Objects/Functions with different __proto__s than normal. i.e. classes/constructors. but also anything the user may have done themselves to the object.
Handle generator functions.
Handle functions with 'computed' names.
Handle functions with 'symbol' names.
Handle serializing Promises as Promises.
Removed the dual Closure/AsyncClosure tree. One existed solely so we could have a tree without promises (for use in testing maybe?). Because this all exists in a part of our codebase that is entirely async, it's fine to have promises in the tree, and to await them when serializing the Closure to a string.
Handle serializing class-constructors and methods. Including properly handling 'super' calls.
For some tools we used, like `dep` and `gometalinter` we were just
calling `go install` which caused us to pick up whatever was in `HEAD`
at that time. Now, we move to a model where we install fixed versions,
which will change per milestone.
While doing this, I changed the way our .travis.yml file runs
everything to move as much as possible out into scripts and do so in a
way that allows us to share as much common logic across our
repositories.
Uploading a bunch of tiny files is inefficient. Worse, it results in a ton of
paths printed to `stdout` -- enough that we could hit Travis' 4MB limit and
kill the job before we'd finished uploading.
Now we create a `.tar.gz` and upload that one, compressed file.
We also noticed that copied files might not be accessible from the `dev`
account, even though that account owns the `eng.pulumi.com` bucket. When
files are uploaded by one AWS account to a bucket owned by another, the
objects are, by default, only readable by the first account (the writer).
Change the ACL so the account that owns the bucket also has full access.
Our use of parens around the individual clauses of our if condition
seemed to cause Travis to ignore the condition and build
regardless. I've asked Travis about this, but for now, drop them (with
a comment) so we stop building pushes for topic branchs.
In order to begin publishing our core SDK package to NPM, we will
need it to be underneath the @pulumi scope so that it may remain
private. Eventually, we can alias pulumi back to it.
This is part of pulumi/pulumi#915.
Previously, we used safelisting to ensure we only build the master
branch on push jobs. Because Travis treated branches and tags as the
same thing when it came to safelisting, it meant that pushes of tags
did not cause builds.
Our versioning workflow requires that we rebuild on tag pushes,
because the tag plays into the version number we embed in our programs
and packages.
Instead, we move to the new Conditional Builds feature of Travis to
also allow us to build tags.
Part of pulumi/sdk#11
In travis, we've seen cases where writes to our standard streams
results in an error like: `/dev/stderr: resource temporarily
unavailable` which causes the tests to panic.
Now, in a perfect world, writes to /dev/stderr would not fail in this
way, but we do not live in a perfect world. Other processes on the
machine may make stderr/stdout non-blocking. We've are now seeing this
failure in Travis more often and it is masking real Pulumi failures
we want to debug.
In Travis, Node sees that `stdout` is a pipe and sets it to nonblocking.
Go's standard library isn't prepared for `stdout` to be nonblocking, so
sometimes long print statements can fail.
Unlike go binaries (where we can cross compile) the node module that
we publish needs to be built on the platform we publish for. Update
our `.travis.yml` file to also build on macOS and fix the publishing
script so we don't don't cross publish Darwin from Linux. Once we have
CI working for Windows, we'll remove the loop over PUBLISH_GOOS and
each build will publish just the artifacts for the host OS.
Defer build and publish logic into the makefiles instead of trying to
use Travis to do so. This let's us sidestep Travis's not great error
handling (see pulumi/home#21 for more details)