Commit Graph

37 Commits

Author SHA1 Message Date
Luke Hoban 96e4b74b15
Support for stack outputs ()
Adds support for top-level exports in the main script of a Pulumi Program to be captured as stack-level output properties.

This create a new `pulumi:pulumi:Stack` component as the root of the resource tree in all Pulumi programs.  That resources has properties for each top-level export in the Node.js script.

Running `pulumi stack` will display the current value of these outputs.
2017-11-17 15:22:41 -08:00
CyrusNajmabadi 36a692390d
Properly capture 'arguments' when creating our serialization closure. ()
* Simplify how we capture 'this' in our serialization logic.
* Properly capture 'arguments'

* add tests for 'arguments' capture.

* Properly serialize out 'arguments'
* Invert 'with' and function closure.
2017-11-15 11:31:17 -08:00
CyrusNajmabadi 89b5a4be71
remove use of 'eval' in tests. ()
* remove use of 'eval' in tests.

* Remove another eval.

* Remove usage of eval.
2017-10-31 14:41:58 -07:00
Joe Duffy cdb2c79e8e
Exit with an error code in the face of unhandled errors ()
As part of fixing the exit bug recently, we accidentally made errors
lead to zero exit codes.  As a result, the Pulumi CLI thought the
prgoram exited ordinarily, and proceeded to do its usual planning and
deployment, rather than terminating abruptly.

This is a byproduct of how Node's process.uncaughtException handler
works.  It hijacks and replaces all usual error logic, including the
process.exit part.  This change simply adds back the non-zero exit.

I also added a test (and fixed one other that began failing
afterwards), so that we can prevent regressions down the road.
2017-10-28 17:05:05 -07:00
joeduffy c61bce3e41 Permit undefined in more places
The prior code was a little too aggressive in rejected undefined
properties, because it assumed any occurrence indicated a resource
that was unavailable due to planning.  This is a by-produt of our
relatively recent decision to flow undefineds freely during planning.

The problem is, it's entirely legitimate to have undefined values
deep down in JavaScript structures, entirely unrelated to resources
whose property values are unknown due to planning.

This change flows undefined more freely.  There really are no
negative consequences of doing so, and avoids hitting some overly
aggressive assertion failures in some important scenarios.  Ideally
we would have a way to know statically whether something is a resource
property, and tighten up the assertions just to catch possible bugs
in the system, but because this is JavaScript, and all the assertions
are happening at runtime, we simply lack the necessary metadata to do so.
2017-10-23 16:02:28 -07:00
Luke Hoban ba98f5e837 Fix bugs in free variable analysis ()
Properties and methods were not being traversed correctly.

Fixes .
2017-10-19 23:20:57 -07:00
joeduffy 599ca8ea43 Add accessors to fetch the Pulumi project and stack names
This change adds functions, `pulumi.getProject()` and `pulumi.getStack()`,
to fetch the names of the project and stack, respectively.  These can be
handy in generating names, specializing areas of the code, etc.

This fixes .
2017-10-19 08:26:57 -07:00
CyrusNajmabadi d929c169de Enable tslinting of the nodejs sdk. () 2017-10-18 15:03:56 -07:00
CyrusNajmabadi d007f040b9 Move from acorn to TypeScript as the parser we use when computing free variables. () 2017-10-18 13:29:53 -07:00
joeduffy 301739c6b5 Add auto-parenting
This changes a few things about "components":

* Rename what was previously ExternalResource to CustomResource,
  and all of the related fields and parameters that this implies.
  This just seems like a much nicer and expected name for what
  these represent.  I realize I am stealing a name we had thought
  about using elsewhere, but this seems like an appropriate use.

* Introduce ComponentResource, to make initializing resources
  that merely aggregate other resources easier to do correctly.

* Add a withParent and parentScope concept to Resource, to make
  allocating children less error-prone.  Now there's no need to
  explicitly adopt children as they are allocated; instead, any
  children allocated as part of the withParent callback will
  auto-parent to the resource provided.  This is used by
  ComponentResource's initialization function to make initialization
  easier, including the distinction between inputs and outputs.
2017-10-15 04:38:26 -07:00
joeduffy fbfca58a3f Implement components
This change implements core support for "components" in the Pulumi
Fabric.  This work is described further in , where
we are still discussing some of the finer points.

In a nutshell, resources no longer imply external providers.  It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.

For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.

To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.

All resources now have the ability to adopt children.  This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).

Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output.  For instance:

    + aws:serverless:Function: (create)
      [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
      => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
      => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
      => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda

The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 18:30:59 -07:00
CyrusNajmabadi b713990b5e Enable 'use const' linter rule. ()
* Enable 'use const' linter rule.
2017-10-10 14:50:55 -07:00
CyrusNajmabadi 556345c68e Include environment when computing the hash of closure. ()
* Include environment when computing the hash of closure.
2017-10-09 21:46:24 -07:00
Luke Hoban 4af1345d4c Move serialization of Closures to JavaScript text to pulumi SDK ()
This logic was previously in the `@pulumi/aws` pacakge.  Moving it into the `pulumi` SDK as part of the overall closure serialization logic to make it more broadly accessible, and to centralize this functionality.

Now that it's all in one place, we may decide to remove the publically exposed `Closure` abstraction completely, which may also enable significant simplicifcation to the logic in closure serialization.

Also add one initial test case for this code.

Fixes .
2017-10-07 14:29:56 -07:00
joeduffy 2e9e0d2a98 Add a simple invoke test case 2017-09-30 14:53:27 -04:00
Luke Hoban ad5ee5bc04 Support module capture without serialization ()
This change adds first class support for capturing objects which are references to loaded Node modules.

If an object to be serialized is found as a loaded module which can be referenced as `require(<name>)`, then is is not serialized and is passed as a new kind of environment entry - `module` which will be de-serialized as a `require` statement.

Supports three cases:
1. built-in modules such as `http` and `path`
2. dependencies in the `node_modules` folder
3. other user-defined modules in the source folder

This allows natural use of `import`s with "inside" code.  For example - note the use of `$` in the outside scope only on the "inside".

```typescript
import * as cloud from "@pulumi/cloud";
import * as $ from "cheerio";
let queue = new pulumi.Topic<string>("sites_to_process");
queue.subscribe("foreachurl", async (url) => {
    let x = $("a", "<a href='foo'>hello</a>");
});
```

Also fixes free variable capture of `this` in arrow functions.

Fixes .
2017-09-28 16:44:00 -07:00
Luke Hoban e65348e246 Support for destructing patterns in free variable computation ()
Also runs `sdk/nodejs` tests by default during build.
2017-09-25 15:01:31 -07:00
pat@pulumi.com 69341fa7c8 `push` is dead; long live `update`.
After discussion with Joe and Luke, we've decided to use `update` instead
of `push` as it more intuitively fits the operation being performed.
2017-09-22 17:23:40 -07:00
pat@pulumi.com 597db186ec Renames: `plan` -> `preview`, `deploy` -> `push`.
Part of .

These changes also remove all command aliases from the `pulumi` command.
2017-09-22 15:28:03 -07:00
Joe Duffy f6e694c72b Rename pulumi-fabric to pulumi
This includes a few changes:

* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.

* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.

* The CLI is renamed from lumi to pulumi.
2017-09-21 19:18:21 -07:00
joeduffy 1c2c972d37 Add back Computed<T> as a short-hand
This adds back Computed<T> as a short-hand for Promise<T | undefined>.
Subtly, all resource properties need to permit undefined flowing through
during planning  Rather than forcing the long-hand version, which is easy
to forget, we'll keep the convention of preferring Computed<T>.  It's
just a typedef and the runtime type is just a Promise.
2017-09-20 09:59:32 -07:00
joeduffy f8ee6c570e Eliminate Computed/Property in favor of Promises
As part of , we've been exploring just using
undefined to indicate that a property value is absent during planning.
We also considered blocking the message loop to simplify the overall
programming model, so that all asynchrony is hidden.

It turns out ThereBeDragons 🐲 anytime you try to block the
message loop.  So, we aren't quite sure about that bit.

But the part we are convicted about is that this Computed/Property
model is far too complex.  Furthermore, it's very close to promises, and
yet frustratingly so far away.  Indeed, the original thinking in
 was simply to use promises, but we wanted to
encourage dataflow styles, rather than control flow.  But we muddied up
our thinking by worrying about awaiting a promise that would never resolve.

It turns out we can achieve a middle ground: resolve planning promises to
undefined, so that they don't lead to hangs, but still use promises so
that asynchrony is explicit in the system.  This also avoids blocking the
message loop.  Who knows, this may actually be a fine final destination.
2017-09-20 09:59:32 -07:00
joeduffy 087deb7643 Add optional dependsOn to Resource constructors
This change adds an optiona dependsOn parameter to Resource constructors,
to "force" a fake dependency between resources.  We have an extremely strong
desire to resort to using this only in unusual cases -- and instead rely
on the natural dependency DAG based on properties -- but experience in other
resource provisioning frameworks tells us that we're likely to need this in
the general case.  Indeed, we've already encountered the need in AWS's
API Gateway resources... and I suspect we'll run into more especially as we
tackle non-serverless resources like EC2 Instances, where "ambient"
dependencies are far more commonplace.

This also makes parallelism the default mode of operation, and we have a
new --serialize flag that can be used to suppress this default behavior.
Full disclosure: I expect this to become more Make-like, i.e. -j 8, where
you can specify the precise width of parallelism, when we tackle
.  I also think there's a good chance we will flip
the default, so that serial execution is the default, so that developers
who don't benefit from the parallelism don't need to worry about dependsOn
in awkward ways.  This tends to be the way most tools (like Make) operate.

This fixes .
2017-09-15 16:38:52 -07:00
joeduffy 8ce07617c9 Implement recursive closure captures
This change implements recursive closure captures.  This permits
cases like the following

    {
        function f() { g(); }
        function g() { f(); }
    }

and the slightly more useful

    class C {
        this.x = 42;
        this.f = () => x;
    }

To do this requires caching the environment objects and permitting
cycles in the resulting environment graph.  The closure emitter code
already knows how to handle this.

In addition, we must mark captures of `this` as free variables.

This resolves .
2017-09-10 07:40:53 -07:00
joeduffy f9995159c6 Fix a handful of things, mostly logging
* Initialize the diganostics logger with opts.Debug when doing
  a Deploy, like we do Plan.

* Don't spew leaked promises if there were Log.errors.

* Serialize logging RPC calls so that they can't appear out of order.

* Print stack traces in more places and, in particular, remember
  the original context for any errors that may occur asynchronously,
  like resource registration and calls to mapValue.

* Include origin stack traces generally in more error messages.

* Add some more mapValue test cases.

* Only undefined-propagate mapValue values during dry-runs.
2017-09-09 13:43:51 -07:00
joeduffy aefe297aa1 Harden dependent resolutions
This fixes a few problems with dependent resolutions and hardens
even more promises-related error paths, so we swallow precisely zero
errors (or at least we hope so).  This also digs through multi-level
chains of promises and computed properties as needed for nested mapValues.
2017-09-06 14:29:17 -07:00
joeduffy 240b54b5be Add typings and tests for mapValues that return computeds 2017-09-06 08:28:11 -07:00
joeduffy 6630de503c Support capturing Computed<T>s and Promise<T>s
This change adds support for awaiting any Computed<T> and Promise<T>s
that were captured inside of a function's closure.  This preserves our
ability to capture, for example, resource state that ends up getting
serialized as the final resource state, rather than a snapshot of the
(mostly unresolved) resource state at the time of serialization.
2017-09-06 07:36:19 -07:00
joeduffy 726e48e094 Add an extra test for nested functions 2017-09-05 15:50:47 -07:00
joeduffy 3164572b6e Fix some free variable capture logic
* Use `global.hasOwnProperty(ident)`, rather than `global[ident] !== undefined`,
  to avoid classifying references to globals as free variables.  Surprise(!!),
  the prior logic wouldn't work for `undefined` itself... 😒

* Expand this check to include the built-in Node.js module variables, namely
  `__dirname`, `__filename`, `exports`, `module`, and `require`, so that
  references to them don't get classified as serializable free variables either.

* Place catch variables in scope, so that `catch (err) { ... }` won't yield
  free variables for references to `err` within `...`.

* Place recursive function definitions into the top-level `var`-like scope of
  variables so that we don't consider references to them free.

* Harden all error pathways in the native C++ add-on so that we terminate
  anytime an exception is in-flight, rather than limping along and making
  things worse...
2017-09-05 15:21:14 -07:00
joeduffy e3a6695399 Depend only on vendored protos 2017-09-05 11:52:33 -07:00
joeduffy a1ab56fc28 Prettify properties
This change makes a few simplifications to how properties are exposed in
the system, mostly in the name of usability, but also to feel a bit more
like "idiomatic JavaScript".  Namely:

* Rename `then` to `mapValue`.  This hopefully helps to suggest that this
  is meant for a dataflow style of programming.

* Move Property<T> into the runtime module, and remove PropertyState<T>,
  collapsing back down to a single type.  This also eliminates some of the
  messy internal runtime casting, accessing of internal members, etc.

* Export a Computed<T> interface from the root of the module.  This is
  the entirety of the public-facing surface area for properties, and
  exposes that single `mapValue` member function.  The internal runtime
  logic understands how to handle Property<T>s specifically in addition
  to Computed<T>s more generally (in case someone writes their own).
2017-09-05 10:55:09 -07:00
joeduffy 2a22a71116 Tidy up resource properties
This changes a few aspects of resource properties:

* Move all runtime-related goo into the runtime module, in an
  internal PromiseState class.  This encapsulates the internal
  state transitions and protects against misuse.  It also allows
  us to clean up the public API for the Property<T> type so that
  it's entirely suitable for external usage.

* Track input and output property values distinctly.  It turns
  out we want to key off events differently.  For example, to marshal
  property values to a resource provider, we only care about the
  inputs.  For final property values that are used in, say, thens
  or as inputs to other properties, we want the output property value.

* Be more precise about when an output is truly final, and known, or
  unknown due to planning/dry-runs.  Note that this does mean that
  we'll encounter unknown values more frequently because, aside from
  IDs and URNs, we can't say for sure that arbitrary properties will never
  change post-creation.  We have ideas on how to denote this; see
   for more details.
2017-09-05 09:31:03 -07:00
joeduffy b80b6afcf1 Lint the test files 2017-09-04 11:35:21 -07:00
joeduffy 7c7610848f Rename asset classes
This change renames String, File, and Remote to StringAsset, FileAsset,
and RemoteAsset, largely to avoid conflicting with the built-in JavaScript
String type, but also because it mirrors our Archive naming strategy.
2017-09-04 11:35:21 -07:00
joeduffy d7688da5e3 Fix a few minor pathing things 2017-09-04 11:35:21 -07:00
joeduffy 3427647f93 Implement free variable calculations
This change implements free variable calculations and wires it up
to closure serialization.  This is recursive, in the sense that
the serializer may need to call back to fetch free variables for
nested functions encountered during serialization.

The free variable calculation works by parsing the serialized
function text and walking the AST, applying the usual scoping rules
to determine what is free.  In particular, it respects nested
function boundaries, and rules around var, let, and const scoping.

We are using Acorn to perform the parsing.  I'd originally gone
down the path of using V8, so that we have one consistent parser
in the game, however unfortunately neither V8's parser nor its AST
is a stable API meant for 3rd parties.  Unlike the exising internal
V8 dependencies, this one got very deep very quickly, and I became
nervous about maintaining all those dependencies.  Furthermore,
by doing it this way, we can write the free variable logic in
JavaScript, which means one fewer C++ component to maintain.

This also includes a fairly significant amount of testing, all
of which passes! 🎉
2017-09-04 11:35:21 -07:00