Commit Graph

13 Commits

Author SHA1 Message Date
Will Jones 0a6876c527
Support serialising reserved NodeJS identifiers ()
This commit addresses part of , in which we fail to serialise
closures whose code would use reserved identifiers like `exports`. This
is due to a change we made where module imports are hoisted to avoid
importing the same module multiple times. Previously, code adopted a
strategy of passing all dependencies using a `with` statement, viz.:

```typescript
function x() {
  return (function () {
    with({ fooBarBaz: require("foo/bar/baz"), ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}

function y() {
  return (function () {
    with({ fooBarBaz: require("foo/bar/baz"), ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}
```

This was changed to remove the duplicate imports, yielding code like:

```typescript
const fooBarBaz = require("foo/bar/baz")

function x() {
  return (function () {
    with({ ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}

function y() {
  return (function () {
    with({ ... }) {
      // Use of fooBarBaz
    }
  }).apply(...)
}
```

However, while the previous approach would work with reserved
identifiers such as `exports` (`with({ exports, ... }) { ... }` is
perfectly acceptable), the new one does not (`const exports = ...` is
not acceptable since NodeJS will not allow redeclaration of the
`exports` global).

This commit combines the two approaches. Modules are only imported once,
but if an import would use a reserved identifier, we generate a fresh
non-conflicting identifier and alias this using a `with` statement. For
example:

```typescript
const fooBarBaz = require("foo/bar/baz")

// __pulumi_closure_import_exports is generated to avoid shadowing the reserved "exports"
const __pulumi_closure_import_exports = require("some/other/module")

function x() {
  return (function () {
    with({ exports: __pulumi_closure_import_exports, ... }) {
      // Use of fooBarBaz and exports
    }
  }).apply(...)
}
```

Note that it is not expected that  will be solved in its entirety.
While this commit fixes code that introduces identifiers like `exports`,
the introduction in question in that issue is caused by the use of
`pulumi.output(...)` in the constructor of a dynamic resource provider.
Since dynamic resource providers are implemented under the hood by
serialising their code, we attempt to serialise `pulumi.output` and its
dependency chain. This commit allows us to make more progress in that
regard, but other things go wrong thereafter. Fixing the deeper issue
that underpins  (and likely other challenges with dynamic
providers) is probably a more involved piece of work.
2024-04-10 15:12:43 +00:00
Fraser Waters b5952e4bfd Pass PULUMI_CONFIG through to provider plugins 2023-04-05 10:17:18 +01:00
Fraser Waters d6fcd846bc Add test to show errors in dynamic NodeJS providers work 2023-03-29 11:21:51 +01:00
Alexander Samsig 8a15fc9a59 [sdk/nodejs] - Add support for custom naming of dynamic provider resource. 2023-01-17 18:57:58 +01:00
Anton Tayanovskyy 151fdff906
Lower the BrokenDynamicProvider regression test from integration to mock () 2021-09-10 17:25:48 -04:00
Luke Hoban f89e9a29f5
Revert "Allow Python dynamic provider resources to be constructed outside of `__main__` ()" ()
This reverts commit ebb0e6aaed.

The changes in  introduced a regression tracked in .  It is not yet clear how to retain the desired behaviour introduced in  while avoiding this regression, so for now we will revert those changes, and re-open  to track a deeper fix.  That fix may require making changes to upstream `dill` to properly support these serialization requirements.

Fixes .
2021-09-01 20:49:04 -07:00
Jan Češpivo e67334db1d
Added support for custom naming of dynamic provider resource ()
Now there is not possible to change a name of dynamic provider resource without copying a code of the `pulumi.sdk.python.lib.pulumi.dynamic.dynamic.Resource` and changing the hard-coded name `"pulumi-python:dynamic:Resource"`. I successfully uses this proposal to make it possible.

Usage:
```python
class CustomResource(
    Resource, name="my-custom-provider:CustomResource"
):
   ...
```

Co-authored-by: Pat Gavlin <pgavlin@gmail.com>
2021-08-17 14:15:53 -07:00
Luke Hoban ebb0e6aaed
Allow Python dynamic provider resources to be constructed outside of `__main__` ()
The underlying library `dill` that we use for serializing dynamic providers into Pulumi state for Python dynamic providers serializes classes differently depending on whether they are in `__main__` or in another module.  We need the by-value serialization to be applied in all cases.

https://github.com/uqfoundation/dill/issues/424 is tracking adding the ability into `dill` to specify this by-value serialization explicitly, but until then, we will temporarily re-write the `__module__` of a provder class prior to serialization, so that `dill` behaves as we need for the dynamic provider use case.

Fixes .
2021-08-12 20:02:17 -07:00
Anton Tayanovskyy b16d50085b
Fix hanging deployments and improve errmsgs for programs with incorrect typings for output values ()
* Do not hang but propagate exception when it happens in resolve_outputs

* Add an integration test for the issue

* Better error message

* Add CHANGELOG_PENDING entry

* Update sdk/python/lib/pulumi/runtime/rpc.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Address PR feedback and tighten path param typing

* Given Windows builder is failing, allow 2x time for the test

* Give some more time to the Windows runner

* Attempt to solve differently

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-06-23 11:27:17 -04:00
Justin Van Patten 0059cf205e
Remove dead Python integration tests ()
These test files are no longer used. All the Python integration tests now use venv.
2021-02-11 08:21:50 -08:00
Justin Van Patten b77ec919d4
Install and use dependencies automatically for new Python projects ()
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.

This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):

```yaml
runtime:
  name: python
  options:
    virtualenv: venv
```

`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.

Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
2020-06-09 16:42:53 -07:00
Matt Ellis 431413dcbb Add requirements.txt to all test projects
We don't actually depend on anything right now because all of these
tests just depend on `pulumi` which is installed as an editable
package.
2019-08-23 15:02:58 -07:00
Luke Hoban 3768e5c690
Python Dynamic Providers ()
Dynamic providers in Python.

This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results.

One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions.  So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python.  This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature.  It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished.  We can explore this separately.

```py
from pulumi import ComponentResource, export, Input, Output
from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult
from typing import Optional
from github import Github, GithubObject

auth = "<auth token>"
g = Github(auth)

class GithubLabelArgs(object):
    owner: Input[str]
    repo: Input[str]
    name: Input[str]
    color: Input[str]
    description: Optional[Input[str]]
    def __init__(self, owner, repo, name, color, description=None):
        self.owner = owner
        self.repo = repo
        self.name = name
        self.color = color
        self.description = description

class GithubLabelProvider(ResourceProvider):
    def create(self, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label(
            name=props["name"],
            color=props["color"],
            description=props.get("description", GithubObject.NotSet))
        return CreateResult(l.name, {**props, **l.raw_data}) 
    def update(self, id, _olds, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.edit(name=props["name"],
               color=props["color"],
               description=props.get("description", GithubObject.NotSet))
        return UpdateResult({**props, **l.raw_data})
    def delete(self, id, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.delete()

class GithubLabel(Resource):
    name: Output[str]
    color: Output[str]
    url: Output[str]
    description: Output[str]
    def __init__(self, name, args: GithubLabelArgs, opts = None):
        full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)}
        super().__init__(GithubLabelProvider(), name, full_args, opts)

label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b"))

export("label_color", label.color)
export("label_url", label.url)
```


Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 10:18:25 -07:00