I used the node SDK as inspiration for this change - there were a few things
we were doing differently in python (specifically handling all cases in
the outer layer of `massage` rather than allowing for more fine-grained
control in a `massage_complex` function like in node. We also take the stack
concept from the node SDK to resolve the immediate issue of duplicate outputs.
This commit modifies `setup.py` to use a Python variable as the source of the package version instead of a placeholder string and adds a fallback for the README definition. This makes the package installable via the `-e` flag directly from its source directory (`sdk/python/lib`), rather than having a build step and having to `-e` install the built directory (`sdk/python/env/src`).
We've had a few issues in the recent past related to pipenv oddities in
CI which lead us to temporarily globally install the Python SDK in CI.
This change removes the use of pipenv in favor of Python's built-in
venv and avoids globally installing the Python SDK.
These changes add support for gathering code coverage data during tests.
For tests that do not involve the Pulumi CLI, this is straightforward: all of
the ecosystems we target already support gathering coverage data, and we follow
the rules accordingly. Support for each language is broken out into its own
commit.
For tests that do involve the Pulumi CLI, the picture is a bit more complicated.
Go does not make it trivial to perform a coverage-instrumented build (go build
does not have a -cover flag, for example). In lieu of official support, we abuse
go test -c and TestMain to produce a build of the CLI that supports collecting
and reporting coverage data.
This commit introduces a 'next' package which we can use as a staging
ground for incrementally adopting new Python 3 code. The next package is
initially populated with the non-runtime portions of the Python SDK,
which is enough to pass all tests when running on Python 3. Future
commits will reach further into the runtime.
This change uses virtualenv to insulate us from platform differences
in our building of the Python SDK, and to create an isolated Python 2
environment. This includes meaning we don't need to worry about the
specific location and behavior of Pylint. I *think* this will work
no matter whether it's Mac, Ubuntu, ArchLinux, Windows, and so on.
We do install to the --user directory in the install target using
`pip install -e`, however, which enables the machine-wide symlinking
that we need to support various workflows.
This fixespulumi/pulumi#1007.
This change gets enough of the Python SDK up and running that the
empty Python program will work. Mostly just scaffolding, but the
basic structure is now in place. The primary remaining work is to
wire up resource creation to the gRPC interfaces.
In summary:
* The basic structure is as follows:
- Everything goes into sdk/python/.
- sdk/python/cmd/pulumi-langhost-python is a Go language host
that simply knows how to spawn Python processes to run out
entrypoint in response to requests by the engine.
- sdk/python/cmd/pulumi-langhost-python-exec is a little Python
shim that is invoked by the language host to run Python programs,
and is responsible for setting up the minimal goo before we can
do so (RPC connections and the like).
- sdk/python/lib/ contains a Python Pip package suitable for PyPi.
- In there, we have two packages: the root pulumi package that
contains all of the basic Pulumi programming model abstractions,
and pulumi.runtime, which contains the implementation of
resource registration, RPC interfacing with the engine, and so on.
* Add logic in our test framework to conditionalize on the language
type and react accordingly. This will allow us to skip Yarn for
Python projects and eventually run Pip if there's a requirements.txt.
* Created the basic project structure, including all of the usual
Make targets for installing into the proper places.
* Building also runs Pylint and we are clean.
There are a few other minor things in here:
* Add an "empty" test for both Node.js and Python. These pass.
* Fix an existing bug in plugin shutdown logic. At some point, we
started waiting for stderr/stdout to flush before shutting down
the plugin; but if certain failures happen "early" during the
plugin launch process, these channels will never get initialized
and so waiting for them deadlocks.
* Recently we seem to have added logic to delete test temp
directories if a failure happened during initialization of said
temp directories. This is unfortunate, because you often need to
look at the temp directory to see what failed. We already clean
them up elsewhere after the full test completes successfully, so
I don't think we need to be doing this, and I've removed it.
Still many loose ends (config, resources, etc), but it's a start!