pulumi/cmd/pulumi-test-language
Fraser Waters c7ed4024ff Support for integers on the wire and state protocol
An exploration into how we could support int64 as well as float64 as a
first class type in Pulumi.

There's some inherit trickiness with this because we use JSON in a few
places, state files most obviously but `PropertyValue` on the wire is
using the protobuf `Struct` and `Value` types which are just mappings to
JSON as well.

JSON doesn't have an int64 type, and while the Go JSON marshaller can
read/write int64s it's not a great behaviour to rely on given mixed
support for that across languages. Further the protobuf `Value` type
only supports float64s.

So we support integers similar to our other special types (like assets)
and wrap them in an object with a special signature field. The value
itself is either a number (if it can safely roundtrip to float64 and
back again) or a string.

Providers, SDKs and the engine communicate support for integers via
`AcceptsIntegers` fields in their interfaces (or similar). When
transmitting to a peer that doesn't support integers (e.g. an old SDK,
or old provider) all integer values are mapped back to standard number
values.

As this changes the serialisation of property values in the state file
it will trigger errors if old engines try to load a state file
containing integers.
2024-08-23 13:36:45 +01:00
..
providers Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00
testdata Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00
README.md Add sequence diagram about pulumi-test-language (#15289) 2024-01-29 22:08:14 +00:00
go.mod Changelog and go.mod updates for v3.129.0 (#16947) 2024-08-12 19:28:33 +00:00
go.sum upgrade pulumi-yaml to 1.9.2 (#16925) 2024-08-09 15:09:23 +00:00
interface.go Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00
interface_test.go Add a test that all our conformance test programs are bindable (#16770) 2024-07-24 06:55:18 +00:00
internal_test.go Parse conformance programs to decide which packages are needed for SDKs (#16872) 2024-08-05 12:07:06 +00:00
l1empty_test.go Run pulumi-test-language tests in parallel (#16060) 2024-04-25 21:32:39 +00:00
l1main_test.go Add asset/archive to conformance tests and fix engine working dir issues (#16100) 2024-05-02 11:32:54 +00:00
l2continue_on_error_test.go Run pulumi-test-language tests in parallel (#16060) 2024-04-25 21:32:39 +00:00
l2destroy_test.go Run pulumi-test-language tests in parallel (#16060) 2024-04-25 21:32:39 +00:00
l2large_test.go Refactor: move plugin kind to apitype (#15946) 2024-04-25 17:30:30 +00:00
l2long_test.go Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00
l2resourceasset_test.go Add conformance tests for remote assets (#16467) 2024-07-03 09:03:24 +00:00
l2resourcesimple_test.go Replace `result.Result` with native errors (#17044) 2024-08-22 14:39:59 +00:00
main.go Add matrix testing (#13705) 2023-09-13 15:17:46 +00:00
runtime_options_test.go Run pulumi-test-language tests in parallel (#16060) 2024-04-25 21:32:39 +00:00
snapshot_test.go Add asset/archive to conformance tests and fix engine working dir issues (#16100) 2024-05-02 11:32:54 +00:00
snapshots.go Add asset/archive to conformance tests and fix engine working dir issues (#16100) 2024-05-02 11:32:54 +00:00
test_host.go Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00
testing.go Replace `result.Result` with native errors (#17044) 2024-08-22 14:39:59 +00:00
tests.go Support for integers on the wire and state protocol 2024-08-23 13:36:45 +01:00

README.md

pulumi-language-test runs a gRPC interface that language plugins can use to run a suite of standard tests.

Architecture

pulumi-language-test is used to run a standard suite of tests against any compliant language plugin.

The diagram below shows the main interactions and data flows for how this system works.

There are three main actors involved. Firstly test which is a test function coordinating the language plugin and pulumi-language-test. Secondly ptl which is the pulumi-language-test process. Finally uut which is the language plugin actually being tested. This will generally be a grpc server running in the same process as the test method.


sequenceDiagram
    test->>ptl: Start ptl process
    ptl-->>test: Read stdout for ptl address to connect to

    Note right of test: All future calls to ptl are via grpc

    test->>+ptl: GetLanguageTests()
    ptl-->>-test: Returns list of test names

    test->>+uut: Serve
    uut-->>-test: Returns uut server address

    test->>+ptl: PrepareLanguageTests(uut)
    ptl->>+uut: Pack(core)
    uut-->>-ptl: Returns name of core artifact
    ptl-->>-test: Returns `token`

    loop for each test
        test->>+ptl: RunLanguageTest(token, test)
        loop for each sdk used in test
            ptl->>+uut: GeneratePackage(sdk)
            uut-->>-ptl: Write package code to temporary directory
            ptl->>ptl: Verify sdk snapshot
            ptl->>+uut: Pack(sdk)
            uut-->>-ptl: Returns name of sdk artifact
        end

        ptl->>+uut: GenerateProject(test)
        uut-->>-ptl: Write project code to temporary directory
        ptl->>ptl: Verify project snapshot

        ptl->>+uut: InstallDependencies(project)
        uut-->>-ptl: 

        note right of ptl: Execute test with engine
        activate ptl
        ptl->>+uut: Run(project)
        uut-->>-ptl: Return run result
        ptl->>ptl: Run test asserts against run result and snapshot
        deactivate ptl

        ptl-->>-test: Returns test result from asserts
    end

    test->>ptl: sigkill

Meta tests

This module contains a number of _test.go files. These are tests of the conformance test system itself. The actual conformance tests are all defined in tests.go.