History

Fraser Waters afb287d2fb Rename filestate to DIY (#15314 ) This goes through the codebase to try and be consistent about names for the diy/filestate/local/selfmanaged backend. Every reference to this backend should now use the terms "DIY". There are a couple of places that still say "local DIY backend" this is referring to a DIY backend using the local filesystem (i.e. `pulumi login --local`).		2024-01-30 15:53:10 +00:00
..
testdata	Test runtime options in conformance tests (#15288 )	2024-01-29 07:32:11 +00:00
README.md	Add sequence diagram about pulumi-test-language (#15289 )	2024-01-29 22:08:14 +00:00
bad_provider.go	Send old inputs to Delete (#14051 )	2023-10-13 14:12:26 +00:00
go.mod	upgrade gocloud.dev take 2 (#15202 )	2024-01-26 13:14:17 +00:00
go.sum	upgrade gocloud.dev take 2 (#15202 )	2024-01-26 13:14:17 +00:00
interface.go	Rename filestate to DIY (#15314 )	2024-01-30 15:53:10 +00:00
interface_test.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
internal_test.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
l1empty_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
l1main_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
l2resourcesimple_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
main.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
runtime_options_test.go	Test runtime options in conformance tests (#15288 )	2024-01-29 07:32:11 +00:00
simple_provider.go	Send old inputs to Delete (#14051 )	2023-10-13 14:12:26 +00:00
testing.go	Use mapset in pulumi-test-language (#14738 )	2023-12-04 20:49:34 +00:00
tests.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00

README.md

pulumi-language-test runs a gRPC interface that language plugins can use to run a suite of standard tests.

Architecture

pulumi-language-test is used to run a standard suite of tests against any compliant language plugin.

The diagram below shows the main interactions and data flows for how this system works.

There are three main actors involved. Firstly test which is a test function coordinating the language plugin and pulumi-language-test. Secondly ptl which is the pulumi-language-test process. Finally uut which is the language plugin actually being tested. This will generally be a grpc server running in the same process as the test method.


sequenceDiagram
    test->>ptl: Start ptl process
    ptl-->>test: Read stdout for ptl address to connect to

    Note right of test: All future calls to ptl are via grpc

    test->>+ptl: GetLanguageTests()
    ptl-->>-test: Returns list of test names

    test->>+uut: Serve
    uut-->>-test: Returns uut server address

    test->>+ptl: PrepareLanguageTests(uut)
    ptl->>+uut: Pack(core)
    uut-->>-ptl: Returns name of core artifact
    ptl-->>-test: Returns `token`

    loop for each test
        test->>+ptl: RunLanguageTest(token, test)
        loop for each sdk used in test
            ptl->>+uut: GeneratePackage(sdk)
            uut-->>-ptl: Write package code to temporary directory
            ptl->>ptl: Verify sdk snapshot
            ptl->>+uut: Pack(sdk)
            uut-->>-ptl: Returns name of sdk artifact
        end

        ptl->>+uut: GenerateProject(test)
        uut-->>-ptl: Write project code to temporary directory
        ptl->>ptl: Verify project snapshot

        ptl->>+uut: InstallDependencies(project)
        uut-->>-ptl: 

        note right of ptl: Execute test with engine
        activate ptl
        ptl->>+uut: Run(project)
        uut-->>-ptl: Return run result
        ptl->>ptl: Run test asserts against run result and snapshot
        deactivate ptl

        ptl-->>-test: Returns test result from asserts
    end

    test->>ptl: sigkill

Meta tests

This module contains a number of _test.go files. These are tests of the conformance test system itself. The actual conformance tests are all defined in tests.go.