History

Fraser Waters 7bb2a3c2ac Auto-fix encrypted keys in the wrong format due to gocloud.dev upgrade regression (#15334 ) When attempting another gocloud.dev upgrade, some users ran into a regression related to the format change of encrypted keys between gocloud versions, which part of the system was not accounting for. This PR addresses the issue and includes a fix that automatically fixes forward state that has an encrypted key in the wrong format, and includes a regression test and test for the auto-fix behavior. Fixes #15329 Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>		2024-02-01 09:39:41 +00:00
..
testdata	Test runtime options in conformance tests (#15288 )	2024-01-29 07:32:11 +00:00
README.md	Add sequence diagram about pulumi-test-language (#15289 )	2024-01-29 22:08:14 +00:00
bad_provider.go	Send old inputs to Delete (#14051 )	2023-10-13 14:12:26 +00:00
go.mod	Auto-fix encrypted keys in the wrong format due to gocloud.dev upgrade regression (#15334 )	2024-02-01 09:39:41 +00:00
go.sum	Auto-fix encrypted keys in the wrong format due to gocloud.dev upgrade regression (#15334 )	2024-02-01 09:39:41 +00:00
interface.go	Rename filestate to DIY (#15314 )	2024-01-30 15:53:10 +00:00
interface_test.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
internal_test.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
l1empty_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
l1main_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
l2resourcesimple_test.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00
main.go	Add matrix testing (#13705 )	2023-09-13 15:17:46 +00:00
runtime_options_test.go	Test runtime options in conformance tests (#15288 )	2024-01-29 07:32:11 +00:00
simple_provider.go	Send old inputs to Delete (#14051 )	2023-10-13 14:12:26 +00:00
testing.go	Use mapset in pulumi-test-language (#14738 )	2023-12-04 20:49:34 +00:00
tests.go	Test main in conformance tests (#15287 )	2024-01-28 20:11:52 +00:00

README.md

pulumi-language-test runs a gRPC interface that language plugins can use to run a suite of standard tests.

Architecture

pulumi-language-test is used to run a standard suite of tests against any compliant language plugin.

The diagram below shows the main interactions and data flows for how this system works.

There are three main actors involved. Firstly test which is a test function coordinating the language plugin and pulumi-language-test. Secondly ptl which is the pulumi-language-test process. Finally uut which is the language plugin actually being tested. This will generally be a grpc server running in the same process as the test method.


sequenceDiagram
    test->>ptl: Start ptl process
    ptl-->>test: Read stdout for ptl address to connect to

    Note right of test: All future calls to ptl are via grpc

    test->>+ptl: GetLanguageTests()
    ptl-->>-test: Returns list of test names

    test->>+uut: Serve
    uut-->>-test: Returns uut server address

    test->>+ptl: PrepareLanguageTests(uut)
    ptl->>+uut: Pack(core)
    uut-->>-ptl: Returns name of core artifact
    ptl-->>-test: Returns `token`

    loop for each test
        test->>+ptl: RunLanguageTest(token, test)
        loop for each sdk used in test
            ptl->>+uut: GeneratePackage(sdk)
            uut-->>-ptl: Write package code to temporary directory
            ptl->>ptl: Verify sdk snapshot
            ptl->>+uut: Pack(sdk)
            uut-->>-ptl: Returns name of sdk artifact
        end

        ptl->>+uut: GenerateProject(test)
        uut-->>-ptl: Write project code to temporary directory
        ptl->>ptl: Verify project snapshot

        ptl->>+uut: InstallDependencies(project)
        uut-->>-ptl: 

        note right of ptl: Execute test with engine
        activate ptl
        ptl->>+uut: Run(project)
        uut-->>-ptl: Return run result
        ptl->>ptl: Run test asserts against run result and snapshot
        deactivate ptl

        ptl-->>-test: Returns test result from asserts
    end

    test->>ptl: sigkill

Meta tests

This module contains a number of _test.go files. These are tests of the conformance test system itself. The actual conformance tests are all defined in tests.go.