mirror of https://github.com/pulumi/pulumi.git
09accbf68a
The majority of our NodeJS Automation API tests are currently configured to use the (default) option of a Pulumi Cloud backend. This is undesirable for several reasons: * Using Pulumi Cloud introduces a network into the mix, meaning there is a greater chance of tests flaking. * Pulumi Cloud is essentially a massive shared global state, so if multiple tests run concurrently, we need to ensure that they don't interfere with each other. We thus have code to generate random stack names, projects, etc. to avoid these sorts of clashes. If we don't, we again run the risk of having flaky tests. * It's slower than not using Pulumi Cloud/the network and instead using e.g. a local file-backed state. This commit reworks as many tests as possible to use local file-backed state. In doing so, we optimistically re-enable some previously skipped tests that were marked as flaky. For hygiene, and in case there is a chance we'll run multiple instances of the suite on a single instance in CI, we use temporary directories for state where possible which are aggressively cleaned up on test suite exit (see the `tmp` NPM package which we already use elsewhere). For tests that use `Pulumi.yaml` files (and thus must have a fixed `backend.url`), we use `file://~`, which is already used in one test and seems to work just fine. As a result of these changes, the tests run significantly faster (from the order of tens of minutes to just a few, if that) and should in theory be less (never, 🙏) flaky. Moreover, it's likely that we could later remove some more of the code in our tests that e.g. generates random names, since there should be no need for this any more. Fixes #8061 Part of #15940 |
||
|---|---|---|
| .. | ||
| correct_project | ||
| json | ||
| nested_config | ||
| tcfg | ||
| testproj | ||
| testproj_dotnet | ||
| yaml | ||
| yml | ||