mirror of https://github.com/pulumi/pulumi.git
767d86e999
Currently we run all tests twice when running the tests in the merge queue. Once with the oldest version of all languages we support, and once with the most recent one. This leaves out all of the ones in the middle. Ideally we would like to test all of those as well, but that's way too expensive to do on every merge. Instead add a cron job that does exactly this test once a day. If it fails this will automatically open an issue for us to take a look. This will end up being quite a lot of jobs, so we're trying to run after the workday finished in the Americas, and before it starts in Europe, as to not overwhelm CI too much, as most employees will be asleep (apart from @brandonpollack23, but hopefully there's still enough CI runners left to pick up those jobs). One potential downside is that we might get more issues opened because of CI flakyness (e.g. especially downloading dependencies etc.), but it's probably worth a try and see if that's going to be too annoying. For the most part these tests should always be green, unless something is really screwy. Hopefully this will also allow us to only run one version set (probably minimum to make sure we're backwards compatible) during merge, so overall it might even reduce the load on the CI system. We should however only do that once this has been proven out, so that will be a separate PR. |
||
|---|---|---|
| .. | ||
| ISSUE_TEMPLATE | ||
| scripts | ||
| workflows | ||
| CODEOWNERS | ||