Commit Graph

73 Commits

Author SHA1 Message Date
Julien 2d2df02b56
Exclude .git and sdk/proto from gofumpt ()
Don’t format files in .git and sdk/proto using gofumpt. The go files in
sdk/proto are generated. If you have a branch name that ends with `.go`,
gofumpt attempts to format this, which will fail.
2024-07-21 10:11:41 +00:00
Fraser Waters dcf0a1024b
Revert "Revert "Run integration tests and dev builds with race detection" ()" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This reverts commit 75340dd942.

Fixes https://github.com/pulumi/pulumi/issues/16018.

This re-enables the locking and race detection. The locking is more
finely scoped to not be held over provider methods like Read/Update.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-09 16:15:41 +00:00
Fraser Waters 75340dd942
Revert "Run integration tests and dev builds with race detection" ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

This reverts the sdk and pkg changes from commit
655b76d030.

It also disables race detection from all builds and tests.

Fixes https://github.com/pulumi/pulumi/issues/15991.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-19 16:57:16 +00:00
Thomas Gummerer 69f5b0d83f
fix goproxy environment variable in makefile ()
# Description

~sourcegraph.com/sourcegraph/appdash-data disappeared from
proxy.golang.org. As a very temporary workaround we can use goproxy.io,
which still has it in its cache. However we should still try to get rid
of the dependency as a high priority.~

While trying to fix the sourcegraph.com/sourcegraph/appdash-data
dependency disappearing issue, one idea that came to mind was trying to
use a different goproxy. We already have a `GOPROXY` variable set in our
Makefiles, so just changing that should have worked. However that
variable had two problems:
- It was quoted in single quotes, which because this is a Makefile and
not shell were added to the environment, confusing Go.
- The variable was not exported, so it would never actually be used. 

Fix these two problems, in case we need to use this in the future.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-19 14:05:01 +00:00
Pat Gavlin 6756c12fd0
[config] Clean up implementation ()
These changes replace the idiosyncratic implementation of some of the
config Map and Value APIs with (hopefully) more straightforward code.

The fundamental representation of a config.Value remains a (value,
secure, object) tuple, where value is either a plain, possible-encrypted
string value or the JSON encoding of an object value. All operations on
values that need to observe the object value itself still decode the
JSON representation into a richer representation. This richer
representation, however, is no longer composed of `any` values: instead,
it is composed of `object` values. These values contain a restricted set
of types and directly track whether or not their contents are a secure
string value. The object-based representation allows for much clearer
implementations of the marshaling and traversal code without breaking
compatibility.

In addition to the new implementation for config.Value, these changes
add a config.Plaintext type that represents a plaintext config value. A
Plaintext value can be created manually or by decrypting a Value, and
can be encrypted and converted to a Value. This allows for more natural
creation and manipulation of config values.
2023-10-09 04:51:21 +00:00
Fraser Waters e5983715bf Fix makefiles 2023-05-26 11:00:45 +01:00
Abhinav Gupta 1e178fb6b5
make: Set -race=true for tests
Run tests in CI with data race detection turned on.

Refs 
2023-01-23 10:01:41 -08:00
Fraser Waters f4ad5c5a56 Remove dotnet 2022-12-13 16:13:53 +00:00
Aaron Friel 2af74b6469 ci: Use reduced smoke testing on Windows & macOS targets 2022-09-21 09:55:06 -07:00
Aaron Friel 746c5f6e3c ci: Enable robust retries on tests 2022-09-14 12:19:09 -07:00
Anton Tayanovskyy b06b860c2a
Remove PROJECT_ROOT Make var () 2022-06-24 10:04:40 -04:00
Ian Wahbe 101847c3ed
Only rebuild protobuf when necessary ()
* Hash *.proto files

* Hash other build files

* Clarify language
2022-06-22 09:08:07 -07:00
Ian Wahbe 9bad651337
Cleanup make ensure ()
* Don't format generated files

* Allow ensure to be stateful

* Cleanup comments and echo flags
2022-05-16 16:47:04 -07:00
Matthieu Coudron d4b9d61d70
allow to override makefile variables ()
* allow to override makefile variables

When packaging pulumi-python for www.nixos.org, there is no /bin/bash
available so allow for an alternative path to the shell.

Same for versions, PYPI version can be set via the environment variable
PULUMI_VERSION.

* Update sdk/nodejs/Makefile

Co-authored-by: Fraser Waters <frassle@gmail.com>

* Update sdk/go/Makefile

Co-authored-by: Fraser Waters <frassle@gmail.com>

Co-authored-by: Fraser Waters <frassle@gmail.com>
2022-03-25 21:56:11 +00:00
Aaron Friel 366072a1d4 ci: tolerate and retry fluke test failures 2022-03-15 09:51:43 -07:00
Aaron Friel ed2923653c ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
Aaron Friel 7b7ec02521 ci: divide and conquer integration tests by sdk and package group 2022-03-04 18:08:23 -08:00
Ian Wahbe 06ba63bb57
Use black code formatter for the python sdk ()
* Add black as a lint + format target for python

* Update lib/pulumi formatting
2022-02-04 01:16:16 +01:00
Anton Tayanovskyy 9b5591cdc4
Revert "Experiment with a locked go toolchain to reduce OOM ()" ()
This reverts commit 722d8344e1.
2022-01-21 17:53:39 -05:00
Anton Tayanovskyy 722d8344e1
Experiment with a locked go toolchain to reduce OOM () 2022-01-14 10:20:36 -05:00
Anton Tayanovskyy 76ba788985
Toward replacing MSBuild with make+bash on Windows ()
* Add coverage folder

* Adjust GHA to run Test target on Windows

* Extend error message on packages not found

* Try with path normalizer

* Edit Makefiles

* Skip SDK codegen tests on Windows

* Skip program gen tests on Windows

* Skip tests that started running on Windows and failing

* Fix non-compiling test

* Skip failing Windows HCL2 tests

* Fix lint

* Merged constants

* Skip one more test failing on Windows

* Disable cov-enabled builds on Windows

* Fix  TestDeterminePulumiPackages on Windows

* Fix TestInstallCleansOldFiles test on Windows

* Fix TestCreatingProjectWithPulumiBackendURL on Windows

* Weaken TestAbbreviateFilePath to pass on Windows

* Fix TestDepRootCalc on Windows

* Fix LocalUrl() to be sensible on Windows

* Fix Go lint issue

* Windows fix for TestComments

* Cross-ref skip tracking issue

* Cross-ref issue to fix asset_test skips

* More cross-ref issues

* Use choco not chocolatey

* Use yarn run to ensure the right tools are selected

* Revert python3->python change in common.mk

* In CI context, use python not python3

* More randomness in temp folder names to aoid Windows collisions

* Go lint
2022-01-07 22:27:14 -05:00
Ian Wahbe 3f79185040
Move `PULUMI_ROOT` to `$HOME/.pulumi-dev` ()
* Move PULUMI_ROOT to relative path $REPO/.bin

* Add .bin to gitignore

* Change home to $HOME/.pulumi-dev

* Cleanup nodejs/readme and remove .bin (gitignore)
2021-12-15 12:32:41 -08:00
Ian Wahbe c95e49e5ab
Use $(PYTHON) for makefile ()
* Use $(PYTHON) for makefile

* Normalize linting
2021-12-08 13:39:39 -08:00
Ian Wahbe b9f57bc6b9
Move `/opt/pulumi` to `$HOME/.pulumi` ()
* Move `/opt/pulumi` to `$HOME/.pulumi`

* Don't suggest adding $HOME/.pulumi to PATH
2021-11-17 10:37:38 -08:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment ()
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
Lee Briggs 0ac075ef8f
add initial pull-request workflow ()
* add initial pull-request workflow

* run SDK test all

* add SDK tests

* fixup make targets

* add dist target

* revert back to 5 updates

* disable test

* add issue for test disabling
2020-09-09 13:37:03 -07:00
Lee Briggs 09c73c1c9b
add build tags to all integration tests ()
* add build tags to all integration tests

* fix go sdk imports

* make transformation Dirs global
2020-09-02 09:11:15 -07:00
Lee Briggs 8c314ec39a
Add brew make targets to sdk makefiles 2020-05-13 20:42:01 -07:00
evanboyle 2d03741a36 add erroneously removed yarn command back to make file 2020-03-18 18:17:18 -07:00
evanboyle d65c65122f update builds to handle new go mod layout 2020-03-18 16:41:23 -07:00
stack72 25aeb237ca Move from go 1.12.x to go 1.13.x 2019-11-18 14:49:31 +01:00
CyrusNajmabadi df06b8fc9b
Add publishing to nuget support () 2019-10-29 20:14:49 -07:00
Matt Ellis b019bc571e Use Python 3
Use `python3` and `pip3` explicitly, as we now depend on Python 3.
2019-09-13 11:58:23 -07:00
david raistrick a603f4f904 fix unquoted $PATH consumption so it doesnt break when paths have spaces. 2019-06-19 12:42:26 -07:00
Kraig Amador 23a52ec0bb
Documentation update to include a step missing that helps new users build Pulumi 2019-05-02 10:18:02 -07:00
James Nugent edab10e9c8 Use Go Modules for dependency tracking
This commit switches from dep to Go 1.12 modules for tracking Pulumi
dependencies. Rather than _building_ using Go modules, we instead use the `go
mod vendor` command to populate a vendor tree in the same way as `dep ensure`
was previously doing.

In order to prevent checksum mismatches, it was necessary to also update CI to
use Go 1.12 instead of 1.11 - which also necessitated fixing some linting errors
which appeared with the upgraded golangci-lint for 1.12.
2019-04-10 08:37:51 +04:00
Sean Gillespie 26cc1085b1
Install missing plugins on startup ()
* Install missing plugins on startup

This commit addresses the problem of missing plugins by scanning the
snapshot and language host on startup for the list of required plugins
and, if there are any plugins that are required but not installed,
installs them. The mechanism by which plugins are installed is exactly
the same as 'pulumi plugin install'.

The installation of missing plugins is best-effort and, if it fails,
will not fail the update.

This commit addresses , where users using Pulumi
in CI often found themselves missing plugins.

* Add CHANGELOG

* Skip downloading plugins if no client provided

* Reduce excessive test output

* Update Gopkg.lock

* Update pkg/engine/destroy.go

Co-Authored-By: swgillespie <sean@pulumi.com>

* CR: make pluginSet a newtype

* CR: Assign loop induction var to local var
2019-03-15 15:01:37 -07:00
CyrusNajmabadi 37912b1e57
Make it simpler to run go tests consistently () 2019-02-26 13:39:33 -08:00
CyrusNajmabadi 9225c0d72d
test-all should not rerun test-fast () 2019-02-25 13:01:52 -08:00
Matt Ellis adb0d9aee6 Use pipenv from the toolset
- Remove the forked copy of the toolset
- Stop installing `pipenv` in sdk/python/Makefile

After this, we'll require that you already have `pipenv` present
before building.
2018-11-08 12:09:21 -08:00
Sean Gillespie 56be1a6677
Implement RPC for Python 3 ()
* Implement RPC for Python 3

* Try not setting PYTHONPATH

* Remove PYTHONPATH line

* Implement Invoke for Python 3

* Implement register resource

* progress

* Rewrite the whole thing

* Fix a few bugs

* All tests pass

* Fix an abnormal shutdown bug

* CR feedback

* Provide a hook for resources to rename properties

As dictionaries and other classes come from the engine, the
translate_property hook can be used to intercept them and rename
properties if desired.

* Fix variable names and comments

* Disable Python integration tests for now
2018-10-31 13:35:31 -07:00
Chris Smith f324a460e9
Remove references to Pulumi private clouds ()
* Remove TODO for issue since fixed in PPCs.

* Update issue reference to source

* Update comment wording

* Remove --ppc arg of stack init

* Remove PPC references in int. testing fx

* Remove vestigial PPC API types
2018-10-24 13:50:35 -07:00
joeduffy b6724c7892 Lock to specific dep version
This changes the Dockerfile to install the same specific dep version
we use in CI. To ensure we don't end up duplicating logic, it refactors
the versions themselves into a separate sourceable script that's shared
between both the Dockerfile and our existing CI scripts.
2018-09-29 16:33:21 -07:00
Matt Ellis c8b1872332
Merge pull request from pulumi/ellismg/fix-1581
Allow eliding name in pulumi.Config .ctor
2018-08-08 14:16:20 -07:00
Matt Ellis a337f17c70 Add a small end to end test for config capturing 2018-08-08 13:49:37 -07:00
Thomas Schersach 62463ab3bc Added dist target for make, will help with Homebrew ()
* Added dist target for make, will help with Homebrew

* Try to install go dependencies before building

* Make sure dep ensure is called before trying to build SDKs

* Removed dep ensure from dist initial step
2018-08-08 13:00:42 -07:00
Chris Smith 6958ad4073
Merge updates to install-common-toolchain.sh () 2018-07-31 23:08:19 -07:00
Matt Ellis 1f638410c7 Include README in pypi package
Use `pandoc` to convert our README.md to a README.rst and then include
that as the long_description in our pypi package.
2018-07-13 18:19:42 -07:00
Sean Gillespie 89b052fc6d
Use Pipenv to manage Python environments ()
* Use Pipenv to manage Python environments

* Rename PIPENV_ARGS to PIPENV_PYTHON_VERSION to avoid confusion. Also remove two unused variables
2018-06-21 18:01:23 -07:00
Matt Ellis 2cfbe81adb Remove dependency on pulumi/home
Stop cloning pulumi/home. This doesn't work in Travis because public
repositories can not have private SSH keys, which we'd need to clone
this repository. All the scripts we consume from there are now in
pulumi/scripts and so we'll just consume them from there.
2018-06-17 22:09:15 -07:00