Commit Graph

23 Commits

Author SHA1 Message Date
Luke Hoban d75251c49f
Add an `organization` intrinsic to PCL ()
We have added `getOrganization` fucntions to all SDKs. So we can now
lift this into PCL. This is needed to support `pulumi.organization` in
YAML, so that convert from YAML can target PCL and other downstream
languages.

Part of https://github.com/pulumi/pulumi-yaml/issues/461.

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-08-19 03:58:19 +00:00
Fraser Waters e9ebdab330
Add a conformance test for invokes ()
This adds a new provider with just a couple of invokes and a single
resource, and two small projects that uses that invoke to fill in a
stack output.

The first just uses one of the invokes to fill stack outputs. 

The second tests that output'y inputs work and that no arguments to the
invoke work.

Currently disabled for Go because it doesn't handle the multi-return
invoke expression in the stack output value context.
2024-08-05 04:32:07 +00:00
Fraser Waters c9c30f0939
Ensure internal provider state doesn't clash with user config ()
Provider internal state is now separated from provider user config. This
allows providers to use a config key like "pluginDownloadURL" which
previously would have conflicted with the engines internal state. It
also allows the engine to add new internal state fields without concern
for clashing with existing or future provider config keys.

"version" is maintained as a root level key because providers already
look at this field, and we use it in conformance tests. However it's
been fixed to be the package version, not the plugin version. That's the
same for normal providers, but for a parameterised provider it will be
the version of the parameterised package, not the base plugin version.
As part of this I've made schema more strict that a provider can't
define its own version field. This would have clashed with the use of
version to set the plugin version to download, and is better set by
users by using the version resource option.

Fixes https://github.com/pulumi/pulumi/issues/16757.

---------

Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-07-30 12:22:32 +00:00
Fraser Waters 4516fbce89
Add a test of sending primitive values for a resource ()
We already had the "simple" provider test that just checked we could
make a resource with a single simple bool field. This adds a new
"primitive' provider and checks we can send strings, numbers, integers,
arrays and maps from the program.
2024-07-22 21:13:11 +00:00
Fraser Waters a9947b4e4e
Python parameterized provider test ()
This adds support for replacement parameterised providers to Python and
a small integration test to check it works e2e.

When using parameterised providers we need to use the new (currently
unstable) RegisterPackage system, instead of sending
Version/DownloadURL/etc via RegisterResourceRequest. Once
RegisterPackage is stable the intention is to change _all_ packages to
use it and for normal packages to fall back to the
RegisterResourceRequest options, while parameterised packages will
error.

The actual parameter value is embedded in the python SDK as a base64
string that we decode before sending to the gRPC endpoint as bytes.
2024-07-16 10:55:38 +00:00
Will Jones a0cea65e70
Add conformance tests for remote assets ()
Conformance tests allow us to test functionality regardless of
programming language used, by setting up a test program using PCL and
making assertions about program state before and after various
operations. This commit adds a set of conformance tests for remote
assets (that is, assets that must be retrieved from some URI), which
have until now been untested in this regard.
2024-07-03 09:03:24 +00:00
Fraser Waters 969e0b9735
Conformance test for provider pre-release versions ()
Noticed this issue while doing SDK gen for parameterised providers, but
figured it deserved its own conformance test. Check that if a provider
has a pre-release semver that the _exact_ version can be reported by the
generated SDK. This already just works for NodeJS, but Python needed a
fix to write the version to `_utilities.py` rather than trying to
unconvert the pypi version from the package.

Also needed to make the conformance test checks for
`GetProgramDependencies` even weaker (which is fine, they are just a
very basic sanity check) because the provider reports a version of
"3.0.0-alpha.1.internal" while the python version is "3.0.0a1+internal".
2024-06-29 10:07:14 +00:00
Fraser Waters bf806f5834
Add asset archive test to conformance tests ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/16092.

Hopefully the really last part of the fixes started by
https://github.com/pulumi/pulumi/pull/16100 and
https://github.com/pulumi/pulumi/pull/16119. This fixes the working
directory handling for asset archives.
2024-06-24 14:23:18 +00:00
Daniel Bradley c6a01328d2
Set range in python codegen default SDK version ()
If the schema doesn't set any specific dependencies, we should be
defaulting to requiring pulumi within the current major version rather
than completely unconstrained.

Discussion:
https://github.com/pulumi/pulumi-kafka/pull/410/files#r1594324390
2024-06-24 13:57:57 +00:00
Will Jones 271553d462
Generate `@deprecated` decorators in Python SDKs ()
The latest version of the core Pulumi SDK contains a decorator,
`@deprecated`, that is used when generating SDK code in order to signify
deprecated properties in a way that can be recognised by other SDK code.
This is useful when writing generic Python code that e.g. traverses
class properties without triggering deprecation warnings for those not
explicitly mentioned in user code. The [original pull
request](https://github.com/pulumi/pulumi/pull/16400) has more details.

Alas, we can't rely on the fact that a user will upgrade _both_ a
particular (generated) provider SDK and the core Pulumi SDK at the same
time. Thus, it's entirely possible that a user bumps their version of
(say) `pulumi_aws`, whilst leaving their `pulumi` library at the same
(compatible, according to specified bounds) version. In doing so they'd
hit errors when the new SDK tried to import the `@deprecated` decorator,
which doesn't exist in the old core SDK.

This commit thus fixes this by altering code generation so that each SDK
receives its own inlined copy of the `@deprecated` decorator, which it
can reference regardless of the version of the core SDK. This decorator
applies the same `_pulumi_deprecated_callable` tag to functions it
decorates, which a sufficiently modern SDK will recognise to avoid
triggering e.g. https://github.com/pulumi/pulumi/issues/15894. Later on,
we can hopefully find a way to avoid doing this and use only a version
of `@deprecated` specified in the core SDK.

Codegen tests have been updated and the inlined decorator has manually
been tested using the AWS Classic SDK.

Addresses
https://github.com/pulumi/pulumi/pull/16400#discussion_r1646562455
2024-06-21 11:34:29 +00:00
Fraser Waters ee6ec150d8
Add explict provider test to conformance tests ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Adds a conformance test for an explicit provider resource.
2024-06-11 14:56:08 +00:00
Fraser Waters 3d4291e2f8
Fix folder archives in the engine ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/16092.

Last part of the fix started in
https://github.com/pulumi/pulumi/pull/16100. That last PR fixed assets
and archive files, but not archive folders.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-06 07:26:48 +00:00
Will Jones 6f4cb73ba9
Sort `requirements.txt` files when generating Python code ()
A couple of our tests now generate multi-line `requirements.txt` whose
order is dependent on that in which we enumerate packages when testing.
This PR changes this so that we always sort them, meaning we can
consistently check against snapshotted results (as opposed to having
flaky tests that only pass when the generated order matches the one we
happened to snapshot last).

Fixes https://github.com/pulumi/pulumi/issues/16113
2024-05-03 15:07:56 +00:00
Fraser Waters bb6b9c675a
Add asset/archive to conformance tests and fix engine working dir issues ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/16092.

Two main changes in this PR. Firstly we add a test for assets/archives
to the conformance tests, and had to fix up some small issues in the
test runner to enable this (copying test data files, correcting project
directory layout, handling multi-word package names).

The other change is to fix the engine to respect the programs working
directory when serialising/deserialising assets and archives. It does
this by passing the programs working directory as a MarshalOption (n.b
this option is only really needed in the engine, program SDK doesn't
really need it because it doesn't change directories).

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-02 11:32:54 +00:00
Fraser Waters f1f4d223f4
Add large string conformance test ()
In the hopes of having a conformance test to trigger
https://github.com/pulumi/pulumi-dotnet/issues/249, this adds a
conformance test that has a provider send back a large (~100mb) string
which is then exported as a stack output.
2024-04-25 08:14:34 +00:00
Thomas Gummerer 4169755866
Implement up --continue-on-error ()
Similar to destroy --continue-on-error, this flag allows `pulumi up`
to continue if any errors are encountered.

Currently when we encounter an error while creating/updating a
resource, we cancel the context of the deployment executor, and thus
the deployment stops once the resources that are being processed in
parallel with the failed one finish being updated.

For --continue-on-error, we ignore these errors, and let the
deployment executor continue.  In order for the deployment executor to
exit eventually we also have to mark these steps as done, as the
deployment executor will otherwise just hang, and callers with open
channels waiting for it to finish/report back will hang indefinitely.

The errors in the step will still be reported back to the user by the
OnResourceStepPost callback.

Fixes https://github.com/pulumi/pulumi/issues/14515

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-04-22 11:12:45 +00:00
Fraser Waters 901c2f5e9c
Add StackReference conformance test ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/15932.

This adds a conformance test that checks that StackReferences work.
Tests a plain string and a secret string output.

To support this test we add a new intrinsic `getOutput` that takes a
stack reference resource and a string and calls the
`get_output/getOutput/GetOutput` method on the stack reference resource
type.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-04-16 11:13:25 +00:00
Fraser Waters 9cd05b41b1
Add array conformance test ()
Add an array output test to the conformance suite. Similar to the other
basic output tests.
2024-04-10 16:00:24 +00:00
Fraser Waters 883f6b6ac9
Add negative number to the number output conformance test () 2024-04-06 12:31:58 +00:00
Fraser Waters 922823e9c8
Test string outputs in conformance tests ()
This adds a conformance test that we can generate programs that return
various strings as stack output values.
2024-03-29 14:24:29 +00:00
Fraser Waters 553f23cf5f
Add a conformance test for number outputs ()
This adds a conformance test that we can generate programs that return
various numbers as stack output values.
2024-03-29 12:12:31 +00:00
Fraser Waters d45ac04b46
Make local dependencies absolute or relative paths ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

We need this for local dependencies in conformance tests for SDKs and
programs. Things like NodeJS just record the path as is in the
package.json put into the packed tgz, if a program then uses that tgz
via a relative path NodeJS doesn't re-resolve the relative relative
paths.
Making these absolute for conformance testing fixes that as all the
conformance tests do run in a stable folder location. We fix writing
that folder path to the snapshots using the new regex replace facility
added in https://github.com/pulumi/pulumi/pull/15747.


## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-25 10:27:56 +00:00
Fraser Waters 2366960ae5
Test both setup.py and pyproject.toml in python conformance tests ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Run python conformance tests with setup.py and pyproject.toml. This is
to ensure SDK generation and resolution works correctly for both project
styles.

Hopefully at some point we can deprecate and remove setup.py support,
and halve the number of tests needed to run here.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-22 21:33:14 +00:00