Commit Graph

399 Commits

Author SHA1 Message Date
Zaid Ajaj 0b250fcf2a
[sdk-gen/go] Fix generating input collection types when used from an array of map of enums ()
# Description

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-05 12:29:30 +00:00
Zaid Ajaj c47cf5e2cd
[sdk-gen/go] Fix compiling plain element type with plain maps ()
# Description

Fixes  when choosing whether to use `ObjectName` (plain) or
`ObjectNameArgs` (non-plain) we have to check whether the non-plain
version was actually available in first place (sometimes it isn't when
the type isn't used anywhere as non-plain) so the fix here is to use the
package context that tracks the usages to determine whether the `Args`
suffix should be used.

Funny though, for other languages like C#, TypeScript and Python we
don't have a concept of a strictly plain type. Instead all types are
non-plain (accepting values for their properties being inputs/outputs)
which means that a plain element type on maps for these languages has no
effect

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-04 12:34:40 +00:00
Zaid Ajaj 59c3de65a2
[sdk-gen/go] Generate non-plain type variants for types used as inputs inside unions ()
# Description

When using types in union cases (using `oneOf` in the schema) then Go
SDK-gen doesn't detect these types as being used as inputs and only
emits the plain version of a type. That is unless
`generateExtraInputTypes: true` is specified in which case Go SDK-gen
will emit the non-plain types as well.

In the case of azure-native modules, `generateExtraInputTypes` is
currently set to `false` (default) as of latest v2.19.0 and is missing a
few non-plain type variants (for example
`FirewallPolicyFilterRuleCollection` from [this
file](https://raw.githubusercontent.com/pulumi/pulumi-azure-native-sdk/master/network/pulumiTypes.go)).
There no single plain type in azure-native and yet it is missing these
non-plain variants.

This PR fixes that by also traversing the element types of schema union
definitions when determining plain-ness of types and how they are used.

I added an example schema with an array of unions (common case in
azure-native) and confirmed that the change actually results in the
non-plain types being generated when `generateExtraInputTypes` is set to
`false`

Fixes https://github.com/pulumi/pulumi-azure-native/issues/1922 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-29 12:37:53 +00:00
Zaid Ajaj 78ccc7437d
[sdk-gen] No longer writing out name and project from alias definitions into generated SDKs ()
# Description

After adding a test schema for aliases in  @Frassle noticed it was
incorrect to write out the name and project of aliases into the SDKs.
This PR fixes this such that we now only write out the `type` of the
alias into generated code.

Supersedes 

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-21 22:40:14 +00:00
Zaid Ajaj 70ba80b7cc
[go/sdk-gen] Fixes plain and optional properties for generated types for Go SDKs using generics ()
# Description

While working on on  I tried to add a test schema for assets and
archives that generate a `generics-only` go SDK but the the output
didn't compile. At first I thought the issue was specific to asset and
archive types but actually it was a broader issue where the
function`pkg.genPlainType(InputObjectType)` would always reduce
`Input[T]` and `Optional[T]` to just `T` on each property of
`InputObjectType`.

It is probably fine to reduce `Input[T]` because we are generating a
plain type after all. However, reducing `Optional[T]` to `T` is
incorrect because the generic variant of go sdks are more strict about
optionality of types.

> It probably works in non-generic SDKs today because it relies on a
runtime cast


Example type `TypeWithAssets` from schema that has a plain and optional
property called `plainAsset`:

  Before it was the following and it didn't compile for generic go sdks
```go

type TypeWithAssets struct {
	PlainAsset   pulumi.AssetOrArchive `pulumi:"plainAsset"`
}

type TypeWithAssetsArgs struct {
	PlainAsset   pulumix.Input[*pulumi.AssetOrArchive] `pulumi:"plainAsset"`
}

func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
	return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) pulumi.AssetOrArchive { return v.PlainAsset })
}
```
 Now it generates:
```go
type TypeWithAssets struct {
	PlainAsset   *pulumi.AssetOrArchive `pulumi:"plainAsset"`
}

type TypeWithAssetsArgs struct {
	PlainAsset   *pulumi.AssetOrArchive               `pulumi:"plainAsset"`
}

func (o TypeWithAssetsOutput) PlainAsset() pulumix.Output[*pulumi.AssetOrArchive] {
	return pulumix.Apply[TypeWithAssets](o, func(v TypeWithAssets) *pulumi.AssetOrArchive { return v.PlainAsset })
}
```
Which is correct and compiles

The behavior for current non-generic SDKs remains unchanged

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-21 16:43:51 +00:00
Fraser Waters c56bb05174
Update golangci-lint () 2023-11-21 15:16:13 +00:00
Anton Tayanovskyy ba039c20f8
Support returning plain values from methods ()
Support returning plain values from methods.

Implements Node, Python and Go support.

Remaining:

- [x] test receiving unknowns
- [x] acceptance tests written and passing locally for Node, Python, Go
clients against a Go server
- [x] acceptance tests passing in CI
- [x] tickets filed for remaining languages
   - [x] https://github.com/pulumi/pulumi-yaml/issues/499
   - [x] https://github.com/pulumi/pulumi-java/issues/1193
   - [x] https://github.com/pulumi/pulumi-dotnet/issues/170 

Known limitations:

- this is technically a breaking change in case there is code out there
that already uses methods that return Plain: true

- struct-wrapping limitation: the provider for the component resource
needs to still wrap the plain-returning Method response with a 1-arg
struct; by convention the field is named "res", and this is how it
travels through the plumbing

- resources cannot return plain values yet

- the provider for the component resource cannot have unknown
configuration, if it does, the methods will not be called
- Per Luke https://github.com/pulumi/pulumi/issues/11520 this might not
be supported/realizable yet

<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Fixes https://github.com/pulumi/pulumi/issues/12709

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-18 06:02:06 +00:00
Zaid Ajaj 9899992657
[go/sdk-gen] Fix generics-only option missing ToOutput(...) methods ()
# Description

PR  removed `ToOutput(...)` methods for output types when
generating non-generic SDK variants. The fix there was added to only
include these when `side-by-side` is enabled, which implicitly included
`generics-only` except for a _single_ place where that wasn't the case 😓
(see comment below in `gen.go`)

This PR fixes issue and includes tests for go sdkgen where `generics`
setting is set to `generics-only`:
 - `output-funcs-go-generics-only`
 - `plain-and-default-go-generics-only`
 - `secrets-go-generics-only`
 - `simple-enum-schema-go-generics-only`
 - `simple-plain-schema-go-generics-only`

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-16 18:41:57 +00:00
Zaid Ajaj 21f33f2cf8
[go/sdkgen] Remove generic ToOutput methods from non-generic go SDKs ()
# Description

Fixes  by removing generic `ToOutput*` methods on types when
generating providers in non-generic setting . This means that we will
only generate these methods when `generics` setting is `side-by-side`
since `generics-only` will not touch these anyways

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-04 12:17:41 +00:00
Zaid Ajaj ac6adb98bf
[go/program-gen] Fix using inline invoke expressions inside resources, objects and arrays ()
# Description

This PR fixes an issue in Go program where if users are writing invoke
expressions inline inside other expressions, then that invoke is
extracted into a temporary variable and then later referenced the same
way it was used. This is because non-output-versioned invokes cannot be
used directly since they return a tuple (InvokeResult, err) so we need
to check for the error before proceeding.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-03 14:22:43 +00:00
Thomas Gummerer 0d6fff3995
codegen/go: output correct array type ()
Currently for an array of objects, codegen for Go outputs a
`pulumi.AnyArray` type. However that does not exist. Make sure it always
outputs the correct `pulumi.Array` type.

While there give Maps the same treatment, as `pulumi.AnyMap` also
doesn't exist.

Fixes #https://github.com/pulumi/pulumi/issues/8072

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-20 16:12:25 +00:00
Fraser Waters cf5b4a2790
Use `assert.NoError` rather than `assert.Nil` ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->
Likewise `require.NoError` instead of `require.Nil`, and `assert.Error`
rather than `assert.NotNil`.

The error variants of these functions print the errors nicer for test
failures using `Error()` rather than `GoString()`.

For bail errors this is _much_ better than the `result.Result` days
where we now get errors like:
```
Error:      	Received unexpected error:
            	BAIL: inner error
```
instead of:
```
Error:      	Expected nil, but got: &simpleResult{}
```

Also print the bail error in `TestPlan.Run` so we can see the
description of it.
2023-10-13 09:46:07 +00:00
Zaid Ajaj d90aef60f9
[go/sdk-gen] Generate output-versioned invokes for functions without inputs ()
# Description

Partially addressing  implements output-versioned invokes for
functions without inputs for go.


## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-02 12:53:06 +00:00
Zaid Ajaj a28454477c
[go/sdk-gen] Generating SDK with generics ()
# Description

This PR implements generating the generic variant of a go SDK from
Pulumi schemas. Currently the idea is to generate a directory `x` inside
the root directory of the go SDK which will contain the same SDK except
using generics and generating far less code than its current counter
part.

Also implements an enum option `$.language.go.generics` which can be set
to the following:
- `none` is the default which maintains the current behavior that
generates legacy types without generics
- `side-by-side` generates the generics sdk variant alongside the
current sdk under directory `x`
- `generics-only` generates only the new sdk with generics at the root
of the package
 
Still a bunch of things to do:
- [x] Generating `InvokeResult]Output` type from `Output[InvokeResult]`
and generating accessor methods for it
- [x] Generating default values for types and using the `pulumix`
subpackage to do so
- [x] Generating generic SDK variants for all test schemas we have and
making sure they compile (currently only testing
`simple-resource-schema` as shown below)
 - [x] Account for plain inputs for components
- [x] Combine pulumix.Join with pulumix.Apply to generate resource
accessor methods
- [x] Problem with `GPtrOutput[T]` and `ArrayOutput[T]` being unwrapped
to `Output[*T]` and `Output[[]T]`
 - [x] Remove excess untyped container types from generated enums
- [x] Fix default values for resource methods with lifted single return
value
 - [x] Secret properties

Currently the following test schemas have opted for `generics:
"side-by-side"`:
 - [x] output-funcs
 - [x] simple-enum-schema
 - [x] secrets
 - [x] simple-plain-schema
 - [x] plain-and-default

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-09-19 10:28:50 +00:00
Abhinav Gupta 025add9c3e
codegen/go: Implement pux.Input[T] for generated types ()
For generated types that impleemnt `pulumi.Output`,
or those that implement `pulumi.Input` with a means of converting
themselves to a `pulumi.Output`,
also generate `ToOutput(context.Context) pux.Output[..]`
to satisfy the `pux.Input[T]` interface.

This allows all these generated types to be used with `pux.Apply`
and other type-safe generic APIs per .

Resolves 
2023-08-28 16:42:37 +00:00
Zaid Ajaj 518d62e205
[go/sdk-gen] Implement option to override the name of the generated internal module ()
# Description

This PR adds a schema option specific to go at
`$.language.go.internalModuleName` which, when set, overrides the name
used for the internal/utilities module and everywhere it is used (see
added test example below)

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->

---------

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2023-08-22 17:16:43 +00:00
Luke Hoban 46e2753d02
Require Go 1.20 for Go Programgen ()
We added program generation that assumes availability of generics as
part of https://github.com/pulumi/pulumi/pull/13149, but didn't bump the
version of Go required in the generated go.mod. Currently, Pulumi is
supported on the supported versions of Go, which includes Go 1.20 and
1.21, so here we bump the Go version to the minimum currently supported
version, which also supports all of the code we generate today.

I was curious how this was not already failing tests, and it appears
it's because we delete the `go.mod` file in our tests and recreate it
with `go mod init`. I am unclear why we do that, but it feels like we
should not, exactly to avoid this sort of problem in the future and to
test the actual code we generate.


8696695cdb/pkg/codegen/go/test.go (L25-L28)

Fixes .
2023-08-16 04:59:07 +00:00
Fraser Waters 99b736b55b Add localDependencies option to GenerateProject
This isn't currently actually used anywhere. I've just threaded it
through to all the program gen functions where it will be needed.

Matrix testing will be using and testing this.
2023-08-08 12:28:19 +01:00
Zaid Ajaj 3e39cb2820 Normalize the declaration name of generated resource components 2023-07-27 15:15:27 +02:00
Abhinav Gupta a24ad1839b
programgen(go): Handle conflicting names in imported packages
This fixes how programgen generates import statements
to handle conflicting imports when two imported packages
have the same name, e.g.

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/ecs
    github.com/pulumi/pulumi-awsx/sdk/go/awsx/ecs

To do this, we add a fileImporter type that tracks these imports.
It prefers unnamed imports for packages unless one of the following is
true:

- the name of the package has already been used by another import
- the name of the package does not match the last component
  of the import path (e.g., `example.com/foo-go` with `package foo`).

If the name has already been used by another import,
it attempts the following in-order:

- Combine the last two path components of the import path
  into an identifier and use that if available.
  e.g., `awsxs3` from `sdk/go/awsx/s3`.
- Append a number to the package name and increment it
  until an unused name is found.
  e.g. `ecs2`, `ecs3`, and so on.

There's a change in how this information is tracked as well.
Previously, this was a pull approach: various calls returned
programImports objects which all got merged together.

This change switches to a push approach:
as code is generated and imports are requested,
they're submitted to the fileImporter which keeps track of them
until the next `Reset()` call.
The above also has a nice side effect of dropping a parameter.

Another change worth explicitly calling out:
Previously, getModOrAlias partially duplicated some of the logic
implemented in getPulumiImport, and used `mod`, `originalMod`
in a non-obvious way.
This generated incorrect imports like the following
(note the two `/aws` at the end):

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/aws

This change replicates more of the logic of getPulumiImport
(now called addPulumiImport) into this function,
and addresses the discrepancy in codegen caused by `mod`/`originalMod`.
The result leaves most existing code unchanged,
except in a couple existing cases where the resulting changes make sense
given the logic for named imports outlined above.

Resolves 
2023-07-25 12:49:37 -07:00
Zaid Ajaj a5975ca585 lint 2023-07-14 01:47:25 +02:00
Zaid Ajaj cd65005389 Fix panic in GenerateProject when version is not set in schema 2023-07-14 01:23:27 +02:00
Zaid Ajaj ed02926277 Allow generating code for unknown invokes in non-strict mode 2023-07-10 15:05:18 +02:00
bors[bot] 0d1d57f1f9
Merge
13136: Add explicit package versioning to Go codegen r=guineveresaenger a=guineveresaenger

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

This PR adds logic to explicitly set a Go SDK version as well as pluginDownloadURL in the SDK itself.
This will allow us to publish tags with explicit versions, ensure the engine references the correct plugin version, and will allow for the provider to use the correct version plugin much more reliably.

This PR adds a new `pulumiVersion` file and refactors the `pulumiUtilities` file to live alongside the new version file in an `internal` folder for utilities. This allows all resources in the provider to call on `internal.PkgVersion` without generating a mod-level utilities file alongside.

I have tested this against a flat structure SDK (pulumi-docker) and a modular SDK (pulumi-okta).
I have also edited the `schema.json` files in the unit tests to reflect the new utilities path.

The changes in the PR will allow us to:

1. Read a version into `pulumiVersion.go` during codegen
2. Commit and tag that commit as `sdk/v<Version>`
3. Have the exact same SDK that was built and tested be available on github
4. Clean up the CI step that explicitly tags the Go SDK.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Guinevere Saenger <guinevere@pulumi.com>
2023-07-07 21:33:39 +00:00
Guinevere Saenger f52d5d3baa add inline commend for isUtil bool 2023-07-06 13:20:04 -07:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
bors[bot] 9af5007b17
Merge
13212: [program-gen/go] Fix aliasing package names using dashes when schema doesn't include package info r=Zaid-Ajaj a=Zaid-Ajaj

### Description

Fixes  such that if a package doesn't have a specific go package override (like the one mentioned in the bug report) and needs an alias (because of using hypthens) then we apply aliasing for it. 

The PR adds a small synthetic schema called `using-dashes` that doesn't have any special go package info overrides and generates test programs from it. 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Zaid Ajaj <zaid.naom@gmail.com>
2023-06-23 09:52:45 +00:00
Abhinav Gupta da184df24a
fix(codegen/go): Use raw string literals for multiline-strings
We already have logic in place to generate mutliline strings with
backticks if they meet certain conditions in genTemplateExpression.
However, the logic came into effect only if the string was complex
enough to warrant use of `Sprintf`
because the function short circuits to just printing a string literal
if the string doesn't need a `Sprintf`.

This changes genStringLiteral to be responsible for the decision
of whether the string warrants backticks or not,
retaining the prior conditions on whether the string would benefit
from being split across multiple lines.

Resolves 
2023-06-22 10:55:45 -07:00
Abhinav Gupta 9d944a4a1f
test(codegen/go): Fix broken TestLiteralExpression
TestLiteralExpression appears to be broken on master:

```
% go test -run TestLiteralExpression
--- FAIL: TestLiteralExpression (0.00s)
    --- FAIL: TestLiteralExpression/"foo_${bar}_%baz" (0.05s)
        gen_program_expression_test.go:252:
                Error Trace:    [..]/pulumi/pkg/codegen/go/gen_program_expression_test.go:252
                Error:          Not equal:
                                expected: "fmt.Sprintf(\"foo %v %vbaz\", bar, \"%\")"
                                actual  : "fmt.Sprintf(\"foo %v%v\", bar, \" %baz\")"

                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1 +1 @@
                                -fmt.Sprintf("foo %v %vbaz", bar, "%")
                                +fmt.Sprintf("foo %v%v", bar, " %baz")
                Test:           TestLiteralExpression/"foo_${bar}_%baz"
```

Both versions are a valid way to represent that information,
so I'm fixing the test case.

I'm not sure how this broken test ended on master, though.
2023-06-20 12:25:42 -07:00
Zaid Ajaj 3bc0e6fe38 Fix aliasing package names using dashes when schema doesn't include go package info override 2023-06-20 18:11:47 +02:00
Guinevere Saenger 1690f91071 don't make every package internal 2023-06-16 09:18:13 -07:00
Guinevere Saenger 1829a17150 make lint 2023-06-15 19:22:26 -07:00
Guinevere Saenger d7efdeaf77 make genHeader function signature explicit 2023-06-15 10:34:42 -07:00
Guinevere Saenger 607a477a5f remove commented out code 2023-06-14 14:00:40 -07:00
Zaid Ajaj 7e5e452909 Extend SkipResourceTypechecking to allow generating unknown resources 2023-06-14 19:02:56 +02:00
Guinevere Saenger 6351e34c77 Generate new tests 2023-06-14 09:34:49 -07:00
Guinevere Saenger 108667c21f Codegen generates an internal package name, and references it correctly in Resource files and elsewhere. 2023-06-13 16:47:11 -07:00
Guinevere Saenger dde6e545db Export all functions in pulumiUtilities. Call them as internal. Do a blank import for the non top level module base path so info.go doesn't complain 2023-06-13 16:47:02 -07:00
Guinevere Saenger 10145b1b4a Import top level package in all resource files
All resources will need default options, so we reference the top
level module and export the resource options functions so they are
accessible in submodules.
2023-06-13 16:46:43 -07:00
Justin Van Patten 31d106980e [programgen/dotnet,go] Implement singleOrNone intrinsic
This change adds support for the `singleOrNone` intrinsic to Go and C# programgen.
2023-06-11 16:23:33 -07:00
Guinevere Saenger 44c2609b78 Explicitly refer to version package by name 2023-06-09 16:09:22 -07:00
Guinevere Saenger f98cb3908f ensure all submodules have the correct version package ref 2023-06-08 16:37:02 -07:00
Guinevere Saenger 9c9f4e1f40 finishing touches on templating logic 2023-06-08 16:00:35 -07:00
Guinevere Saenger 59640dd102 have defaults 2023-06-08 16:00:22 -07:00
Guinevere Saenger 03a7511c1f Create correct pulumiVersion.go file and reference it correctly in the config mod 2023-06-08 16:00:02 -07:00
Zaid Ajaj 2ce1fdb091 Fix panic in go program-gen when encountering splat expressions 2023-06-07 13:22:41 +02:00
Zaid Ajaj 703c53cfeb Turn error diagnostics into warnings 2023-06-02 17:17:59 +02:00
Zaid Ajaj af09d174cd Do not error out when generaing not yet implemented ForExpressions 2023-06-02 16:59:46 +02:00
bors[bot] 5528db96f6
Merge
13053: [go/program-gen] Do not error when generated Go code cannot be formatted r=Zaid-Ajaj a=Zaid-Ajaj

### Description

When generating go code in program-gen, we format the code before saving it to disk. However, if formatting fails, we don't generated anything at all and we simply error. This PR changes this behaviour such that we fall back to and generate the original, non-formatted code when formatting fails. 

Fixes part of https://github.com/pulumi/pulumi-terraform-bridge/issues/1158

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Zaid Ajaj <zaid.naom@gmail.com>
2023-06-01 08:56:23 +00:00
Zaid Ajaj 6c4952d4fd Add a warning diagnostic when formatting code fails 2023-05-30 19:32:57 +02:00