Commit Graph

106 Commits

Author SHA1 Message Date
Will Jones c496921d44
Enable some more linting rules ()
Issue  lists a number of extra linting checks that we could enable
in order to make our Go code more robust. This commit implements as many
as seem sensible:

* `durationcheck`, which checks for multiplication of `time.Duration`s,
which can lead to unexpected behaviour (e.g. `time.Second * time.Second`
is *not* one second)
* `goprintffuncname`, which checks that `Printf`-like functions are
appropriately suffixed with `f` to indicate as such
* `tenv`, which checks for `os.Setenv` in tests where `t.Setenv` is
generally a better solution
* `wastedassign`, which checks for assignments whose values are never
used (such as initial values before an `if` where both branches then
overwrite the value)
* `whitespace`, which checks for blank lines at the beginning and end of
blocks such as functions, `if`s, `for`s and so on.

This commit does *not* enable the following checks listed in :

* `wrapcheck`, which insists that third-party library errors are always
`%w`rapped -- we have a lot of cases where we don't do this and it's
probably a bit more involved than "just wrap them" in terms of making
sure we don't break anything (maybe)
* `predeclared`, which checks for shadowing of existing Go identifiers
-- we use `old` and `new` a lot, especially in step generation, so this
is probably a slightly bigger clean-up/one we might want to opt out of
* `mnd` (magic number detection) -- we have a lot of failures on this
* `nilnil` -- we only have a couple of failures on this; these could
probably be handled with `//nolint` but for now I've opted not to take
this route.
2024-10-03 17:37:13 +00:00
Zaid Ajaj d30f4b5cf6
[go/conformance] Fix l2-invoke-simple and l2-invoke-variants tests ()
### Description

This PR fixes l2-invoke-simple and l2-invoke-variants conformance tests
for Go. I was struggling originally to get plain invokes, which return
`(Result, error)` to be rewritten correctly into temp variables but it
broke a few more test examples and didn't go well.

This time, I chose the path of least resistance: enabling
`pcl.PreferOutputVersionedInvokes` which makes invoke functions in PCL
choose the output variant of the invoke regardless of whether their
inputs have anything of type output and the return type `ResultOutput`
is simpler to work with. We also do this for nodejs and python.

The function body of the output-versioned invokes in Go follow an almost
identical code path to the plain ones so this might be Good Enough™ to
assert correctness of invokes.
2024-09-27 13:25:54 +00:00
Zaid Ajaj 77d77233f5
Fix l1-output-map Go conformance test ()
### Description 

Fixing l1-output-map in Go program-gen because object expressions didn't
lift their values.

### Up for discussion

Making l1-output-map less strict about having an an _empty_ map output
register as an actual map without elements or not registering anything.
2024-09-26 17:21:05 +00:00
Julien 68524af701
Enable goheader rule and add missing license headers ()
This commit adds the `goheader` rule to `golangci-lint` to enforce that
all our Go source code includes appropriate licence headers, fixing up
files that currently fail that check.

---------

Co-authored-by: Will Jones <will@sacharissa.co.uk>
2024-09-09 12:05:45 +00:00
Luke Hoban d75251c49f
Add an `organization` intrinsic to PCL ()
We have added `getOrganization` fucntions to all SDKs. So we can now
lift this into PCL. This is needed to support `pulumi.organization` in
YAML, so that convert from YAML can target PCL and other downstream
languages.

Part of https://github.com/pulumi/pulumi-yaml/issues/461.

---------

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2024-08-19 03:58:19 +00:00
Fraser Waters 22d28187db
Use pulumi.Sprintf in program gen ()
Noticed by @thomas11 on Slack
(https://pulumi.slack.com/archives/C02FXTZEZ6W/p1722410344206549). We
had a number of places doing `pulumi.String(fmt.Sprintf(template,
args))`, when they could just use `pulumi.Sprintf`. This updates
programgen to use that.
2024-08-01 09:24:25 +00:00
Zaid Ajaj 4297d38ace
[Go] Re-enable l1-stack-reference conformance test ()
By fixing the emitted function for `getOutput` in Go program-gen which
is specific to stack references.

Locally the generated program compiles but I get a runtime error:
```
Error:      	Not equal: 
expected: resource.PropertyValue{V:"plain"}
actual  : resource.PropertyValue{V:(*resource.Secret)(0x14000aa6050)}

Diff:
--- Expected
+++ Actual
@@ -1,3 +1,7 @@
 (resource.PropertyValue) {
- V: (string) (len=5) "plain"
+ V: (*resource.Secret)({
+  Element: (resource.PropertyValue) {
+   V: (string) (len=5) "plain"
+  }
+ })
 }
Messages:   	expected property "plain" to be {plain}
```
However I _think_ this is a local problem with some cached Go SDK
artifacts because when go the implementation of `GetOutput` (from the
generated program) it brings me to a different implementation than the
one I find on latest SDK so testing in CI to make sure

EDIT: fixed by removing hardcoded `/` in the used path when replacing
Pulumi SDK reference

Fixes 
2024-07-30 08:12:38 +00:00
Zaid Ajaj 7c14d59c2b
[Go] Re-enable l1-output-string conformance test ()
The issue was that `NUL` characters weren't being escaped and ended up
in the literal string in the generated Go program which the Go compiler
isn't a big fan of. This PR fixes the problem.

Resolves 
2024-07-29 16:35:31 +00:00
Fraser Waters 678d916202
Remove an unused parameter from argumentTypeName ()
Noticed this while fixing https://github.com/pulumi/pulumi/pull/16753.
The first parameter to argumentTypeName was totally unused.
2024-07-23 07:03:26 +00:00
Fraser Waters 3460c9963f
Enable l1-output-number conformance test for Go ()
This required fixing `__convert` in Go programgen to handle expressions
that needed to result in input-y values. This also fixed the
"single-or-none" codegen test.
2024-07-22 17:12:28 +00:00
Zaid Ajaj f8236e125e
[docs/go-program-gen] Fix generating constructor syntax examples in Go for package awsx ()
Currently, when generating docs for the AWSX package, Go constructor
syntax examples were not generated due to a panic. This causes all
constructor examples to not be emitted in the docs.

The panic occurred when trying to get the version of referenced packages
in the PCL program to emit import paths. However, _transitive_ package
references were not resolved in the PCL binder when binding resource
types. This PR fixes the problem such that now we do find the transitive
package references from any input or output property of the resources
being bound.

In the case of the awsx package, the top-level package is awsx itself
and aws is the transitive dependency. Anytime in codegen we call
`program.PackageReferences()` we should get both of them. Added a unit
test for this as well.

Testing this fix locally against the awsx package showed constructor
examples being generated for every language, however there was still a
problem in the _formatting_ of Go code which is also fixed (see change
in `gen_program_expressions.go`)

Resolves part of 
2024-07-08 23:23:47 +00:00
Zaid Ajaj 2394f0de7a
[docs/go-program-gen] Fix generating constructor syntax examples for kubernetes ()
In this PR we fix generating constructor syntax examples for Kubernetes.
The problem initially was due to generating a PCL program with syntax
errors for the kubernetes schema. The syntax error was happening because
we emitted properties `$ref` and `$schema` which are not valid
identifiers in PCL. Fixing this was simple enough, we only needed to
quote these properties that start with dollar signs and we get a valid
PCL program.

Full PCL program for kubernetes:
https://gist.github.com/Zaid-Ajaj/abe899430a0b5f99a428934dcac75d52

However, the valid PCL program wouldn't convert to Go with program-gen
and it would hang without showing any errors or stack traces. I wrote a
script to split the programs for each resource and convert each
separately. This way I narrowed down the problem to this program:

```
resource "def" "kubernetes:apiextensions.k8s.io/v1:CustomResourceDefinition" {
  apiVersion = "string"
  kind = "string"
  spec = {
    versions = [{}]
  }
}
```

Debugging this in Go program-gen didn't show an obvious error and it
kept on hanging. However I noticed that we lowering expressions twice:
once for the entirety of resource inputs when generating resources and
again when generating the expressions separately. Lowering expressions
has been the source of many bugs and it seems to trip up program-gen a
lot and suspected something wrong was going on here, so I simplified it
to lower the expressions once when we generate resource inputs. This
fixed the hang issue as well as a few of our test programs (see diffs
for test programs and tests schemas)

Tested this against the full kubernetes schema and we can now generate
the full Go program:
https://gist.github.com/Zaid-Ajaj/3ac734536969a989edae92219968d51f

> The second time we lower expressions not only is redundant but also
can generate invalid code if the lowered expressions have temporary
variables because program-gen would just emit these variables inside
inline expressions like objects and lists which gives invalid Go code.

Resolves part of 
2024-07-05 12:42:41 +00:00
Zaid Ajaj de54e1aa47
[docs] Fix generating constructor examples for resources that have numeric enums as input ()
# Description

Fixes  

The original issue is that the intermediate PCL we generate used enum
names instead of enum values for numeric enum inputs. This PR changes it
so that the PCL program now uses the first numeric value for the first
enum case then subsequently fixing downstream program-gen bugs that
didn't know how to handle numeric values as inputs for enums.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-05-30 22:43:12 +00:00
Zaid Ajaj 7627d8097e
[program-gen/go,dotnet] Fixes emited code for object expressions assigned to properties of type Any ()
# Description

Fixes  by specifically handling object expressions that are
annotated as `Any` and emitting the correct code.

- In case of Go, expression `X` is emitted as `pulumi.Any(X)` where the
type name here is `map[string]interface{}`.
- In case of C#, we generate `Dictionary<string, object?>` expressions
(similar to what we do inside `toJSON` calls)
- Updates the test aws-native schema from 0.13.0 to 0.99.0

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-24 00:06:57 +00:00
Zaid Ajaj 3bdc65c6e5
[program-gen] Fix enum resolution from types of the form Union[string, Enum] and emit fully qualified enum cases ()
# Description

This PR improves enum type resolution from strings. When we try to
resolve `Union[string, Enum]` for a string expression, we choose
`string` because it is the more general type since not every string is
assignable to `Enum`. However, here we spacial case strings that are
actually part of that `Enum`.

The result is that `pcl.LowerConversion` will choose `Enum` from
`Union[string, Enum]` when the value of the input string is compatible
with the enum. This greatly improves program-gen for all of typescript,
python, csharp and go which now will emit the fully qualified enum cases
instead of emitting strings.

Closes https://github.com/pulumi/pulumi-dotnet/issues/41 which is
supposed to be a duplicate of
https://github.com/pulumi/pulumi-azure-native/issues/2616 but that is
not the case (the former is about unions of objects, the latter is
unions of enums and strings)

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-15 17:49:12 +00:00
Zaid Ajaj 27041c93d8
[program-gen/go] Fix optional primitive values being derefrenced ()
# Description

Fixes  where the primitive optional types were being derefrenced.
It took me a while to figure where the problem was because I couldn't
really reproduce the original issue with a new PCL program. The
`aws-optionals-pp` test case program in PCL would repro the issue when
converted from the CLI via `convert --from pcl` but not from the unit
tests even when using the exact same schema 🤔 this was really weird.
That said, the fix turned out straightforward and it does fix the same
issue in generated component resources.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-03-10 04:49:06 +00:00
Zaid Ajaj 31879f4a06
[go/program-gen] Add unit tests for argumentTypeName and go optional functions ()
# Description

This PR adds unit test for `argumentTypeName` and a few missing
uncovered cases from `GenFunctionExpression` in Go program-gen. I did
have to remove a check `Title(x) == "pulumi.Any"` that would never
evaluate to `true` but I didn't think this warrants a changelog for the
CLI

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-22 15:33:56 +00:00
Zaid Ajaj 1a8abbf8c8
[program-gen/go] Fix required config variables of type bool and number ()
# Description

While covering more parts of go codegen, I've seen that config variables
are broken 😓 specifically when requiring config variables using
`RequireFloat` it should be `RequireFloat64` and `RequireBoolean` should
be `RequireBool`.
Moreover, it seems that `RequireObject` doesn't work at all since the
function signature doesn't match the way it was generated (see )

C# has a similar issue with optional untyped objects as config
variables. For now have skipped compilation for those.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-20 13:16:37 +00:00
Zaid Ajaj 177e2e090d
[program-gen] Fix generated utility functions for filebase64, filebase64sha256, sha1 and mimeType ()
# Description

While writing program tests for generated helper utility functions
`filebase64`, `filebase64sha256`, `sha1` and `mimeType` with the idea to
increase code coverage, it turned out that those are completely broken
in all of the languages containing syntax errors, missing imports and
wrong indentation. This PR fixes them and extends the `functions`
program to show how they now look like and to show that they compile.
Also adding example usage of `stack()`, `project()` and `cwd()` in the
test program.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-15 11:26:00 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Zaid Ajaj ac6adb98bf
[go/program-gen] Fix using inline invoke expressions inside resources, objects and arrays ()
# Description

This PR fixes an issue in Go program where if users are writing invoke
expressions inline inside other expressions, then that invoke is
extracted into a temporary variable and then later referenced the same
way it was used. This is because non-output-versioned invokes cannot be
used directly since they return a tuple (InvokeResult, err) so we need
to check for the error before proceeding.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-03 14:22:43 +00:00
Thomas Gummerer 0d6fff3995
codegen/go: output correct array type ()
Currently for an array of objects, codegen for Go outputs a
`pulumi.AnyArray` type. However that does not exist. Make sure it always
outputs the correct `pulumi.Array` type.

While there give Maps the same treatment, as `pulumi.AnyMap` also
doesn't exist.

Fixes #https://github.com/pulumi/pulumi/issues/8072

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-10-20 16:12:25 +00:00
Zaid Ajaj ed02926277 Allow generating code for unknown invokes in non-strict mode 2023-07-10 15:05:18 +02:00
Fraser Waters 571fadae3f Use slice.Prealloc instead of make([]T, 0, ...)
Fixes https://github.com/pulumi/pulumi/issues/12738

https://github.com/pulumi/pulumi/pull/11834 turned on the prealloc
linter and changed a load of slice uses from just `var x T[]` to `x :=
make([]T, 0, preallocSize)`. This was good for performance but it turns
out there are a number of places in the codebase that treat a `nil`
slice as semnatically different to an empty slice.

Trying to test that, or even reason that through for every callsite is
untractable, so this PR replaces all expressions of the form `make([]T,
0, size)` with a call to `slice.Prealloc[T](size)`. When size is 0 that
returns a nil array, rather than an empty array.
2023-06-29 11:27:50 +01:00
Abhinav Gupta da184df24a
fix(codegen/go): Use raw string literals for multiline-strings
We already have logic in place to generate mutliline strings with
backticks if they meet certain conditions in genTemplateExpression.
However, the logic came into effect only if the string was complex
enough to warrant use of `Sprintf`
because the function short circuits to just printing a string literal
if the string doesn't need a `Sprintf`.

This changes genStringLiteral to be responsible for the decision
of whether the string warrants backticks or not,
retaining the prior conditions on whether the string would benefit
from being split across multiple lines.

Resolves 
2023-06-22 10:55:45 -07:00
Justin Van Patten 31d106980e [programgen/dotnet,go] Implement singleOrNone intrinsic
This change adds support for the `singleOrNone` intrinsic to Go and C# programgen.
2023-06-11 16:23:33 -07:00
Zaid Ajaj 2ce1fdb091 Fix panic in go program-gen when encountering splat expressions 2023-06-07 13:22:41 +02:00
Zaid Ajaj 703c53cfeb Turn error diagnostics into warnings 2023-06-02 17:17:59 +02:00
Zaid Ajaj af09d174cd Do not error out when generaing not yet implemented ForExpressions 2023-06-02 16:59:46 +02:00
Zaid Ajaj 6528a9fe7a Go module support as component resources in program-gen 2023-05-08 16:20:04 +02:00
Zaid Ajaj b90b5a5bf9 Add "NotImplemented" PCL function intrinsic 2023-03-10 12:14:28 +01:00
Abhinav Gupta acaf79bc10
all: Drop //nolint:goconst
Drops the nolint:goconst directive
now that the goconst linter is no longer enabled.
2023-03-09 11:15:21 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta cc32691bc1
chore(all): strings.Replace(..., -1) => strings.Replace(...)
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
2023-03-01 13:22:33 -08:00
Abhinav Gupta aefcd86af2
pkg/codegen/go: Prefer contract.Assertf over contract.Assert
Migrates uses of contract.{Assert, AssertNoError} in pkg/codegen/go
to use Assertf and AssertNoErrorf so that
we write more meaningful messages when these contracts are violated.

Incremental step towards deprecating the non-f variants.

Refs 
2023-02-13 16:27:39 -08:00
Fraser Waters 3936ed9b4f Add unsecret intrinsic function 2023-01-31 14:18:01 +00:00
Abhinav Gupta 42e7dc1fa3
pcl/codegen: Fix typeNameID data race
generator.argumentTypeName increments a typeNameID global variable.
This causes a data race when tests run in parallel:

```
WARNING: DATA RACE
Read at 0x000005d9dd28 by goroutine 20:
  github.com/pulumi/pulumi/pkg/v3/codegen/go.(*generator).argumentTypeName()
      /home/runner/work/pulumi/pulumi/pkg/codegen/go/gen_program_expressions.go:775 +0x84
  github.com/pulumi/pulumi/pkg/v3/codegen/go.(*generator).genObjectConsExpression()
  ...

Previous write at 0x000005d9dd28 by goroutine 19:
  github.com/pulumi/pulumi/pkg/v3/codegen/go.(*generator).argumentTypeName()
      /home/runner/work/pulumi/pulumi/pkg/codegen/go/gen_program_expressions.go:775 +0xa4
  github.com/pulumi/pulumi/pkg/v3/codegen/go.(*generator).genTemplateExpression()
      /home/runner/work/pulumi/pulumi/pkg/codegen/go/gen_program_expressions.go:672 +0xbe
```

This variable is not used anywhere else in the system.
Fix the data race by deleting the variable.
2023-01-19 11:07:26 -08:00
bors[bot] 8915911825
Merge
11819: Cleanup diag error for protected resources r=dixler a=iwahbe

Change the error message for deleting protected resources so it sounds less like we tried and failed to delete the resource, and more like we chose not to. 

Fixes 


11834: Preallocate slices with a known capacity. r=dixler a=RobbieMcKinstry

Enable the prealloc linter, which identifies slices with a known capacity, but are not preallocated, which results in unnecessary allocations and memcpys.

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] N/A: enabling a linter. I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] **N/A: not a user-facing change.** I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


11842: Changelog and go.mod updates for v3.51.1 r=dixler a=justinvp



11845: Move SecretsProvider to pkg/secrets r=dixler a=Frassle

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Trying to push DefaultSecretsProvider up past the backends resulted in a module loop between pkg/resource/stack and pkg/resource/deploy. This places SecretProvider in the secrets module to avoid that.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works - N/A Just moving interface declaration 
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change - N/A Internal code changes
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version - No
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Robbie McKinstry <robbie@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: Robbie McKinstry <thesnowmancometh@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Fraser Waters <fraser@pulumi.com>
2023-01-12 20:11:42 +00:00
Abhinav Gupta 3d64521cf2
unused: Remove unused functions and types
Several functions and types were reported as unused.

Most notably, jsonTemp and the method in jsonSpiller that uses it
were all unused.
2023-01-12 09:55:34 -08:00
Robbie McKinstry 4959522a53
Repair tests expecting nil slices 2023-01-11 21:53:04 -08:00
Zaid Ajaj 330676a0d0 Initial implementation of simplified invokes for dotnet and nodejs 2023-01-11 14:17:14 -08:00
Robbie McKinstry 1f78baae71
Preallocate slices with a known capacity.
Enable the prealloc linter, which identifies slices
with a known capacity, but are not preallocated, which
results in unnecessary allocations and memcpys.
2023-01-11 12:52:51 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Abhinav Gupta 0bff0b8716 sdk/go: Remove 'nolint' directives from package docs
Go treats comments that match the following regex as directives.

    //[a-z0-9]+:[a-z0-9]

Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)

Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.

    // nolint:foo
    //nolint: foo

This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.

Verification:
Here's the output of `go doc` on some entities
before and after this change.

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

nolint: lll, interfacer

nolint: lll, interfacer

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```

After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine. nolint: revive
```

After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine.
```

Note that golangci-lint offers a 'nolintlint'  linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.

Resolves 

Related: https://github.com/golangci/golangci-lint/issues/892

[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
  tee "$FILES" |
  xargs perl -p -i -e '
    s|// nolint|//nolint|g;
    s|//nolint: |//nolint:|g;
  '
rg '.go$' < "$FILES" | xargs gofmt -w -s
2023-01-06 09:06:47 -08:00
Aaron Friel a682a1ab06
Revert "Simplified invokes: SDK-gen and program-gen implementation for dotnet and nodejs" 2022-12-20 17:47:29 -08:00
bors[bot] 43f7e4d6b9
Merge
11670: Fix incorrect pointers for array of objects [go/codegen] r=aq17 a=aq17

Fixes 
Fix incorrect pointer argument codegen when objects are nested inside an array whose type is not a pointer.

Example:
`&s3.BucketLoggingArgs` is valid here
```
bucket, err := s3.NewBucket(ctx, "bucket", &s3.BucketArgs{
			Loggings: s3.BucketLoggingArray{
				&s3.BucketLoggingArgs{
					TargetBucket: logs.Bucket,
				},
			},
		})

```
but `&ec2.SubnetSpecArgs{}` is invalid here:
```
SubnetSpecs: []ec2.SubnetSpecArgs{
				&ec2.SubnetSpecArgs{
					Type:     ec2.SubnetTypePublic,
					CidrMask: 22,
				},
				&ec2.SubnetSpecArgs{
					Type:     ec2.SubnetTypePrivate,
					CidrMask: 20,
				},
			},
```
Instead, it should be: 
```
SubnetSpecs: []ec2.SubnetSpecArgs{
				{
					Type:     ec2.SubnetTypePublic,
					CidrMask: 22,
				},
				{
					Type:     ec2.SubnetTypePrivate,
					CidrMask: 20,
				},
			},
```



Co-authored-by: aq17 <aqiu@pulumi.com>
2022-12-19 20:47:21 +00:00
aq17 825de881b8 Fix incorrect pointers for array of objects [go/codegen] 2022-12-19 12:45:27 -08:00
Zaid Ajaj a6f26d2355 Initial implementation of simplified invokes for dotnet and nodejs 2022-12-16 13:39:03 +01:00
aq17 d780e3a074 Check for optional/ Ptr types within Unions [go/programgen] 2022-12-12 15:49:20 -08:00
Ian Wahbe 7ae9c181d0 Don't use *schema.Package in go codegen 2022-12-08 17:51:50 +01:00