Commit Graph

102 Commits

Author SHA1 Message Date
Zaid Ajaj 1a8abbf8c8
[program-gen/go] Fix required config variables of type bool and number ()
# Description

While covering more parts of go codegen, I've seen that config variables
are broken 😓 specifically when requiring config variables using
`RequireFloat` it should be `RequireFloat64` and `RequireBoolean` should
be `RequireBool`.
Moreover, it seems that `RequireObject` doesn't work at all since the
function signature doesn't match the way it was generated (see )

C# has a similar issue with optional untyped objects as config
variables. For now have skipped compilation for those.

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [ ] I have run `make lint` to verify my code passes the lint check
  - [ ] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-20 13:16:37 +00:00
Fraser Waters 16d9f4c167
Enable perfsprint linter ()
<!--- 
Thanks so much for your contribution! If this is your first time
contributing, please ensure that you have read the
[CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md)
documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context. -->

Prompted by a comment in another review:
https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945

This lints that we don't use `fmt.Errorf` when `errors.New` will
suffice, it also covers a load of other cases where `Sprintf` is
sub-optimal.

Most of these edits were made by running `perfsprint --fix`.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-12-12 12:19:42 +00:00
Fraser Waters c56bb05174
Update golangci-lint () 2023-11-21 15:16:13 +00:00
Zaid Ajaj ac6adb98bf
[go/program-gen] Fix using inline invoke expressions inside resources, objects and arrays ()
# Description

This PR fixes an issue in Go program where if users are writing invoke
expressions inline inside other expressions, then that invoke is
extracted into a temporary variable and then later referenced the same
way it was used. This is because non-output-versioned invokes cannot be
used directly since they return a tuple (InvokeResult, err) so we need
to check for the error before proceeding.

Fixes 

## Checklist

- [ ] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [x] I have added tests that prove my fix is effective or that my
feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2023-11-03 14:22:43 +00:00
Luke Hoban 46e2753d02
Require Go 1.20 for Go Programgen ()
We added program generation that assumes availability of generics as
part of https://github.com/pulumi/pulumi/pull/13149, but didn't bump the
version of Go required in the generated go.mod. Currently, Pulumi is
supported on the supported versions of Go, which includes Go 1.20 and
1.21, so here we bump the Go version to the minimum currently supported
version, which also supports all of the code we generate today.

I was curious how this was not already failing tests, and it appears
it's because we delete the `go.mod` file in our tests and recreate it
with `go mod init`. I am unclear why we do that, but it feels like we
should not, exactly to avoid this sort of problem in the future and to
test the actual code we generate.


8696695cdb/pkg/codegen/go/test.go (L25-L28)

Fixes .
2023-08-16 04:59:07 +00:00
Fraser Waters 99b736b55b Add localDependencies option to GenerateProject
This isn't currently actually used anywhere. I've just threaded it
through to all the program gen functions where it will be needed.

Matrix testing will be using and testing this.
2023-08-08 12:28:19 +01:00
Zaid Ajaj 3e39cb2820 Normalize the declaration name of generated resource components 2023-07-27 15:15:27 +02:00
Abhinav Gupta a24ad1839b
programgen(go): Handle conflicting names in imported packages
This fixes how programgen generates import statements
to handle conflicting imports when two imported packages
have the same name, e.g.

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/ecs
    github.com/pulumi/pulumi-awsx/sdk/go/awsx/ecs

To do this, we add a fileImporter type that tracks these imports.
It prefers unnamed imports for packages unless one of the following is
true:

- the name of the package has already been used by another import
- the name of the package does not match the last component
  of the import path (e.g., `example.com/foo-go` with `package foo`).

If the name has already been used by another import,
it attempts the following in-order:

- Combine the last two path components of the import path
  into an identifier and use that if available.
  e.g., `awsxs3` from `sdk/go/awsx/s3`.
- Append a number to the package name and increment it
  until an unused name is found.
  e.g. `ecs2`, `ecs3`, and so on.

There's a change in how this information is tracked as well.
Previously, this was a pull approach: various calls returned
programImports objects which all got merged together.

This change switches to a push approach:
as code is generated and imports are requested,
they're submitted to the fileImporter which keeps track of them
until the next `Reset()` call.
The above also has a nice side effect of dropping a parameter.

Another change worth explicitly calling out:
Previously, getModOrAlias partially duplicated some of the logic
implemented in getPulumiImport, and used `mod`, `originalMod`
in a non-obvious way.
This generated incorrect imports like the following
(note the two `/aws` at the end):

    github.com/pulumi/pulumi-aws/sdk/v5/go/aws/aws

This change replicates more of the logic of getPulumiImport
(now called addPulumiImport) into this function,
and addresses the discrepancy in codegen caused by `mod`/`originalMod`.
The result leaves most existing code unchanged,
except in a couple existing cases where the resulting changes make sense
given the logic for named imports outlined above.

Resolves 
2023-07-25 12:49:37 -07:00
Zaid Ajaj cd65005389 Fix panic in GenerateProject when version is not set in schema 2023-07-14 01:23:27 +02:00
Zaid Ajaj ed02926277 Allow generating code for unknown invokes in non-strict mode 2023-07-10 15:05:18 +02:00
Zaid Ajaj 3bc0e6fe38 Fix aliasing package names using dashes when schema doesn't include go package info override 2023-06-20 18:11:47 +02:00
Zaid Ajaj 7e5e452909 Extend SkipResourceTypechecking to allow generating unknown resources 2023-06-14 19:02:56 +02:00
Zaid Ajaj 6c4952d4fd Add a warning diagnostic when formatting code fails 2023-05-30 19:32:57 +02:00
Zaid Ajaj da3d7c9844 Do not error when generated Go code cannot be formatted 2023-05-30 14:45:38 +02:00
Zaid Ajaj f09dcaa373 Fix main program imports 2023-05-11 19:55:17 +02:00
Zaid Ajaj 6528a9fe7a Go module support as component resources in program-gen 2023-05-08 16:20:04 +02:00
Zaid Ajaj 0988177853 Implement description as comments or docstring for config variables in program-gen 2023-03-21 15:01:16 +01:00
Zaid Ajaj b90b5a5bf9 Add "NotImplemented" PCL function intrinsic 2023-03-10 12:14:28 +01:00
Abhinav Gupta 407e6953e3
codegen/{go,js,.net,py}: Generate retainOnDelete
Uses the newly added `options.retainOnDelete` in PCL
to generate the RetainOnDelete resource option
for Go, NodeJS, .NET, and Python.

Java support cannot be added without a release of pulumi/pulumi
because the new field in pcl.ResourceOptions is not yet visible
to Java.

Resolves 
2023-03-06 13:57:23 -08:00
Abhinav Gupta 7aa5b77a0c
all: Reformat with gofumpt
Per team discussion, switching to gofumpt.

[gofumpt][1] is an alternative, stricter alternative to gofmt.
It addresses other stylistic concerns that gofmt doesn't yet cover.

  [1]: https://github.com/mvdan/gofumpt

See the full list of [Added rules][2], but it includes:

- Dropping empty lines around function bodies
- Dropping unnecessary variable grouping when there's only one variable
- Ensuring an empty line between multi-line functions
- simplification (`-s` in gofmt) is always enabled
- Ensuring multi-line function signatures end with
  `) {` on a separate line.

  [2]: https://github.com/mvdan/gofumpt#Added-rules

gofumpt is stricter, but there's no lock-in.
All gofumpt output is valid gofmt output,
so if we decide we don't like it, it's easy to switch back
without any code changes.

gofumpt support is built into the tooling we use for development
so this won't change development workflows.

- golangci-lint includes a gofumpt check (enabled in this PR)
- gopls, the LSP for Go, includes a gofumpt option
  (see [installation instrutions][3])

  [3]: https://github.com/mvdan/gofumpt#installation

This change was generated by running:

```bash
gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error)
```

The following files were manually tweaked afterwards:

- pkg/cmd/pulumi/stack_change_secrets_provider.go:
  one of the lines overflowed and had comments in an inconvenient place
- pkg/cmd/pulumi/destroy.go:
  `var x T = y` where `T` wasn't necessary
- pkg/cmd/pulumi/policy_new.go:
  long line because of error message
- pkg/backend/snapshot_test.go:
  long line trying to assign three variables in the same assignment

I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 09:00:24 -08:00
Abhinav Gupta f3e39d2f2f
chore: WriteString(Sprintf(..)) => Fprintf(..)
Replace `buffer.WriteString(fmt.Sprintf(..))` calls,
where buffer is one of `bytes.Buffer`, `strings.Builder`, or `bufio.Writer`,
with equivalent `fmt.Fprintf` calls -- all those types are io.Writers.
2023-03-01 13:22:33 -08:00
Abhinav Gupta aefcd86af2
pkg/codegen/go: Prefer contract.Assertf over contract.Assert
Migrates uses of contract.{Assert, AssertNoError} in pkg/codegen/go
to use Assertf and AssertNoErrorf so that
we write more meaningful messages when these contracts are violated.

Incremental step towards deprecating the non-f variants.

Refs 
2023-02-13 16:27:39 -08:00
Fraser Waters 3936ed9b4f Add unsecret intrinsic function 2023-01-31 14:18:01 +00:00
Abhinav Gupta 3d64521cf2
unused: Remove unused functions and types
Several functions and types were reported as unused.

Most notably, jsonTemp and the method in jsonSpiller that uses it
were all unused.
2023-01-12 09:55:34 -08:00
Abhinav Gupta 1158d4acee
all: Drop ioutil
Stop using io/ioutil across the entire repository.
The io/ioutil package was deprecated in Go 1.16 (2021-02)
with replacements provided in other packages.
Specifically:

    ioutil.Discard   => io.Discard
    ioutil.NopCloser => io.NopCloser
    ioutil.ReadAll   => io.ReadAll
    ioutil.ReadFile  => os.ReadFile
    ioutil.TempDir   => os.MkdirTemp
    ioutil.TempFile  => os.CreateTemp
    ioutil.WriteFile => os.WriteFile

This change switches all of these entities
across the repository.

Following this change,
the only references to ioutil are in schema files:

    % rg -l ioutil
    pkg/codegen/testing/test/testdata/aws-4.26.0.json
    pkg/codegen/testing/test/testdata/aws-4.36.0.json
    pkg/codegen/testing/test/testdata/aws-4.37.1.json
    pkg/codegen/testing/test/testdata/aws-5.4.0.json
    pkg/codegen/testing/test/testdata/aws-5.16.2.json

The bulk of this change was generated automatically
with manual touch ups afterwards.
2023-01-06 16:35:14 -08:00
Abhinav Gupta 0bff0b8716 sdk/go: Remove 'nolint' directives from package docs
Go treats comments that match the following regex as directives.

    //[a-z0-9]+:[a-z0-9]

Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)

Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.

    // nolint:foo
    //nolint: foo

This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.

Verification:
Here's the output of `go doc` on some entities
before and after this change.

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

nolint: lll, interfacer

nolint: lll, interfacer

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```

After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine. nolint: revive
```

After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine.
```

Note that golangci-lint offers a 'nolintlint'  linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.

Resolves 

Related: https://github.com/golangci/golangci-lint/issues/892

[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
  tee "$FILES" |
  xargs perl -p -i -e '
    s|// nolint|//nolint|g;
    s|//nolint: |//nolint:|g;
  '
rg '.go$' < "$FILES" | xargs gofmt -w -s
2023-01-06 09:06:47 -08:00
aq17 825de881b8 Fix incorrect pointers for array of objects [go/codegen] 2022-12-19 12:45:27 -08:00
aq17 af8e87b946 Add test for error initialization [go/codegen] 2022-12-14 12:41:11 -08:00
aq17 e3a60af8cb Fix undeclared err codegen bug 2022-12-13 11:54:39 -08:00
Ian Wahbe 7ae9c181d0 Don't use *schema.Package in go codegen 2022-12-08 17:51:50 +01:00
Ian Wahbe 095f5a9fbf Convert invoke results to ouputs when needed 2022-11-28 16:14:07 -08:00
bors[bot] d6b79be66e
Merge
11442: Don't import types referenced from local variables r=iwahbe a=iwahbe

Don't import types if the type is only referenced in a scoped traversal (`foo.bar`), not named directly.

Part of https://github.com/pulumi/pulumi/issues/11427
Relates to https://github.com/pulumi/pulumi/issues/8324


Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-11-23 23:53:49 +00:00
Ian Wahbe 4cc4108da0 Cleanup import generation 2022-11-23 13:22:18 -08:00
Ian Wahbe 2463846339 Don't import types referenced from local variables 2022-11-22 16:35:00 -08:00
Ian Wahbe a601fdb080 Support a logical name for config vars 2022-11-02 12:39:57 -07:00
Ian Wahbe b2ce84c195 Add `modulePath` to go 2022-10-10 16:01:53 -07:00
bors[bot] adef574983
Merge
10978: [go/program-gen] Missing default case handling when generating local variables r=Zaid-Ajaj a=Zaid-Ajaj

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes 

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Zaid Ajaj <zaid.naom@gmail.com>
2022-10-10 12:17:57 +00:00
Zaid Ajaj 4c7580471c Fixes int constant range expressions for go 2022-10-09 18:28:49 +02:00
Zaid Ajaj 03a7ee138c Missing default case handling for local variables 2022-10-09 16:48:16 +02:00
aq17 575c2549aa Fix codegen for __apply functions 2022-09-28 10:32:33 -07:00
Eng Zer Jun 57b25b45d1
refactor: use `os.ReadDir` for lightweight directory reading
`os.ReadDir` was added in Go 1.16 as part of the deprecation of `ioutil`
package. It is a more efficient implementation than `ioutil.ReadDir`.

Reference: https://pkg.go.dev/io/ioutil#ReadDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-09-24 15:20:43 +08:00
aq17 39dc78d75c changelog 2022-09-16 09:43:12 -07:00
aq17 63c8a30e49 Fix incorrect import for non-pulumi owned package on convert 2022-09-16 09:43:12 -07:00
Alex Qiu 2a81117dd6
Fix invalid resource type on `pulumi convert` to Go () 2022-09-09 16:11:38 -07:00
Alex Qiu 15d86e53d8
Fix import path for non-pulumi owner providers () 2022-08-29 17:16:22 -07:00
Alex Qiu b1557bc5ec
Fix StackReference code gen () 2022-08-19 10:27:05 -07:00
Aaron Friel df8913f4ff
[codegen/go] Use correct method name for loading float config values () 2022-08-18 20:26:07 -07:00
Ian Wahbe 94422f97f8
Expose external package cache + global default (Take 2) ()
* Expose external package cache

* CL

* Add a default global cache

* More cache passing

* Compare `Cache` entries by pointer equality

* Add a check for providers when doing imports

* Pass the cache

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-08-12 20:04:21 +02:00
Aaron Friel cca0a39e70
Revert "Expose external package cache ()" ()
This reverts commit 16c173963a.
2022-08-11 15:04:14 -07:00
Ian Wahbe 16c173963a
Expose external package cache ()
* Expose external package cache

* CL

* Add a default global cache

* More cache passing

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
2022-08-11 13:28:27 -07:00