Commit Graph

53 Commits

Author SHA1 Message Date
Zaid Ajaj 8912eae4c6 Fixes python panic when emiting code for index expressions that aren't typechecked 2023-06-09 01:44:09 +02:00
Zaid Ajaj 65ad123780 Implement singleOrNone intrinsic for typescript and python 2023-05-25 22:12:13 +02:00
Zaid Ajaj 588e21fde7 Impleneted python program-gen for PCL components 2023-04-05 13:19:52 +02:00
Abhinav Gupta acda1e3b13
all: Fix revive issues
Fixes the following issues found by revive
included in the latest release of golangci-lint.

Full list of issues:

**pkg**

```
backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive)
backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive)
codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive)
codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive)
resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive)
resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
```

**sdk**

```
go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)
go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive)
go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive)
nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive)
```

**tests**

```
integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
2023-03-21 08:55:11 -07:00
Zaid Ajaj b90b5a5bf9 Add "NotImplemented" PCL function intrinsic 2023-03-10 12:14:28 +01:00
Abhinav Gupta acaf79bc10
all: Drop //nolint:goconst
Drops the nolint:goconst directive
now that the goconst linter is no longer enabled.
2023-03-09 11:15:21 -08:00
bors[bot] 4f3a1bce17
Merge
12267: chore(all): strings.Replace(..., -1) => strings.Replace(...) r=abhinav a=abhinav

Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.


12268: chore: WriteString(Sprintf(..)) => Fprintf(..) r=abhinav a=abhinav

Replace `buffer.WriteString(fmt.Sprintf(..))` calls,
where buffer is one of `bytes.Buffer`, `strings.Builder`, or `bufio.Writer`,
with equivalent `fmt.Fprintf` calls -- all those types are io.Writers.


12337: Freeze v3.56.0 r=dixler a=dixler



Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: Kyle Dixler <kyle@pulumi.com>
2023-03-02 21:50:14 +00:00
Abhinav Gupta cc32691bc1
chore(all): strings.Replace(..., -1) => strings.Replace(...)
Replaces all instances of `strings.Replace(s, old, new, -1)`
with the equivalent `strings.ReplaceAll(s, old, new)`.
This function has been available since Go 1.12.
2023-03-01 13:22:33 -08:00
Abhinav Gupta 2f81a47a0d
pkg/codegen/python: Prefer contract.Assertf over Assert
Incremental step towards 

Migrates uses of contract.{Assert, AssertNoError, Require}
to `*f` variants so that we're required to provide more error context.

Refs 
2023-03-01 13:22:32 -08:00
Fraser Waters 3936ed9b4f Add unsecret intrinsic function 2023-01-31 14:18:01 +00:00
Zaid Ajaj 330676a0d0 Initial implementation of simplified invokes for dotnet and nodejs 2023-01-11 14:17:14 -08:00
Abhinav Gupta 0bff0b8716 sdk/go: Remove 'nolint' directives from package docs
Go treats comments that match the following regex as directives.

    //[a-z0-9]+:[a-z0-9]

Comments that are directives don't show in an entity's documentation.
5a550b6951 (diff-f56160fd9fcea272966a8a1d692ad9f49206fdd8dbcbfe384865a98cd9bc2749R165)

Our code has `//nolint` directives that now show in the API Reference.
This is because these directives are in one of the following forms,
which don't get this special treatment.

    // nolint:foo
    //nolint: foo

This change fixes all such directives found by the regex:
`// nolint|//nolint: `.
See bottom of commit for command used for the fix.

Verification:
Here's the output of `go doc` on some entities
before and after this change.

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

nolint: lll, interfacer

nolint: lll, interfacer

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
```

After
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi | head -n8
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

const EnvOrganization = "PULUMI_ORGANIZATION" ...
var ErrPlugins = errors.New("pulumi: plugins requested")
func BoolRef(v bool) *bool
func Float64Ref(v float64) *float64
func IntRef(v int) *int
func IsSecret(o Output) bool
```

Before
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine. nolint: revive
```

After:
```
% go doc github.com/pulumi/pulumi/sdk/v3/go/pulumi URN_
package pulumi // import "github.com/pulumi/pulumi/sdk/v3/go/pulumi"

func URN_(o string) ResourceOption
    URN_ is an optional URN of a previously-registered resource of this type to
    read from the engine.
```

Note that golangci-lint offers a 'nolintlint'  linter
that finds such miuses of nolint,
but it also finds other issues so I've deferred that to a follow up PR.

Resolves 

Related: https://github.com/golangci/golangci-lint/issues/892

[git-generate]
FILES=$(mktemp)
rg -l '// nolint|//nolint: ' |
  tee "$FILES" |
  xargs perl -p -i -e '
    s|// nolint|//nolint|g;
    s|//nolint: |//nolint:|g;
  '
rg '.go$' < "$FILES" | xargs gofmt -w -s
2023-01-06 09:06:47 -08:00
Aaron Friel a682a1ab06
Revert "Simplified invokes: SDK-gen and program-gen implementation for dotnet and nodejs" 2022-12-20 17:47:29 -08:00
Zaid Ajaj a6f26d2355 Initial implementation of simplified invokes for dotnet and nodejs 2022-12-16 13:39:03 +01:00
Aaron Friel 4840af7c0c Fix Python codegen for secret() intrinsic 2022-12-11 05:07:18 -08:00
Ian Wahbe 65b6b01abd Don't use *schema.Package in python codegen 2022-12-07 17:54:38 +01:00
aq17 927e9d4a69 Improve error msg for invalid enum values on convert 2022-12-01 15:29:22 -08:00
Zaid Ajaj a63d8b61db refactor: annotateObjectProperties no longer skips top-level object and annotates unions 2022-10-08 14:46:25 +02:00
Zaid Ajaj d583af6008 recursively annotate expressions under invoke calls with their associated schema types 2022-10-07 14:02:12 +02:00
aq17 575c2549aa Fix codegen for __apply functions 2022-09-28 10:32:33 -07:00
Aaron Friel a4b1d6b2a7 ci: gofmt 1.18+ clean 2022-09-21 09:48:39 -07:00
Ian Wahbe a4803f43ea
Remove programgen conversion assertions () 2022-07-13 14:29:34 -07:00
Zaid Ajaj 515cf6191c
Fixes panic when generating go from pulumi yaml ()
* Fixes  panic when generating go from pulumi yaml

Co-authored-by: Aaron Friel <friel@pulumi.com>
Co-authored-by: Zaid Ajaj <zaid@pulumi.com>

* rewrite rewriteConversions to accumulate diagnostics, improve errors

* rewrite diags

* update java dependency

* chore: fix go.mod/go.sum

* fix: traversal of multi-part template string expressions

Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
Co-authored-by: Aaron Friel <friel@pulumi.com>
Co-authored-by: Zaid Ajaj <zaid@pulumi.com>
2022-07-07 13:15:47 -07:00
Aaron Friel be2501e245
codegen for all assets and archives types ()
* codegen for all assets and archives types

* chore: add changelog entry

* Update CHANGELOG_PENDING.md

Co-authored-by: Fraser Waters <fraser@pulumi.com>

* fix C# codegen with explicit AssetOrArchive type

Co-authored-by: Fraser Waters <fraser@pulumi.com>
2022-04-25 12:59:30 -07:00
Ian Wahbe 183fa89163
Add an EnumType to the PCL model ()
* Add an EnumType to the PCL model

Languages implementation:
- [X] C#
- [X] Python
- [X] TypeScript
- [X] Go

* Fix assert

* Take namespace settings into account

* Implement enums for Go

* Update tests and add documentation.

* Add a test + handle output&unsafe

* Get outputs compiling for Go

* Line up types for Go

* Add nodejs

* Add Python

* Change expression form

* Fix nil check (`==` -> `!=`)

* Standardize nodejs & python checks

This should fix the Node OOM error.

* Rename enum-py to typed-enum-pp

* Don't reference schema types in `hcl2/model`

* Fix nits
2022-04-18 11:03:42 +02:00
Ian Wahbe 66f8b357e2
Expand pcl intrinsic functions ()
* Add TS support

* Add python support

* Add go support

* Add C# support
2022-03-15 17:05:36 -07:00
T-Vova 8f3e7f40ce
Implemented filebase64sha256 & fileArchive support for codegen ()
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2022-01-21 16:03:25 +02:00
Anton Tayanovskyy 1cedb29193
Programgen support for fn_output forms in Python () ()
* Update Python programgen to use fn_output forms

* Fix skips

* Fix CHANGELOG

* Accept changes, backing out Fargate change
2021-11-18 17:43:13 -05:00
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. ()
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00
T-Vova e696fb6c50
Implemented filebase64() support for Code Generator ()
* Added filebase64 support for Golang

* Fixed function signature

* Added filebase64 support for Typescript

* Added filebase64 support for Python

* Added filebase64 support for Dotnet, fixed Sha1

* Fixed helper method list

Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 23:09:28 +01:00
T-Vova 87e4c92c66
Implemented `sha1()` support for Code Generator ()
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 21:40:38 +03:00
Pat Gavlin 1409d88438
[codegen/*] Add join and toBase64 functions. ()
Co-authored-by: Komal Ali <komal@pulumi.com>
2021-07-28 20:41:23 -07:00
Justin Van Patten 42a1896bf9
[codegen/python] Delete dead code around casing tables ()
Coincident with the release of Pulumi 3.0, we updated the provider SDK codegen for Python to no longer use casing tables for translating Python snake_case names to Pulumi camelCase names (and vice versa). Instead, the mapping is encoded in decorators applied on class properties.

Some of the code that was used to generate and use the casing tables has persisted. This commits removes this code, as it's no longer necessary, and will improve the quality of our generated examples.
2021-07-27 08:37:49 -07:00
Pat Gavlin 7b1d6ec1ac
Reify `Input` and `Optional` types in the schema type system. ()
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Komal ff219a2381
Use input/output classes in program gen () 2020-08-06 15:09:23 -07:00
Levi Blackstone ba18683d9a
[codegen] Generate null values for model.NoneType () 2020-08-06 14:12:27 -06:00
Pat Gavlin a8a20ecb4b
[codegen/*] Add support for explicit secrets. ()
- Add a new builtin function, secret, that marks its input value as
  secret
- Add support for this function to the various code generators

Fixes .
2020-06-30 11:35:24 -07:00
Pat Gavlin 69ba47cff2
[codegen/*] Add support for resource options. ()
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.

One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.

These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.

Fixes .
2020-06-29 16:33:52 -07:00
Mikhail Shilkov f66100ce28
Initial support for C# program gen ()
Initial support for C# program gen
2020-05-19 10:18:38 +02:00
Pat Gavlin 255620ea1e
[codegen/nodejs] Await invokes in async contexts. ()
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Pat Gavlin ca5cb05482 [hcl2/model] String literal + traversal fixes.
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-04-30 16:38:36 -07:00
Pat Gavlin 1d6cce98fe
[codegen/python] Fix nested quotes. ()
Unlike most languages with interpolated strings, Python's formatted
string literals do not allow the nesting of quotes. For example,
this expression is not legal Python:

    f"Foo {"bar"} baz"

If an interpolation requires quotes, those quotes nust differ from the
quotes used by the enclosing literal. We can fix the previous example
by rewriting it with single quotes:

    f"Foo {'bar'} baz"

However, this presents a problem if there are more than two levels of
nesting, as Python only has two kinds of quotes (four if the outermost
string uses """ or '''): in this case, the expression becomes
unspellable, and must be assigned to a local that is then used in place
of the original expression. So this:

    f"Foo {bar[f'index {baz["qux"]}']} zed"

becomes this:

    index = "qux"
    f"Foo {bar[f'index {baz[index]}']}"

To put it bluntly, Python code generation reqiures register allocation,
but for quotes. These changes implement exactly that.

These changes also include a fix for traversals that access values that
are dictionaries rather than objects, and must use indexers rather than
attributes.
2020-04-30 16:34:25 -07:00
Pat Gavlin 5271eda247
[codegen/python] Fix case mapping. ()
Mapping happens at package scope, not type scope. This is what causes
some of the surprising mixes of camel and snake casing in our packages.

In order to accommodate this, build the case mapping tables up front,
and refer to them when performing case mapping.
2020-04-30 14:16:56 -07:00
Pat Gavlin 2e9499a000
[codegen/hcl2] Fix the apply rewriter. ()
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
2020-04-24 22:04:24 -07:00
Pat Gavlin af344d9b72
Fix Python module names. ()
In particular, ensure that they are keyword-safe. This affects
`aws:lambda:*` in particular: prior to these changes, we were generating
code into `pulumi_aws.lamdba`, which is not referencable due to its use
of the `lamdba` keyword. With these changes, we generate code into
`pulumi_aws.lambda_`.

There is also a small fix for multi-line non-formatted strings: these
strings do not need escaped braces.
2020-04-24 08:52:36 -07:00
Pat Gavlin 9204d70c54 Fix Python input property names.
- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
2020-04-21 13:55:33 -07:00
Pat Gavlin 1d672563da Fix output property names in Python codegen.
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
2020-04-21 10:25:27 -07:00
Pat Gavlin 807cf44cd7
Codegen: collect imports from function calls. ()
Codegen: collect imports from function calls; fix Python resource names.

Just what it says on the tin.
2020-04-21 10:24:42 -07:00
Pat Gavlin 398878de31
Various codegen updates. ()
- Define `null` in Pulumi HCL2
- Bind Pulumi HCL2 in topological order s.t. variable types can be
  properly computed
- Fix resources that range over bools and numbers
- Add element, length, lookup, readFile, and split functions
- Do not rewrite function signatures with input types during binding
- Fix splat expression binding for non-lists
- Add support for evaluating expressions
- Add support for operator precedence to code generators
- Add support for constants to the HCL2 IR
- Add support for generating ranged resources in Python
- Add support for generating conditional resource in Node and Python
- Fix various naming issues in Python
2020-04-16 16:44:34 -07:00