pulumi/pkg/codegen
Abhinav Gupta 9b43e56add
pkg/hcl2/model: Make String methods thread-safe
ObjectType.String and a few other types
do a fair bit of computation to build a string representation.
They memoize the result in an internal `s` field.
This causes a data race if two goroutines call `String()`
on the same object at the same time.

Tests in pkg/codegen/pcl ran into the following
from ObjectType.String.

```
WARNING: DATA RACE
Write at 0x00c00055c030 by goroutine 36:
  github.com/pulumi/pulumi/pkg/v3/codegen/hcl2/model.(*ObjectType).string()
      [..]/pulumi/pkg/codegen/hcl2/model/type_object.go:300 +0x418
  [.. snip ..]
  github.com/pulumi/pulumi/pkg/v3/codegen/pcl.TestApplyRewriter.func1()
      [..]/pulumi/pkg/codegen/pcl/rewrite_apply_test.go:176 +0x68
  testing.tRunner()
      /opt/homebrew/Cellar/go/1.19.5/libexec/src/testing/testing.go:1446 +0x188
  testing.(*T).Run.func1()
      /opt/homebrew/Cellar/go/1.19.5/libexec/src/testing/testing.go:1493 +0x40

Previous read at 0x00c00055c030 by goroutine 37:
  github.com/pulumi/pulumi/pkg/v3/codegen/hcl2/model.(*ObjectType).string()
      [..]/pulumi/pkg/codegen/hcl2/model/type_object.go:276 +0x40
  github.com/pulumi/pulumi/pkg/v3/codegen/hcl2/model.(*OutputType).string()
      [..]/pulumi/pkg/codegen/hcl2/model/type_output.go:110 +0x4c
  [..]
```

To fix this, turn the memoized field into an `atomic.Value`
in all types that memoize it.

When two goroutines race to calculate this value,
one of them will do extra work that will be thrown away,
but it won't cause a data race.

Refs #10092
2023-01-18 16:18:17 -08:00
..
cgstrings Revert "Revert "Go and Python codegen support symbols with hyphens in their names"" 2022-10-17 09:37:07 -07:00
convert Make the PluginMapper lazy 2022-12-14 15:26:33 +00:00
docs Revert "Update package details anchor tags" 2023-01-13 11:16:29 -08:00
dotnet gosimple: printf, time.Since, TrimPrefix, etc. 2023-01-12 09:55:34 -08:00
gen_program_test chore: Update makefile, ensure make test_codegen_go 2022-10-17 00:22:48 -07:00
go golangci-lint: Enable staticcheck 2023-01-14 16:59:46 -08:00
hcl2 pkg/hcl2/model: Make String methods thread-safe 2023-01-18 16:18:17 -08:00
importer all: Drop ioutil 2023-01-06 16:35:14 -08:00
nodejs pkg: Drop unnecessary printfs 2023-01-13 12:41:56 -08:00
pcl golangci-lint: Enable staticcheck 2023-01-14 16:59:46 -08:00
python pkg: Drop unnecessary printfs 2023-01-13 12:41:56 -08:00
report all: Drop ioutil 2023-01-06 16:35:14 -08:00
schema golangci-lint: Enable staticcheck 2023-01-14 16:59:46 -08:00
testing Merge #11857 #11858 #11859 2023-01-13 22:33:15 +00:00
README.md Update README.md (#9575) 2022-05-10 13:51:02 +01:00
docs.go Support "lifting" single-valued method returns to their return type (#8111) 2021-10-01 11:33:02 -07:00
docs_test.go ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
utilities.go Don't use *schema.Package in python codegen 2022-12-07 17:54:38 +01:00
utilities_test.go ci: radical idea - what if slow tests & no stdout makes GH consider runner dead? 2022-03-06 14:52:13 -08:00
utilities_types.go Initial implementation of simplified invokes for dotnet and nodejs 2023-01-11 14:17:14 -08:00

README.md

Pulumi CrossCode

The github.com/pulumi/pulumi/pkg/v3/codegen package defines the core components of Pulumi's CrossCode technology. CrossCode provides a set of foundational capabilities for working across a variety of programming languages supported by the Pulumi platform.

The core components of CrossCode in this package are:

  • Schema: The definition of Pulumi Schema, a language-neutral specification of cloud resource models. Pulumi Schema is the interface definition language for all Pulumi Packages, and is used as input to SDK code generation for each supported Pulumi language.
  • SDK Code Generation for Node.js, Python, Go and .NET: These libraries define how to create Pulumi SDKs from a Pulumi Schema definition of a package. The resulting SDKs expose the resource, components and functions from that package into the the Pulumi programming model defined for the given language.
  • Docs Generation: In addition to generating per-language SDKs, CrossCode supports generating language-neutral documentation for a package from it's Pulumi Schema. This documentation is currently hosted in the Pulumi Registry, but can in principle be hosted in other contexts as well.
  • Pulumi Configuration Language: An internal representation of Pulumi programs which supports all core concepts of the Pulumi programming model in a minimal form. Although not exposed directly to users today, this intermediate representation is used to support a variety of program conversion tasks, from and to various supported Pulumi languages.
  • Program Generation for Node.js, Python, Go and .NET: Support for lowering Pulumi Configuration Language into each of the supported Pulumi languages, such that examples and programs can be generated for the language.

These foundations enable a vast array of features supported in the Pulumi Platform, including:

  • Pulumi support for Node.js, Python, Go, .NET, Java and YAML: Each Pulumi language is supported by defining a representation of the Pulumi resource model in that language, and then implementing SDK Code Generation and Program Generation for the language.
  • Pulumi Packages: Pulumi packages define a set of resources using Pulumi Schema, and use the CrossCode SDK Code Generators for every Pulumi language automatically.
  • pulumi import: Cloud infrastructure resources deployed outside of Pulumi can be imported into Pulumi, including generated Pulumi code in your language of choice which defines the infrastructure. This builds on the program generation support from CrossCode.
  • tf2pulumi, arm2pulumi, crd2pulumi, kube2pulumi and cf2pulumi: These tools convert the source IaC format into an intermediate Pulumi Configuration Language model, and then use the CrossCode program generation support to convert that ultimately into the language a Pulumi user wants to use for their infrastructure.
  • pulumi convert: The pulumi convert command allows Pulumi YAML programs to be converted into programs in any other Pulumi language. Because Pulumi YAML is a proper subset of what can be expressed in Pulumi Configuration Language, this conversion from Pulumi YAML to Pulumi Configuration Language and then into each Pulumi language can be done faithfully.
  • Pulumi Registry: The Pulumi Registry provides discovery and documentation hosting for all Pulumi Packages. It is powered by the Pulumi Schema and CrossCode documentation generation features.

Learn more about Pulumi CrossCode at https://www.pulumi.com/crosscode/.