pulumi/sdk/go/common/resource/plugin/langruntime.go

190 lines
7.6 KiB
Go
Raw Permalink Normal View History

2018-05-22 19:43:36 +00:00
// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package plugin
import (
"context"
"fmt"
"io"
"path/filepath"
"github.com/hashicorp/hcl/v2"
pass through property values to RunRequest (#14273) # Description This is an alternative to https://github.com/pulumi/pulumi/pull/14244. Instead of adding type information to the run request, pass the config through as property values. Property values are properly encoded on the wire, and can be unmarshalled on the other end including type information, so this should be a more future proof way to go forward. Eventually we'll want to parse the config directly into property values, but that can be left for the future, as it's a bigger change. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-20 10:44:16 +00:00
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/config"
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
pulumirpc "github.com/pulumi/pulumi/sdk/v3/proto/go"
structpb "google.golang.org/protobuf/types/known/structpb"
)
// ProgramInfo contains minimal information about the program to be run.
type ProgramInfo struct {
root string
program string
entryPoint string
options map[string]any
}
func NewProgramInfo(rootDirectory, programDirectory, entryPoint string, options map[string]any) ProgramInfo {
isFileName := func(path string) bool {
return filepath.Base(path) == path
}
Ensure project plugins are absolute paths (#15470) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/15467. This tightens the restriction on paths passed to `NewProgramInfo`. Previously it allowed relative paths like "./providers/my_provider". That is now an error. This is correct behaviour. The fields of this structure are passed via protobuf and the descriptions for them in the proto spec are that they should always be absolute paths. Where we build plugin paths we ensure that if they're relative we resolve them to what they are relative to. That is generally _not_ the current working directory so `filepath.Abs` doesn't do the right thing here. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-22 11:43:18 +00:00
if !filepath.IsAbs(rootDirectory) {
panic(fmt.Sprintf("rootDirectory '%s' is not a valid path when creating ProgramInfo", rootDirectory))
}
Ensure project plugins are absolute paths (#15470) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/15467. This tightens the restriction on paths passed to `NewProgramInfo`. Previously it allowed relative paths like "./providers/my_provider". That is now an error. This is correct behaviour. The fields of this structure are passed via protobuf and the descriptions for them in the proto spec are that they should always be absolute paths. Where we build plugin paths we ensure that if they're relative we resolve them to what they are relative to. That is generally _not_ the current working directory so `filepath.Abs` doesn't do the right thing here. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-22 11:43:18 +00:00
if !filepath.IsAbs(programDirectory) {
panic(fmt.Sprintf("programDirectory '%s' is not a valid path when creating ProgramInfo", programDirectory))
}
if !isFileName(entryPoint) && entryPoint != "." {
panic(fmt.Sprintf("entryPoint '%s' was not a valid file name when creating ProgramInfo", entryPoint))
}
return ProgramInfo{
root: rootDirectory,
program: programDirectory,
entryPoint: entryPoint,
options: options,
}
}
// The programs root directory, i.e. where the Pulumi.yaml file is.
func (info ProgramInfo) RootDirectory() string {
return info.root
}
// The programs directory, generally the same as or a subdirectory of the root directory.
func (info ProgramInfo) ProgramDirectory() string {
return info.program
}
// The programs main entrypoint, either a file path relative to the program directory or "." for the program directory.
func (info ProgramInfo) EntryPoint() string {
return info.entryPoint
}
// Runtime plugin options for the program
func (info ProgramInfo) Options() map[string]any {
return info.options
}
func (info ProgramInfo) String() string {
return fmt.Sprintf("root=%s, program=%s, entryPoint=%s", info.root, info.program, info.entryPoint)
}
func (info ProgramInfo) Marshal() (*pulumirpc.ProgramInfo, error) {
opts, err := structpb.NewStruct(info.options)
if err != nil {
return nil, fmt.Errorf("failed to marshal options: %w", err)
}
return &pulumirpc.ProgramInfo{
RootDirectory: info.root,
ProgramDirectory: info.program,
EntryPoint: info.entryPoint,
Options: opts,
}, nil
}
// LanguageRuntime is a convenient interface for interacting with language runtime plugins. These tend to be
// dynamically loaded as plugins, although this interface hides this fact from the calling code.
type LanguageRuntime interface {
// Closer closes any underlying OS resources associated with this plugin (like processes, RPC channels, etc).
io.Closer
// GetRequiredPlugins computes the complete set of anticipated plugins required by a program.
GetRequiredPlugins(info ProgramInfo) ([]workspace.PluginSpec, error)
// Run executes a program in the language runtime for planning or deployment purposes. If
// info.DryRun is true, the code must not assume that side-effects or final values resulting
// from resource deployments are actually available. If it is false, on the other hand, a real
// deployment is occurring and it may safely depend on these.
//
// Returns a triple of "error message", "bail", or real "error". If "bail", the caller should
// return result.Bail immediately and not print any further messages to the user.
Run(info RunInfo) (string, bool, error)
// GetPluginInfo returns this plugin's information.
GetPluginInfo() (workspace.PluginInfo, error)
Move InstallDependencies to the language plugin (#9294) * Move InstallDependencies to the language plugin This changes `pulumi new` and `pulumi up <template>` to invoke the language plugin to install dependencies, rather than having the code to install dependencies hardcoded into the cli itself. This does not change the way policypacks or plugin dependencies are installed. In theory we can make pretty much the same change to just invoke the language plugin, but baby steps we don't need to make that change at the same time as this. We used to feed the result of these install commands (dotnet build, npm install, etc) directly through to the CLI stdout/stderr. To mostly maintain that behaviour the InstallDependencies gRCP method streams back bytes to be written to stdout/stderr, those bytes are either read from pipes or a pty that we run the install commands with. The use of a pty is controlled by the global colorisation option in the cli. An alternative designs was to use the Engine interface to Log the results of install commands. This renders very differently to just writing directly to the standard outputs and I don't think would support control codes so well. The design as is means that `npm install` for example is still able to display a progress bar and colors even though we're running it in a separate process and streaming its output back via gRPC. The only "oddity" I feel that's fallen out of this work is that InstallDependencies for python used to overwrite the virtualenv runtime option. It looks like this was because our templates don't bother setting that. Because InstallDependencies doesn't have the project file, and at any rate will be used for policy pack projects in the future, I've moved that logic into `pulumi new` when it mutates the other project file settings. I think we should at some point cleanup so the templates correctly indicate to use a venv, or maybe change python to assume a virtual env of "venv" if none is given? * Just warn if pty fails to open * Add tests and return real tty files * Add to CHANGELOG * lint * format * Test strings * Log pty opening for trace debugging * s/Hack/Workaround * Use termios * Tweak terminal test * lint * Fix windows build
2022-04-03 14:54:59 +00:00
// InstallDependencies will install dependencies for the project, e.g. by running `npm install` for nodejs projects.
InstallDependencies(info ProgramInfo) error
// About returns information about the language runtime.
About() (AboutInfo, error)
// GetProgramDependencies returns information about the dependencies for the given program.
GetProgramDependencies(info ProgramInfo, transitiveDependencies bool) ([]DependencyInfo, error)
// RunPlugin executes a plugin program and returns its result asynchronously.
RunPlugin(info RunPluginInfo) (io.Reader, io.Reader, context.CancelFunc, error)
// GenerateProject generates a program project in the given directory. This will include metadata files such
// as Pulumi.yaml and package.json.
GenerateProject(sourceDirectory, targetDirectory, project string,
strict bool, loaderTarget string, localDependencies map[string]string) (hcl.Diagnostics, error)
// GeneratePlugin generates an SDK package.
Return diagnostics from GeneratePackage (#14661) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14660. Fairly simple change to bring the GeneratePackage RPC method into alignment with the other codegen methods and use returned diagnostics rather than just error values. `gen-sdk` is updated to print those diagnostics, and the python/node/go runtimes updated to return the diagnostics from schema binding as diagnostics rather than just an error value. Might be worth at some point seeing if the rest of package generation could use diagnostics rather than error values, but that's a larger lift. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-05 17:47:52 +00:00
GeneratePackage(
Use the local artifacts, not released artifacts in conformance tests (#15777) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This threads the "local_dependencies" property through to GeneratePackage, following exactly the same semantics as for "GenerateProgram". Fixes https://github.com/pulumi/pulumi/issues/15074. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-03-26 13:10:34 +00:00
directory string, schema string, extraFiles map[string][]byte,
loaderTarget string, localDependencies map[string]string,
Return diagnostics from GeneratePackage (#14661) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes https://github.com/pulumi/pulumi/issues/14660. Fairly simple change to bring the GeneratePackage RPC method into alignment with the other codegen methods and use returned diagnostics rather than just error values. `gen-sdk` is updated to print those diagnostics, and the python/node/go runtimes updated to return the diagnostics from schema binding as diagnostics rather than just an error value. Might be worth at some point seeing if the rest of package generation could use diagnostics rather than error values, but that's a larger lift. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [ ] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-05 17:47:52 +00:00
) (hcl.Diagnostics, error)
// GenerateProgram is similar to GenerateProject but doesn't include any metadata files, just the program
// source code.
GenerateProgram(program map[string]string, loaderTarget string) (map[string][]byte, hcl.Diagnostics, error)
// Pack packs a library package into a language specific artifact in the given destination directory.
Add SupportPack to schemas to write out in the new style (#15713) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This adds a new flag to the schema metadata to tell codegen to use the new proposed style of SDKs where we fill in versions and write go.mods etc. I've reworked pack to operate on packages assuming they're in this new style. That is pack no longer has the responsibility to fill in any version information. This updates python and node codegen to write out SDKs in this new style, and fixes their core libraries to still be buildable via pack. There are two approaches to fixing those, I've chosen option 1 below but could pretty easily rework for option 2. 1) Write the version information directly to the SDKs at the same time as we edit the .version file. To simplify this I've added a new 'set-version.py' script that takes a version string an writes it to all the relevant places (.version, package.json, etc). 2) Write "pack" in the language host to search up the directory tree for the ".version" file and then fill in the version information as we we're doing before with envvar tricks and copying and editing package.json. I think 1 is simpler long term, but does force some amount of cleanup in unrelated bits of the system right now (release makefiles need a small edit). 2 is much more localised but keeps this complexity that sdk/nodejs sdk/python aren't actually valid source modules. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-03-22 09:25:46 +00:00
Pack(packageDirectory string, destinationDirectory string) (string, error)
}
Program dependencies aren't necessarily semantic versions (#15409) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This weakens the contract of GetProgramDependencies that the versions returned are just "strings". The format of those strings is defined by the language plugin. This is especially useful for python which uses PEP440 not semantic versions for it's pip dependencies. There wasn't anywhere in the system that took a strong dependency on these being semvers. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-09 22:18:42 +00:00
// DependencyInfo contains information about a dependency reported by a language runtime.
// These are the languages dependencies, they are not necessarily Pulumi packages.
type DependencyInfo struct {
Program dependencies aren't necessarily semantic versions (#15409) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> This weakens the contract of GetProgramDependencies that the versions returned are just "strings". The format of those strings is defined by the language plugin. This is especially useful for python which uses PEP440 not semantic versions for it's pip dependencies. There wasn't anywhere in the system that took a strong dependency on these being semvers. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-02-09 22:18:42 +00:00
// The name of the dependency.
Name string
// The version of the dependency. Unlike most versions in the system this is not guaranteed to be a semantic
// version.
Version string
}
type AboutInfo struct {
Executable string
Version string
Metadata map[string]string
}
type RunPluginInfo struct {
Info ProgramInfo
WorkingDirectory string
Args []string
Env []string
}
// RunInfo contains all of the information required to perform a plan or deployment operation.
type RunInfo struct {
Info ProgramInfo // the information about the program to run.
pass through property values to RunRequest (#14273) # Description This is an alternative to https://github.com/pulumi/pulumi/pull/14244. Instead of adding type information to the run request, pass the config through as property values. Property values are properly encoded on the wire, and can be unmarshalled on the other end including type information, so this should be a more future proof way to go forward. Eventually we'll want to parse the config directly into property values, but that can be left for the future, as it's a bigger change. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-20 10:44:16 +00:00
MonitorAddress string // the RPC address to the host resource monitor.
Project string // the project name housing the program being run.
Stack string // the stack name being evaluated.
Pwd string // the program's working directory.
Args []string // any arguments to pass to the program.
Config map[config.Key]string // the configuration variables to apply before running.
ConfigSecretKeys []config.Key // the configuration keys that have secret values.
ConfigPropertyMap resource.PropertyMap // the configuration as a property map.
DryRun bool // true if we are performing a dry-run (preview).
QueryMode bool // true if we're only doing a query.
Parallel int // the degree of parallelism for resource operations (<=1 for serial).
Organization string // the organization name housing the program being run (might be empty).
}