pulumi/sdk/go/common/resource/plugin/plugin.go

480 lines
16 KiB
Go
Raw Permalink Normal View History

2022-11-01 15:15:09 +00:00
// Copyright 2016-2022, Pulumi Corporation.
2018-05-22 19:43:36 +00:00
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Begin resource modeling and planning This change introduces a new package, pkg/resource, that will form the foundation for actually performing deployment plans and applications. It contains the following key abstractions: * resource.Provider is a wrapper around the CRUD operations exposed by underlying resource plugins. It will eventually defer to resource.Plugin, which itself defers -- over an RPC interface -- to the actual plugin, one per package exposing resources. The provider will also understand how to load, cache, and overall manage the lifetime of each plugin. * resource.Resource is the actual resource object. This is created from the overall evaluation object graph, but is simplified. It contains only serializable properties, for example. Inter-resource references are translated into serializable monikers as part of creating the resource. * resource.Moniker is a serializable string that uniquely identifies a resource in the Mu system. This is in contrast to resource IDs, which are generated by resource providers and generally opaque to the Mu system. See marapongo/mu#69 for more information about monikers and some of their challenges (namely, designing a stable algorithm). * resource.Snapshot is a "snapshot" taken from a graph of resources. This is a transitive closure of state representing one possible configuration of a given environment. This is what plans are created from. Eventually, two snapshots will be diffable, in order to perform incremental updates. One way of thinking about this is that a snapshot of the old world's state is advanced, one step at a time, until it reaches a desired snapshot of the new world's state. * resource.Plan is a plan for carrying out desired CRUD operations on a target environment. Each plan consists of zero-to-many Steps, each of which has a CRUD operation type, a resource target, and a next step. This is an enumerator because it is possible the plan will evolve -- and introduce new steps -- as it is carried out (hence, the Next() method). At the moment, this is linearized; eventually, we want to make this more "graph-like" so that we can exploit available parallelism within the dependencies. There are tons of TODOs remaining. However, the `mu plan` command is functioning with these new changes -- including colorization FTW -- so I'm landing it now. This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 20:31:48 +00:00
package plugin
Begin resource modeling and planning This change introduces a new package, pkg/resource, that will form the foundation for actually performing deployment plans and applications. It contains the following key abstractions: * resource.Provider is a wrapper around the CRUD operations exposed by underlying resource plugins. It will eventually defer to resource.Plugin, which itself defers -- over an RPC interface -- to the actual plugin, one per package exposing resources. The provider will also understand how to load, cache, and overall manage the lifetime of each plugin. * resource.Resource is the actual resource object. This is created from the overall evaluation object graph, but is simplified. It contains only serializable properties, for example. Inter-resource references are translated into serializable monikers as part of creating the resource. * resource.Moniker is a serializable string that uniquely identifies a resource in the Mu system. This is in contrast to resource IDs, which are generated by resource providers and generally opaque to the Mu system. See marapongo/mu#69 for more information about monikers and some of their challenges (namely, designing a stable algorithm). * resource.Snapshot is a "snapshot" taken from a graph of resources. This is a transitive closure of state representing one possible configuration of a given environment. This is what plans are created from. Eventually, two snapshots will be diffable, in order to perform incremental updates. One way of thinking about this is that a snapshot of the old world's state is advanced, one step at a time, until it reaches a desired snapshot of the new world's state. * resource.Plan is a plan for carrying out desired CRUD operations on a target environment. Each plan consists of zero-to-many Steps, each of which has a CRUD operation type, a resource target, and a next step. This is an enumerator because it is possible the plan will evolve -- and introduce new steps -- as it is carried out (hence, the Next() method). At the moment, this is linearized; eventually, we want to make this more "graph-like" so that we can exploit available parallelism within the dependencies. There are tons of TODOs remaining. However, the `mu plan` command is functioning with these new changes -- including colorization FTW -- so I'm landing it now. This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 20:31:48 +00:00
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
import (
2017-02-20 20:34:15 +00:00
"bufio"
"encoding/json"
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
"errors"
2022-11-01 15:15:09 +00:00
"fmt"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
"io"
"os"
"os/exec"
"path/filepath"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
"strconv"
"strings"
"sync/atomic"
"syscall"
"time"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
multierror "github.com/hashicorp/go-multierror"
Add a span for plugin startup/launch time (#14100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description Adds a tracing span that covers the time spent waiting for a plugin to initialize. This will make it easier to distinguish how much provider initialization time is contributing to overall pulumi command duration. Fixes https://github.com/pulumi/pulumi/issues/13986 (Traces from within the provider plugins are tracked in separate issues) ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works It's pretty difficult to test this method independently without some refactoring. Since this change is diagnostic only, I lean toward adding without test to avoid dragging in a potentially complicated restructuring, but open to suggestions. <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-06 20:47:18 +00:00
opentracing "github.com/opentracing/opentracing-go"
"golang.org/x/net/context"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/connectivity"
"google.golang.org/grpc/status"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
"github.com/pulumi/pulumi/sdk/v3/go/common/apitype"
"github.com/pulumi/pulumi/sdk/v3/go/common/diag"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
)
// PulumiPluginJSON represents additional information about a package's associated Pulumi plugin.
// For Python, the content is inside a pulumi-plugin.json file inside the package.
// For Node.js, the content is within the package.json file, under the "pulumi" node.
// For .NET, the content is inside a pulumi-plugin.json file inside the NuGet package.
// For Go, the content is inside a pulumi-plugin.json file inside the module.
type PulumiPluginJSON struct {
// Indicates whether the package has an associated resource plugin. Set to false to indicate no plugin.
Resource bool `json:"resource"`
// Optional plugin name. If not set, the plugin name is derived from the package name.
Name string `json:"name,omitempty"`
// Optional plugin version. If not set, the version is derived from the package version (if possible).
Version string `json:"version,omitempty"`
// Optional plugin server. If not set, the default server is used when installing the plugin.
Server string `json:"server,omitempty"`
}
func (plugin *PulumiPluginJSON) JSON() ([]byte, error) {
json, err := json.MarshalIndent(plugin, "", " ")
if err != nil {
return nil, err
}
return append(json, '\n'), nil
}
func LoadPulumiPluginJSON(path string) (*PulumiPluginJSON, error) {
b, err := os.ReadFile(path)
if err != nil {
// Deliberately not wrapping the error here so that os.IsNotExist checks can be used to determine
// if the file could not be opened due to it not existing.
return nil, err
}
plugin := &PulumiPluginJSON{}
if err := json.Unmarshal(b, plugin); err != nil {
return nil, err
}
return plugin, nil
}
type plugin struct {
stdoutDone <-chan bool
stderrDone <-chan bool
Bin string
Args []string
// Env specifies the environment of the plugin in the same format as go's os/exec.Cmd.Env
// https://golang.org/pkg/os/exec/#Cmd (each entry is of the form "key=value").
2022-04-12 14:32:54 +00:00
Env []string
Conn *grpc.ClientConn
// Function to trigger the plugin to be killed. If the plugin is a process this will just SIGKILL it.
Kill func() error
Stdin io.WriteCloser
Stdout io.ReadCloser
Stderr io.ReadCloser
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
// pluginRPCConnectionTimeout dictates how long we wait for the plugin's RPC to become available.
var pluginRPCConnectionTimeout = time.Second * 10
// A unique ID provided to the output stream of each plugin. This allows the output of the plugin
// to be streamed to the display, while still allowing that output to be sent a small piece at a
// time.
var nextStreamID int32
// errRunPolicyModuleNotFound is returned when we determine that the plugin failed to load because
// the stack's Pulumi SDK did not have the required modules. i.e. is too old.
var errRunPolicyModuleNotFound = errors.New("pulumi SDK does not support policy as code")
// errPluginNotFound is returned when we try to execute a plugin but it is not found on disk.
var errPluginNotFound = errors.New("plugin not found")
2022-11-01 15:15:09 +00:00
func dialPlugin(portNum int, bin, prefix string, dialOptions []grpc.DialOption) (*grpc.ClientConn, error) {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
port := strconv.Itoa(portNum)
2022-11-01 15:15:09 +00:00
// Now that we have the port, go ahead and create a gRPC client connection to it.
2022-11-01 15:15:09 +00:00
conn, err := grpc.Dial("127.0.0.1:"+port, dialOptions...)
if err != nil {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("could not dial plugin [%v] over RPC: %w", bin, err)
}
// Now wait for the gRPC connection to the plugin to become ready.
// TODO[pulumi/pulumi#337]: in theory, this should be unnecessary. gRPC's default WaitForReady behavior
// should auto-retry appropriately. On Linux, however, we are observing different behavior. In the meantime
// while this bug exists, we'll simply do a bit of waiting of our own up front.
timeout, _ := context.WithTimeout(context.Background(), pluginRPCConnectionTimeout)
for {
s := conn.GetState()
if s == connectivity.Ready {
// The connection is supposedly ready; but we will make sure it is *actually* ready by sending a dummy
// method invocation to the server. Until it responds successfully, we can't safely proceed.
outer:
for {
err = grpc.Invoke(timeout, "", nil, nil, conn)
if err == nil {
break // successful connect
all: Fix revive issues Fixes the following issues found by revive included in the latest release of golangci-lint. Full list of issues: **pkg** ``` backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive) backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive) codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive) codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive) resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive) resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive) ``` **sdk** ``` go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive) go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive) go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive) nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive) ``` **tests** ``` integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) ```
2023-03-20 23:48:02 +00:00
}
all: Fix revive issues Fixes the following issues found by revive included in the latest release of golangci-lint. Full list of issues: **pkg** ``` backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive) backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive) codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive) codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive) resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive) resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive) ``` **sdk** ``` go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive) go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive) go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive) nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive) ``` **tests** ``` integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) ```
2023-03-20 23:48:02 +00:00
// We have an error; see if it's a known status and, if so, react appropriately.
status, ok := status.FromError(err)
if ok {
turn on the golangci-lint exhaustive linter (#15028) Turn on the golangci-lint exhaustive linter. This is the first step towards catching more missing cases during development rather than in tests, or in production. This might be best reviewed commit-by-commit, as the first commit turns on the linter with the `default-signifies-exhaustive: true` option set, which requires a lot less changes in the current codebase. I think it's probably worth doing the second commit as well, as that will get us the real benefits, even though we end up with a little bit more churn. However it means all the `switch` statements are covered, which isn't the case after the first commit, since we do have a lot of `default` statements that just call `assert.Fail`. Fixes #14601 ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2024-01-17 16:50:41 +00:00
//nolint:exhaustive // we have a default case for other statuses
all: Fix revive issues Fixes the following issues found by revive included in the latest release of golangci-lint. Full list of issues: **pkg** ``` backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive) backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive) codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive) codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive) resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive) resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive) ``` **sdk** ``` go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive) go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive) go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive) nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive) ``` **tests** ``` integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) ```
2023-03-20 23:48:02 +00:00
switch status.Code() {
case codes.Unavailable:
// The server is unavailable. This is the Linux bug. Wait a little and retry.
time.Sleep(time.Millisecond * 10)
continue // keep retrying
default:
// Since we sent "" as the method above, this is the expected response. Ready to go.
break outer
}
}
all: Fix revive issues Fixes the following issues found by revive included in the latest release of golangci-lint. Full list of issues: **pkg** ``` backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive) backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive) backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive) backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive) cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive) codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive) codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive) codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive) codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive) resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive) resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive) ``` **sdk** ``` go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive) go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive) go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive) go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive) go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive) nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive) python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive) ``` **tests** ``` integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive) ```
2023-03-20 23:48:02 +00:00
// Unexpected error; get outta dodge.
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("%v plugin [%v] did not come alive: %w", prefix, bin, err)
}
break
}
// Not ready yet; ask the gRPC client APIs to block until the state transitions again so we can retry.
if !conn.WaitForStateChange(timeout, s) {
return nil, fmt.Errorf("%v plugin [%v] did not begin responding to RPC connections", prefix, bin)
}
}
return conn, nil
}
func newPlugin(ctx *Context, pwd, bin, prefix string, kind apitype.PluginKind,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
args, env []string, dialOptions []grpc.DialOption,
) (*plugin, error) {
if logging.V(9) {
var argstr string
for i, arg := range args {
if i > 0 {
argstr += ","
}
argstr += arg
}
Add a span for plugin startup/launch time (#14100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description Adds a tracing span that covers the time spent waiting for a plugin to initialize. This will make it easier to distinguish how much provider initialization time is contributing to overall pulumi command duration. Fixes https://github.com/pulumi/pulumi/issues/13986 (Traces from within the provider plugins are tracked in separate issues) ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works It's pretty difficult to test this method independently without some refactoring. Since this change is diagnostic only, I lean toward adding without test to avoid dragging in a potentially complicated restructuring, but open to suggestions. <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-06 20:47:18 +00:00
logging.V(9).Infof("newPlugin(): Launching plugin '%v' from '%v' with args: %v", prefix, bin, argstr)
}
Add a span for plugin startup/launch time (#14100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description Adds a tracing span that covers the time spent waiting for a plugin to initialize. This will make it easier to distinguish how much provider initialization time is contributing to overall pulumi command duration. Fixes https://github.com/pulumi/pulumi/issues/13986 (Traces from within the provider plugins are tracked in separate issues) ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works It's pretty difficult to test this method independently without some refactoring. Since this change is diagnostic only, I lean toward adding without test to avoid dragging in a potentially complicated restructuring, but open to suggestions. <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-10-06 20:47:18 +00:00
// Create a span for the plugin initialization
opts := []opentracing.StartSpanOption{
opentracing.Tag{Key: "prefix", Value: prefix},
opentracing.Tag{Key: "bin", Value: bin},
opentracing.Tag{Key: "pulumi-decorator", Value: prefix + ":" + bin},
}
if ctx != nil && ctx.tracingSpan != nil {
opts = append(opts, opentracing.ChildOf(ctx.tracingSpan.Context()))
}
tracingSpan := opentracing.StartSpan("newPlugin", opts...)
defer tracingSpan.Finish()
// Try to execute the binary.
plug, err := execPlugin(ctx, bin, prefix, kind, args, pwd, env)
if err != nil {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("failed to load plugin %s: %w", bin, err)
}
contract.Assertf(plug != nil, "plugin %v canot be nil", bin)
// If we did not successfully launch the plugin, we still need to wait for stderr and stdout to drain.
defer func() {
if plug.Conn == nil {
contract.IgnoreError(plug.Close())
}
}()
outStreamID := atomic.AddInt32(&nextStreamID, 1)
errStreamID := atomic.AddInt32(&nextStreamID, 1)
// For now, we will spawn goroutines that will spew STDOUT/STDERR to the relevant diag streams.
var sawPolicyModuleNotFoundErr bool
runtrace := func(t io.Reader, stderr bool, done chan<- bool) {
reader := bufio.NewReader(t)
for {
msg, readerr := reader.ReadString('\n')
// Even if we've hit the end of the stream, we want to check for non-empty content.
// The reason is that if the last line is missing a \n, we still want to include it.
if strings.TrimSpace(msg) != "" {
// We may be trying to run a plugin that isn't present in the SDK installed with the Policy Pack.
// e.g. the stack's package.json does not contain a recent enough @pulumi/pulumi.
//
// Rather than fail with an opaque error because we didn't get the gRPC port, inspect if it
// is a well-known problem and return a better error as appropriate.
if strings.Contains(msg, "Cannot find module '@pulumi/pulumi/cmd/run-policy-pack'") {
sawPolicyModuleNotFoundErr = true
}
if stderr {
ctx.Diag.Infoerrf(diag.StreamMessage("" /*urn*/, msg, errStreamID))
} else {
ctx.Diag.Infof(diag.StreamMessage("" /*urn*/, msg, outStreamID))
}
Improve output formatting This change improves our output formatting by generally adding fewer prefixes. As shown in pulumi/pulumi#359, we were being excessively verbose in many places, including prefixing every console.out with "langhost[nodejs].stdout: ", displaying full stack traces for simple errors like missing configuration, etc. Overall, this change includes the following: * Don't prefix stdout and stderr output from the program, other than the standard "info:" prefix. I experimented with various schemes here, but they all felt gratuitous. Simply emitting the output seems fine, especially as it's closer to what would happen if you just ran the program under node. * Do NOT make writes to stderr fail the plan/deploy. Previously we assumed that any console.errors, for instance, meant that the overall program should fail. This simply isn't how stderr is treated generally and meant you couldn't use certain logging techniques and libraries, among other things. * Do make sure that stderr writes in the program end up going to stderr in the Pulumi CLI output, however, so that redirection works as it should. This required a new Infoerr log level. * Make a small fix to the planning logic so we don't attempt to print the summary if an error occurs. * Finally, add a new error type, RunError, that when thrown and uncaught does not result in a full stack trace being printed. Anyone can use this, however, we currently use it for config errors so that we can terminate with a pretty error message, rather than the monstrosity shown in pulumi/pulumi#359.
2017-09-23 12:20:11 +00:00
}
// If we've hit the end of the stream, break out and close the channel.
if readerr != nil {
break
}
}
close(done)
}
// Set up a tracer on stderr before going any further, since important errors might get communicated this way.
stderrDone := make(chan bool)
plug.stderrDone = stderrDone
go runtrace(plug.Stderr, true, stderrDone)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
// Now that we have a process, we expect it to write a single line to STDOUT: the port it's listening on. We only
// read a byte at a time so that STDOUT contains everything after the first newline.
2022-11-01 15:15:09 +00:00
var portString string
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
b := make([]byte, 1)
for {
2017-06-09 19:51:31 +00:00
n, readerr := plug.Stdout.Read(b)
if readerr != nil {
2022-04-12 14:32:54 +00:00
killerr := plug.Kill()
contract.IgnoreError(killerr) // We are ignoring because the readerr trumps it.
// If from the output we have seen, return a specific error if possible.
if sawPolicyModuleNotFoundErr {
return nil, errRunPolicyModuleNotFound
}
// Fall back to a generic, opaque error.
2022-11-01 15:15:09 +00:00
if portString == "" {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("could not read plugin [%v] stdout: %w", bin, readerr)
}
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("failure reading plugin [%v] stdout (read '%v'): %w",
bin, portString, readerr)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
if n > 0 && b[0] == '\n' {
break
}
2022-11-01 15:15:09 +00:00
portString += string(b[:n])
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
2023-01-19 10:45:16 +00:00
// Trim any whitespace from the first line (this is to handle things like windows that will write
// "1234\r\n", or slightly odd providers that might add whitespace like "1234 ")
portString = strings.TrimSpace(portString)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
// Parse the output line (minus the '\n') to ensure it's a numeric port.
2022-11-01 15:15:09 +00:00
var port int
if port, err = strconv.Atoi(portString); err != nil {
2022-04-12 14:32:54 +00:00
killerr := plug.Kill()
contract.IgnoreError(killerr) // ignoring the error because the existing one trumps it.
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf(
"%v plugin [%v] wrote a non-numeric port to stdout ('%v'): %w", prefix, bin, port, err)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
// After reading the port number, set up a tracer on stdout just so other output doesn't disappear.
stdoutDone := make(chan bool)
plug.stdoutDone = stdoutDone
go runtrace(plug.Stdout, false, stdoutDone)
2017-02-20 20:34:15 +00:00
2022-11-01 15:15:09 +00:00
conn, err := dialPlugin(port, bin, prefix, dialOptions)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
if err != nil {
return nil, err
}
// Done; store the connection and return the plugin info.
plug.Conn = conn
return plug, nil
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
// execPlugin starts the plugin executable.
func execPlugin(ctx *Context, bin, prefix string, kind apitype.PluginKind,
all: Reformat with gofumpt Per team discussion, switching to gofumpt. [gofumpt][1] is an alternative, stricter alternative to gofmt. It addresses other stylistic concerns that gofmt doesn't yet cover. [1]: https://github.com/mvdan/gofumpt See the full list of [Added rules][2], but it includes: - Dropping empty lines around function bodies - Dropping unnecessary variable grouping when there's only one variable - Ensuring an empty line between multi-line functions - simplification (`-s` in gofmt) is always enabled - Ensuring multi-line function signatures end with `) {` on a separate line. [2]: https://github.com/mvdan/gofumpt#Added-rules gofumpt is stricter, but there's no lock-in. All gofumpt output is valid gofmt output, so if we decide we don't like it, it's easy to switch back without any code changes. gofumpt support is built into the tooling we use for development so this won't change development workflows. - golangci-lint includes a gofumpt check (enabled in this PR) - gopls, the LSP for Go, includes a gofumpt option (see [installation instrutions][3]) [3]: https://github.com/mvdan/gofumpt#installation This change was generated by running: ```bash gofumpt -w $(rg --files -g '*.go' | rg -v testdata | rg -v compilation_error) ``` The following files were manually tweaked afterwards: - pkg/cmd/pulumi/stack_change_secrets_provider.go: one of the lines overflowed and had comments in an inconvenient place - pkg/cmd/pulumi/destroy.go: `var x T = y` where `T` wasn't necessary - pkg/cmd/pulumi/policy_new.go: long line because of error message - pkg/backend/snapshot_test.go: long line trying to assign three variables in the same assignment I have included mention of gofumpt in the CONTRIBUTING.md.
2023-03-03 16:36:39 +00:00
pluginArgs []string, pwd string, env []string,
) (*plugin, error) {
args := buildPluginArguments(pluginArgumentOptions{
pluginArgs: pluginArgs,
tracingEndpoint: cmdutil.TracingEndpoint,
logFlow: logging.LogFlow,
logToStderr: logging.LogToStderr,
verbose: logging.Verbose,
})
// Check to see if we have a binary we can invoke directly
if _, err := os.Stat(bin); os.IsNotExist(err) {
// If we don't have the expected binary, see if we have a "PulumiPlugin.yaml" or "PulumiPolicy.yaml"
pluginDir := filepath.Dir(bin)
var runtimeInfo workspace.ProjectRuntimeInfo
if kind == apitype.ResourcePlugin || kind == apitype.ConverterPlugin {
proj, err := workspace.LoadPluginProject(filepath.Join(pluginDir, "PulumiPlugin.yaml"))
if err != nil {
return nil, fmt.Errorf("loading PulumiPlugin.yaml: %w", err)
}
runtimeInfo = proj.Runtime
} else if kind == apitype.AnalyzerPlugin {
proj, err := workspace.LoadPluginProject(filepath.Join(pluginDir, "PulumiPolicy.yaml"))
if err != nil {
return nil, fmt.Errorf("loading PulumiPolicy.yaml: %w", err)
}
runtimeInfo = proj.Runtime
} else {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, errors.New("language plugins must be executable binaries")
}
logging.V(9).Infof("Launching plugin '%v' from '%v' via runtime '%s'", prefix, pluginDir, runtimeInfo.Name())
// ProgramInfo needs pluginDir to be an absolute path
pluginDir, err = filepath.Abs(pluginDir)
if err != nil {
return nil, fmt.Errorf("getting absolute path for plugin directory: %w", err)
}
info := NewProgramInfo(pluginDir, pluginDir, ".", runtimeInfo.Options())
runtime, err := ctx.Host.LanguageRuntime(runtimeInfo.Name(), info)
if err != nil {
Enable perfsprint linter (#14813) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Prompted by a comment in another review: https://github.com/pulumi/pulumi/pull/14654#discussion_r1419995945 This lints that we don't use `fmt.Errorf` when `errors.New` will suffice, it also covers a load of other cases where `Sprintf` is sub-optimal. Most of these edits were made by running `perfsprint --fix`. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @Pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
2023-12-12 12:19:42 +00:00
return nil, fmt.Errorf("loading runtime: %w", err)
}
stdout, stderr, kill, err := runtime.RunPlugin(RunPluginInfo{
Info: info,
WorkingDirectory: ctx.Pwd,
Args: pluginArgs,
Env: env,
})
if err != nil {
return nil, err
}
return &plugin{
Bin: bin,
Args: args,
Env: env,
Kill: func() error { kill(); return nil },
Stdout: io.NopCloser(stdout),
Stderr: io.NopCloser(stderr),
}, nil
}
cmd := exec.Command(bin, args...)
cmdutil.RegisterProcessGroup(cmd)
cmd.Dir = pwd
if len(env) > 0 {
cmd.Env = env
}
in, _ := cmd.StdinPipe()
out, _ := cmd.StdoutPipe()
err, _ := cmd.StderrPipe()
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
if err := cmd.Start(); err != nil {
// If we try to run a plugin that isn't found, intercept the error
// and instead return a custom one so we can more easily check for
// it upstream
//
// In the case of PAC, note that the plugin usually _does_ exist.
// It is a shell script like "pulumi-analyzer-policy". But during
// the execution of that script, it fails with the ENOENT error.
if pathErr, ok := err.(*os.PathError); ok {
syscallErr, ok := pathErr.Err.(syscall.Errno)
if ok && syscallErr == syscall.ENOENT {
return nil, errPluginNotFound
}
}
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
return nil, err
}
2022-04-12 14:32:54 +00:00
kill := func() error {
var result *multierror.Error
// On each platform, plugins are not loaded directly, instead a shell launches each plugin as a child process, so
// instead we need to kill all the children of the PID we have recorded, as well. Otherwise we will block waiting
// for the child processes to close.
if err := cmdutil.KillChildren(cmd.Process.Pid); err != nil {
result = multierror.Append(result, err)
}
// IDEA: consider a more graceful termination than just SIGKILL.
if err := cmd.Process.Kill(); err != nil {
result = multierror.Append(result, err)
}
return result.ErrorOrNil()
2022-04-12 14:32:54 +00:00
}
return &plugin{
Bin: bin,
Args: args,
Env: env,
2022-04-12 14:32:54 +00:00
Kill: kill,
Stdin: in,
Stdout: out,
Stderr: err,
}, nil
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}
type pluginArgumentOptions struct {
pluginArgs []string
tracingEndpoint string
logFlow, logToStderr bool
verbose int
}
func buildPluginArguments(opts pluginArgumentOptions) []string {
var args []string
// Flow the logging information if set.
if opts.logFlow {
if opts.logToStderr {
args = append(args, "--logtostderr")
}
if opts.verbose > 0 {
args = append(args, "-v="+strconv.Itoa(opts.verbose))
}
}
if opts.tracingEndpoint != "" {
args = append(args, "--tracing", opts.tracingEndpoint)
}
args = append(args, opts.pluginArgs...)
return args
}
func (p *plugin) Close() error {
if p.Conn != nil {
contract.IgnoreClose(p.Conn)
}
2022-04-12 14:32:54 +00:00
result := p.Kill()
// Wait for stdout and stderr to drain if we attached to the plugin we won't have a stdout/err
if p.stdoutDone != nil {
<-p.stdoutDone
}
if p.stderrDone != nil {
<-p.stderrDone
}
2017-10-31 00:35:36 +00:00
return result
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
}