2022-11-01 15:15:09 +00:00
|
|
|
// Copyright 2016-2022, Pulumi Corporation.
|
2018-05-22 19:43:36 +00:00
|
|
|
//
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
// you may not use this file except in compliance with the License.
|
|
|
|
// You may obtain a copy of the License at
|
|
|
|
//
|
|
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
//
|
|
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
// See the License for the specific language governing permissions and
|
|
|
|
// limitations under the License.
|
Begin resource modeling and planning
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 20:31:48 +00:00
|
|
|
|
Overhaul resources, planning, and environments
This change, part of pulumi/lumi#90, overhauls quite a bit of the
core resource, planning, environments, and related areas.
The biggest amount of movement comes from the splitting of pkg/resource
into multiple sub-packages. This results in:
- pkg/resource: just the core resource data structures.
- pkg/resource/deployment: all planning and deployment logic.
- pkg/resource/environment: all environment, configuration, and
serialized checkpoint structures and logic.
- pkg/resource/plugin: all dynamically loaded analyzer and
provider logic, including the actual loading and RPC mechanisms.
This also splits the resource abstraction up. We now have:
- resource.Resource: a shared interface.
- resource.Object: a resource that is connected to a live object
that will periodically observe mutations due to ongoing
evaluation of computations. Snapshots of its state may be
taken; however, this is purely a "pre-planning" abstraction.
- resource.State: a snapshot of a resource's state that is frozen.
In other words, it is no longer connected to a live object.
This is what will store provider outputs (ID and properties),
and is what may be serialized into a deployment record.
The branch is in a half-baked state as of this change; more changes
are to come...
2017-06-08 23:37:40 +00:00
|
|
|
package plugin
|
Begin resource modeling and planning
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 20:31:48 +00:00
|
|
|
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
import (
|
2017-02-20 20:34:15 +00:00
|
|
|
"bufio"
|
2020-12-04 03:22:16 +00:00
|
|
|
"encoding/json"
|
2023-12-12 12:19:42 +00:00
|
|
|
"errors"
|
2022-11-01 15:15:09 +00:00
|
|
|
"fmt"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
"io"
|
|
|
|
"os"
|
|
|
|
"os/exec"
|
2022-10-04 08:58:01 +00:00
|
|
|
"path/filepath"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
"strconv"
|
2018-04-13 22:44:35 +00:00
|
|
|
"strings"
|
2018-05-07 22:11:52 +00:00
|
|
|
"sync/atomic"
|
2020-02-28 20:48:53 +00:00
|
|
|
"syscall"
|
2017-09-08 22:11:09 +00:00
|
|
|
"time"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
|
2023-08-31 16:29:55 +00:00
|
|
|
multierror "github.com/hashicorp/go-multierror"
|
2023-10-06 20:47:18 +00:00
|
|
|
opentracing "github.com/opentracing/opentracing-go"
|
2017-09-08 22:11:09 +00:00
|
|
|
"golang.org/x/net/context"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
"google.golang.org/grpc"
|
2017-09-08 22:11:09 +00:00
|
|
|
"google.golang.org/grpc/codes"
|
|
|
|
"google.golang.org/grpc/connectivity"
|
|
|
|
"google.golang.org/grpc/status"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
|
2024-04-25 17:30:30 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/apitype"
|
2021-03-17 13:20:05 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/diag"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
|
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/logging"
|
2022-10-04 08:58:01 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/workspace"
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
)
|
|
|
|
|
2020-12-04 03:22:16 +00:00
|
|
|
// PulumiPluginJSON represents additional information about a package's associated Pulumi plugin.
|
2021-12-15 18:41:44 +00:00
|
|
|
// For Python, the content is inside a pulumi-plugin.json file inside the package.
|
2020-12-04 03:22:16 +00:00
|
|
|
// For Node.js, the content is within the package.json file, under the "pulumi" node.
|
2021-12-15 18:41:44 +00:00
|
|
|
// For .NET, the content is inside a pulumi-plugin.json file inside the NuGet package.
|
|
|
|
// For Go, the content is inside a pulumi-plugin.json file inside the module.
|
2020-12-04 03:22:16 +00:00
|
|
|
type PulumiPluginJSON struct {
|
|
|
|
// Indicates whether the package has an associated resource plugin. Set to false to indicate no plugin.
|
|
|
|
Resource bool `json:"resource"`
|
|
|
|
// Optional plugin name. If not set, the plugin name is derived from the package name.
|
|
|
|
Name string `json:"name,omitempty"`
|
|
|
|
// Optional plugin version. If not set, the version is derived from the package version (if possible).
|
|
|
|
Version string `json:"version,omitempty"`
|
|
|
|
// Optional plugin server. If not set, the default server is used when installing the plugin.
|
|
|
|
Server string `json:"server,omitempty"`
|
|
|
|
}
|
|
|
|
|
|
|
|
func (plugin *PulumiPluginJSON) JSON() ([]byte, error) {
|
|
|
|
json, err := json.MarshalIndent(plugin, "", " ")
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2021-12-02 02:07:23 +00:00
|
|
|
return append(json, '\n'), nil
|
2020-12-04 03:22:16 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
func LoadPulumiPluginJSON(path string) (*PulumiPluginJSON, error) {
|
2023-01-06 22:39:16 +00:00
|
|
|
b, err := os.ReadFile(path)
|
2020-12-04 03:22:16 +00:00
|
|
|
if err != nil {
|
|
|
|
// Deliberately not wrapping the error here so that os.IsNotExist checks can be used to determine
|
|
|
|
// if the file could not be opened due to it not existing.
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2021-12-02 00:07:49 +00:00
|
|
|
plugin := &PulumiPluginJSON{}
|
2020-12-04 03:22:16 +00:00
|
|
|
if err := json.Unmarshal(b, plugin); err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
return plugin, nil
|
|
|
|
}
|
|
|
|
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
type plugin struct {
|
2017-10-16 20:44:37 +00:00
|
|
|
stdoutDone <-chan bool
|
|
|
|
stderrDone <-chan bool
|
|
|
|
|
2019-12-16 22:51:02 +00:00
|
|
|
Bin string
|
|
|
|
Args []string
|
|
|
|
// Env specifies the environment of the plugin in the same format as go's os/exec.Cmd.Env
|
|
|
|
// https://golang.org/pkg/os/exec/#Cmd (each entry is of the form "key=value").
|
2022-04-12 14:32:54 +00:00
|
|
|
Env []string
|
|
|
|
Conn *grpc.ClientConn
|
|
|
|
// Function to trigger the plugin to be killed. If the plugin is a process this will just SIGKILL it.
|
|
|
|
Kill func() error
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
Stdin io.WriteCloser
|
|
|
|
Stdout io.ReadCloser
|
|
|
|
Stderr io.ReadCloser
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
|
|
|
|
2017-09-08 22:11:09 +00:00
|
|
|
// pluginRPCConnectionTimeout dictates how long we wait for the plugin's RPC to become available.
|
|
|
|
var pluginRPCConnectionTimeout = time.Second * 10
|
|
|
|
|
2018-05-07 22:11:52 +00:00
|
|
|
// A unique ID provided to the output stream of each plugin. This allows the output of the plugin
|
|
|
|
// to be streamed to the display, while still allowing that output to be sent a small piece at a
|
|
|
|
// time.
|
|
|
|
var nextStreamID int32
|
|
|
|
|
2019-10-09 20:51:10 +00:00
|
|
|
// errRunPolicyModuleNotFound is returned when we determine that the plugin failed to load because
|
|
|
|
// the stack's Pulumi SDK did not have the required modules. i.e. is too old.
|
|
|
|
var errRunPolicyModuleNotFound = errors.New("pulumi SDK does not support policy as code")
|
|
|
|
|
2020-02-28 20:48:53 +00:00
|
|
|
// errPluginNotFound is returned when we try to execute a plugin but it is not found on disk.
|
|
|
|
var errPluginNotFound = errors.New("plugin not found")
|
|
|
|
|
2022-11-01 15:15:09 +00:00
|
|
|
func dialPlugin(portNum int, bin, prefix string, dialOptions []grpc.DialOption) (*grpc.ClientConn, error) {
|
2023-12-12 12:19:42 +00:00
|
|
|
port := strconv.Itoa(portNum)
|
2022-11-01 15:15:09 +00:00
|
|
|
|
2022-04-19 11:41:18 +00:00
|
|
|
// Now that we have the port, go ahead and create a gRPC client connection to it.
|
2022-11-01 15:15:09 +00:00
|
|
|
conn, err := grpc.Dial("127.0.0.1:"+port, dialOptions...)
|
2022-04-19 11:41:18 +00:00
|
|
|
if err != nil {
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("could not dial plugin [%v] over RPC: %w", bin, err)
|
2022-04-19 11:41:18 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Now wait for the gRPC connection to the plugin to become ready.
|
|
|
|
// TODO[pulumi/pulumi#337]: in theory, this should be unnecessary. gRPC's default WaitForReady behavior
|
|
|
|
// should auto-retry appropriately. On Linux, however, we are observing different behavior. In the meantime
|
|
|
|
// while this bug exists, we'll simply do a bit of waiting of our own up front.
|
|
|
|
timeout, _ := context.WithTimeout(context.Background(), pluginRPCConnectionTimeout)
|
|
|
|
for {
|
|
|
|
s := conn.GetState()
|
|
|
|
if s == connectivity.Ready {
|
|
|
|
// The connection is supposedly ready; but we will make sure it is *actually* ready by sending a dummy
|
|
|
|
// method invocation to the server. Until it responds successfully, we can't safely proceed.
|
|
|
|
outer:
|
|
|
|
for {
|
|
|
|
err = grpc.Invoke(timeout, "", nil, nil, conn)
|
|
|
|
if err == nil {
|
|
|
|
break // successful connect
|
all: Fix revive issues
Fixes the following issues found by revive
included in the latest release of golangci-lint.
Full list of issues:
**pkg**
```
backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive)
backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive)
codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive)
codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive)
resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive)
resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
```
**sdk**
```
go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)
go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive)
go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive)
nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive)
```
**tests**
```
integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
2023-03-20 23:48:02 +00:00
|
|
|
}
|
2022-04-19 11:41:18 +00:00
|
|
|
|
all: Fix revive issues
Fixes the following issues found by revive
included in the latest release of golangci-lint.
Full list of issues:
**pkg**
```
backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive)
backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive)
codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive)
codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive)
resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive)
resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
```
**sdk**
```
go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)
go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive)
go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive)
nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive)
```
**tests**
```
integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
2023-03-20 23:48:02 +00:00
|
|
|
// We have an error; see if it's a known status and, if so, react appropriately.
|
|
|
|
status, ok := status.FromError(err)
|
|
|
|
if ok {
|
turn on the golangci-lint exhaustive linter (#15028)
Turn on the golangci-lint exhaustive linter. This is the first step
towards catching more missing cases during development rather than
in tests, or in production.
This might be best reviewed commit-by-commit, as the first commit turns
on the linter with the `default-signifies-exhaustive: true` option set,
which requires a lot less changes in the current codebase.
I think it's probably worth doing the second commit as well, as that
will get us the real benefits, even though we end up with a little bit
more churn. However it means all the `switch` statements are covered,
which isn't the case after the first commit, since we do have a lot of
`default` statements that just call `assert.Fail`.
Fixes #14601
## Checklist
- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
- [x] I have formatted my code using `gofumpt`
<!--- Please provide details if the checkbox below is to be left
unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my
feature works
<!---
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the
`changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the
Pulumi Cloud,
then the service should honor older versions of the CLI where this
change would not exist.
You must then bump the API version in
/pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi
Cloud API version
<!-- @Pulumi employees: If yes, you must submit corresponding changes in
the service repo. -->
2024-01-17 16:50:41 +00:00
|
|
|
//nolint:exhaustive // we have a default case for other statuses
|
all: Fix revive issues
Fixes the following issues found by revive
included in the latest release of golangci-lint.
Full list of issues:
**pkg**
```
backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive)
backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive)
codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive)
codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive)
resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive)
resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
```
**sdk**
```
go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)
go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive)
go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive)
nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive)
```
**tests**
```
integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
2023-03-20 23:48:02 +00:00
|
|
|
switch status.Code() {
|
|
|
|
case codes.Unavailable:
|
|
|
|
// The server is unavailable. This is the Linux bug. Wait a little and retry.
|
|
|
|
time.Sleep(time.Millisecond * 10)
|
|
|
|
continue // keep retrying
|
|
|
|
default:
|
|
|
|
// Since we sent "" as the method above, this is the expected response. Ready to go.
|
|
|
|
break outer
|
|
|
|
}
|
2022-04-19 11:41:18 +00:00
|
|
|
}
|
all: Fix revive issues
Fixes the following issues found by revive
included in the latest release of golangci-lint.
Full list of issues:
**pkg**
```
backend/display/object_diff.go:47:10: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
backend/display/object_diff.go:716:12: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:742:14: redefines-builtin-id: redefinition of the built-in function delete (revive)
backend/display/object_diff.go:983:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (revive)
backend/httpstate/backend.go:1814:4: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/backend.go:1824:5: redefines-builtin-id: redefinition of the built-in function cap (revive)
backend/httpstate/client/client.go:444:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
backend/httpstate/client/client.go:455:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/org.go:113:4: if-return: redundant if ...; err != nil check, just return error instead. (revive)
cmd/pulumi/util.go:216:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
codegen/docs/gen.go:428:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/hcl2/model/expression.go:2151:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:151:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:329:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/hcl2/syntax/comments.go:381:5: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/nodejs/gen.go:1367:5: redefines-builtin-id: redefinition of the built-in function copy (revive)
codegen/python/gen_program_expressions.go:136:2: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/python/gen_program_expressions.go:142:3: redefines-builtin-id: redefinition of the built-in function close (revive)
codegen/report/report.go:126:6: redefines-builtin-id: redefinition of the built-in function panic (revive)
codegen/schema/docs_test.go:210:10: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
codegen/schema/schema.go:790:2: redefines-builtin-id: redefinition of the built-in type any (revive)
codegen/schema/schema.go:793:4: redefines-builtin-id: redefinition of the built-in type any (revive)
resource/deploy/plan.go:506:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
resource/deploy/snapshot_test.go:59:3: redefines-builtin-id: redefinition of the built-in function copy (revive)
resource/deploy/state_builder.go:108:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
```
**sdk**
```
go/common/resource/plugin/context.go:142:2: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/resource/plugin/plugin.go:142:12: superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)
go/common/resource/properties_diff.go:114:2: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:117:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:122:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:127:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/resource/properties_diff.go:132:4: redefines-builtin-id: redefinition of the built-in function len (revive)
go/common/util/deepcopy/copy.go:30:1: redefines-builtin-id: redefinition of the built-in function copy (revive)
go/common/workspace/creds.go:242:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:569:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi-language-go/main.go:706:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
go/pulumi/run_test.go:925:2: redefines-builtin-id: redefinition of the built-in type any (revive)
go/pulumi/run_test.go:933:3: redefines-builtin-id: redefinition of the built-in type any (revive)
nodejs/cmd/pulumi-language-nodejs/main.go:778:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:1011:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/cmd/pulumi-language-python/main.go:863:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
python/python.go:230:2: redefines-builtin-id: redefinition of the built-in function print (revive)
```
**tests**
```
integration/integration_util_test.go:282:11: superfluous-else: if block ends with a continue statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
2023-03-20 23:48:02 +00:00
|
|
|
|
|
|
|
// Unexpected error; get outta dodge.
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("%v plugin [%v] did not come alive: %w", prefix, bin, err)
|
2022-04-19 11:41:18 +00:00
|
|
|
}
|
|
|
|
break
|
|
|
|
}
|
|
|
|
// Not ready yet; ask the gRPC client APIs to block until the state transitions again so we can retry.
|
|
|
|
if !conn.WaitForStateChange(timeout, s) {
|
2023-12-12 19:09:19 +00:00
|
|
|
return nil, fmt.Errorf("%v plugin [%v] did not begin responding to RPC connections", prefix, bin)
|
2022-04-19 11:41:18 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return conn, nil
|
|
|
|
}
|
|
|
|
|
2024-04-25 17:30:30 +00:00
|
|
|
func newPlugin(ctx *Context, pwd, bin, prefix string, kind apitype.PluginKind,
|
2023-03-03 16:36:39 +00:00
|
|
|
args, env []string, dialOptions []grpc.DialOption,
|
|
|
|
) (*plugin, error) {
|
2018-05-15 22:28:00 +00:00
|
|
|
if logging.V(9) {
|
2017-09-03 19:37:56 +00:00
|
|
|
var argstr string
|
2017-09-08 22:11:09 +00:00
|
|
|
for i, arg := range args {
|
|
|
|
if i > 0 {
|
|
|
|
argstr += ","
|
|
|
|
}
|
|
|
|
argstr += arg
|
2017-09-03 19:37:56 +00:00
|
|
|
}
|
2023-10-06 20:47:18 +00:00
|
|
|
logging.V(9).Infof("newPlugin(): Launching plugin '%v' from '%v' with args: %v", prefix, bin, argstr)
|
2017-09-03 19:37:56 +00:00
|
|
|
}
|
|
|
|
|
2023-10-06 20:47:18 +00:00
|
|
|
// Create a span for the plugin initialization
|
|
|
|
opts := []opentracing.StartSpanOption{
|
|
|
|
opentracing.Tag{Key: "prefix", Value: prefix},
|
|
|
|
opentracing.Tag{Key: "bin", Value: bin},
|
|
|
|
opentracing.Tag{Key: "pulumi-decorator", Value: prefix + ":" + bin},
|
|
|
|
}
|
|
|
|
if ctx != nil && ctx.tracingSpan != nil {
|
|
|
|
opts = append(opts, opentracing.ChildOf(ctx.tracingSpan.Context()))
|
|
|
|
}
|
|
|
|
tracingSpan := opentracing.StartSpan("newPlugin", opts...)
|
|
|
|
defer tracingSpan.Finish()
|
|
|
|
|
2017-08-31 21:31:33 +00:00
|
|
|
// Try to execute the binary.
|
2022-10-04 08:58:01 +00:00
|
|
|
plug, err := execPlugin(ctx, bin, prefix, kind, args, pwd, env)
|
2017-08-31 21:31:33 +00:00
|
|
|
if err != nil {
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("failed to load plugin %s: %w", bin, err)
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
}
|
2023-02-15 01:06:56 +00:00
|
|
|
contract.Assertf(plug != nil, "plugin %v canot be nil", bin)
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
|
2017-10-17 03:51:18 +00:00
|
|
|
// If we did not successfully launch the plugin, we still need to wait for stderr and stdout to drain.
|
|
|
|
defer func() {
|
|
|
|
if plug.Conn == nil {
|
|
|
|
contract.IgnoreError(plug.Close())
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
|
2018-05-07 22:11:52 +00:00
|
|
|
outStreamID := atomic.AddInt32(&nextStreamID, 1)
|
|
|
|
errStreamID := atomic.AddInt32(&nextStreamID, 1)
|
|
|
|
|
Tidy up more lint
This change fixes a few things:
* Most importantly, we need to place a leading "." in the paths
to Gometalinter, otherwise some sub-linters just silently skip
the directory altogether. errcheck is one such linter, which
is a very important one!
* Use an explicit Gometalinter.json file to configure the various
settings. This flips on a few additional linters that aren't
on by default (line line length checking). Sadly, a few that
I'd like to enable take waaaay too much time, so in the future
we may consider a nightly job (this includes code similarity,
unused parameters, unused functions, and others that generally
require global analysis).
* Now that we're running more, however, linting takes a while!
The core Lumi project now takes 26 seconds to lint on my laptop.
That's not terrible, but it's long enough that we don't want to
do the silly "run them twice" thing our Makefiles were previously
doing. Instead, we shall deploy some $$($${PIPESTATUS[1]}-1))-fu
to rely on the fact that grep returns 1 on "zero lines".
* Finally, fix the many issues that this turned up.
I think(?) we are done, except, of course, for needing to drive
down some of the cyclomatic complexity issues (which I'm possibly
going to punt on; see pulumi/lumi#259 for more details).
2017-06-22 19:09:46 +00:00
|
|
|
// For now, we will spawn goroutines that will spew STDOUT/STDERR to the relevant diag streams.
|
2019-10-09 20:51:10 +00:00
|
|
|
var sawPolicyModuleNotFoundErr bool
|
2017-10-16 20:44:37 +00:00
|
|
|
runtrace := func(t io.Reader, stderr bool, done chan<- bool) {
|
2017-04-19 22:01:04 +00:00
|
|
|
reader := bufio.NewReader(t)
|
2018-04-13 22:44:35 +00:00
|
|
|
|
2018-05-07 22:11:52 +00:00
|
|
|
for {
|
|
|
|
msg, readerr := reader.ReadString('\n')
|
2019-10-09 20:51:10 +00:00
|
|
|
|
Fix #3982, missing output which lacks newlines (#8671)
* Fix #3982, missing output which lacks newlines
If the last line printed to stdout or stderr was missing a
terminating newline, it would go entirely missing (in all languages).
The reason for this is a bug in the engine's handling of plugin
outputs: Go's Reader.ReadString('\n') returns a string containing what
was read and/or an error; if the string terminated in a '\n', the
error is nil, and the entire line is returned; if the stream ends,
however, a non-nil error is returned *and* what was read is returned,
even though it wasn't terminated in a newline. The fix is simple:
instead of ignoring that text, we use it, and *then* exit the read-loop.
Also added some test cases since this is subtle and easy to regress.
* Add a changelog entry
2022-01-03 22:39:10 +00:00
|
|
|
// Even if we've hit the end of the stream, we want to check for non-empty content.
|
|
|
|
// The reason is that if the last line is missing a \n, we still want to include it.
|
2018-05-07 22:11:52 +00:00
|
|
|
if strings.TrimSpace(msg) != "" {
|
Fix #3982, missing output which lacks newlines (#8671)
* Fix #3982, missing output which lacks newlines
If the last line printed to stdout or stderr was missing a
terminating newline, it would go entirely missing (in all languages).
The reason for this is a bug in the engine's handling of plugin
outputs: Go's Reader.ReadString('\n') returns a string containing what
was read and/or an error; if the string terminated in a '\n', the
error is nil, and the entire line is returned; if the stream ends,
however, a non-nil error is returned *and* what was read is returned,
even though it wasn't terminated in a newline. The fix is simple:
instead of ignoring that text, we use it, and *then* exit the read-loop.
Also added some test cases since this is subtle and easy to regress.
* Add a changelog entry
2022-01-03 22:39:10 +00:00
|
|
|
// We may be trying to run a plugin that isn't present in the SDK installed with the Policy Pack.
|
|
|
|
// e.g. the stack's package.json does not contain a recent enough @pulumi/pulumi.
|
|
|
|
//
|
|
|
|
// Rather than fail with an opaque error because we didn't get the gRPC port, inspect if it
|
|
|
|
// is a well-known problem and return a better error as appropriate.
|
|
|
|
if strings.Contains(msg, "Cannot find module '@pulumi/pulumi/cmd/run-policy-pack'") {
|
|
|
|
sawPolicyModuleNotFoundErr = true
|
|
|
|
}
|
|
|
|
|
2018-05-07 22:11:52 +00:00
|
|
|
if stderr {
|
2018-08-31 19:33:01 +00:00
|
|
|
ctx.Diag.Infoerrf(diag.StreamMessage("" /*urn*/, msg, errStreamID))
|
2018-05-07 22:11:52 +00:00
|
|
|
} else {
|
2018-08-31 19:33:01 +00:00
|
|
|
ctx.Diag.Infof(diag.StreamMessage("" /*urn*/, msg, outStreamID))
|
2018-05-07 22:11:52 +00:00
|
|
|
}
|
Improve output formatting
This change improves our output formatting by generally adding
fewer prefixes. As shown in pulumi/pulumi#359, we were being
excessively verbose in many places, including prefixing every
console.out with "langhost[nodejs].stdout: ", displaying full
stack traces for simple errors like missing configuration, etc.
Overall, this change includes the following:
* Don't prefix stdout and stderr output from the program, other
than the standard "info:" prefix. I experimented with various
schemes here, but they all felt gratuitous. Simply emitting
the output seems fine, especially as it's closer to what would
happen if you just ran the program under node.
* Do NOT make writes to stderr fail the plan/deploy. Previously
we assumed that any console.errors, for instance, meant that
the overall program should fail. This simply isn't how stderr
is treated generally and meant you couldn't use certain
logging techniques and libraries, among other things.
* Do make sure that stderr writes in the program end up going to
stderr in the Pulumi CLI output, however, so that redirection
works as it should. This required a new Infoerr log level.
* Make a small fix to the planning logic so we don't attempt to
print the summary if an error occurs.
* Finally, add a new error type, RunError, that when thrown and
uncaught does not result in a full stack trace being printed.
Anyone can use this, however, we currently use it for config
errors so that we can terminate with a pretty error message,
rather than the monstrosity shown in pulumi/pulumi#359.
2017-09-23 12:20:11 +00:00
|
|
|
}
|
Fix #3982, missing output which lacks newlines (#8671)
* Fix #3982, missing output which lacks newlines
If the last line printed to stdout or stderr was missing a
terminating newline, it would go entirely missing (in all languages).
The reason for this is a bug in the engine's handling of plugin
outputs: Go's Reader.ReadString('\n') returns a string containing what
was read and/or an error; if the string terminated in a '\n', the
error is nil, and the entire line is returned; if the stream ends,
however, a non-nil error is returned *and* what was read is returned,
even though it wasn't terminated in a newline. The fix is simple:
instead of ignoring that text, we use it, and *then* exit the read-loop.
Also added some test cases since this is subtle and easy to regress.
* Add a changelog entry
2022-01-03 22:39:10 +00:00
|
|
|
|
|
|
|
// If we've hit the end of the stream, break out and close the channel.
|
|
|
|
if readerr != nil {
|
|
|
|
break
|
|
|
|
}
|
2017-04-19 22:01:04 +00:00
|
|
|
}
|
2018-04-13 22:44:35 +00:00
|
|
|
|
2017-10-16 20:44:37 +00:00
|
|
|
close(done)
|
2017-04-19 22:01:04 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Set up a tracer on stderr before going any further, since important errors might get communicated this way.
|
2017-10-17 03:51:18 +00:00
|
|
|
stderrDone := make(chan bool)
|
|
|
|
plug.stderrDone = stderrDone
|
2017-10-16 20:44:37 +00:00
|
|
|
go runtrace(plug.Stderr, true, stderrDone)
|
2017-04-19 22:01:04 +00:00
|
|
|
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
// Now that we have a process, we expect it to write a single line to STDOUT: the port it's listening on. We only
|
|
|
|
// read a byte at a time so that STDOUT contains everything after the first newline.
|
2022-11-01 15:15:09 +00:00
|
|
|
var portString string
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
b := make([]byte, 1)
|
|
|
|
for {
|
2017-06-09 19:51:31 +00:00
|
|
|
n, readerr := plug.Stdout.Read(b)
|
|
|
|
if readerr != nil {
|
2022-04-12 14:32:54 +00:00
|
|
|
killerr := plug.Kill()
|
2019-10-09 20:51:10 +00:00
|
|
|
contract.IgnoreError(killerr) // We are ignoring because the readerr trumps it.
|
|
|
|
|
|
|
|
// If from the output we have seen, return a specific error if possible.
|
|
|
|
if sawPolicyModuleNotFoundErr {
|
|
|
|
return nil, errRunPolicyModuleNotFound
|
|
|
|
}
|
|
|
|
|
|
|
|
// Fall back to a generic, opaque error.
|
2022-11-01 15:15:09 +00:00
|
|
|
if portString == "" {
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("could not read plugin [%v] stdout: %w", bin, readerr)
|
2017-04-19 22:01:04 +00:00
|
|
|
}
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("failure reading plugin [%v] stdout (read '%v'): %w",
|
|
|
|
bin, portString, readerr)
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
|
|
|
if n > 0 && b[0] == '\n' {
|
|
|
|
break
|
|
|
|
}
|
2022-11-01 15:15:09 +00:00
|
|
|
portString += string(b[:n])
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
2023-01-19 10:45:16 +00:00
|
|
|
// Trim any whitespace from the first line (this is to handle things like windows that will write
|
|
|
|
// "1234\r\n", or slightly odd providers that might add whitespace like "1234 ")
|
|
|
|
portString = strings.TrimSpace(portString)
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
|
|
|
|
// Parse the output line (minus the '\n') to ensure it's a numeric port.
|
2022-11-01 15:15:09 +00:00
|
|
|
var port int
|
|
|
|
if port, err = strconv.Atoi(portString); err != nil {
|
2022-04-12 14:32:54 +00:00
|
|
|
killerr := plug.Kill()
|
Tidy up more lint
This change fixes a few things:
* Most importantly, we need to place a leading "." in the paths
to Gometalinter, otherwise some sub-linters just silently skip
the directory altogether. errcheck is one such linter, which
is a very important one!
* Use an explicit Gometalinter.json file to configure the various
settings. This flips on a few additional linters that aren't
on by default (line line length checking). Sadly, a few that
I'd like to enable take waaaay too much time, so in the future
we may consider a nightly job (this includes code similarity,
unused parameters, unused functions, and others that generally
require global analysis).
* Now that we're running more, however, linting takes a while!
The core Lumi project now takes 26 seconds to lint on my laptop.
That's not terrible, but it's long enough that we don't want to
do the silly "run them twice" thing our Makefiles were previously
doing. Instead, we shall deploy some $$($${PIPESTATUS[1]}-1))-fu
to rely on the fact that grep returns 1 on "zero lines".
* Finally, fix the many issues that this turned up.
I think(?) we are done, except, of course, for needing to drive
down some of the cyclomatic complexity issues (which I'm possibly
going to punt on; see pulumi/lumi#259 for more details).
2017-06-22 19:09:46 +00:00
|
|
|
contract.IgnoreError(killerr) // ignoring the error because the existing one trumps it.
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf(
|
|
|
|
"%v plugin [%v] wrote a non-numeric port to stdout ('%v'): %w", prefix, bin, port, err)
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
|
|
|
|
2017-04-19 22:01:04 +00:00
|
|
|
// After reading the port number, set up a tracer on stdout just so other output doesn't disappear.
|
2017-10-17 03:51:18 +00:00
|
|
|
stdoutDone := make(chan bool)
|
|
|
|
plug.stdoutDone = stdoutDone
|
2017-10-16 20:44:37 +00:00
|
|
|
go runtrace(plug.Stdout, false, stdoutDone)
|
2017-02-20 20:34:15 +00:00
|
|
|
|
2022-11-01 15:15:09 +00:00
|
|
|
conn, err := dialPlugin(port, bin, prefix, dialOptions)
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
if err != nil {
|
2022-04-19 11:41:18 +00:00
|
|
|
return nil, err
|
2017-09-08 22:11:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Done; store the connection and return the plugin info.
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
plug.Conn = conn
|
|
|
|
return plug, nil
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
|
|
|
|
2020-02-28 20:48:53 +00:00
|
|
|
// execPlugin starts the plugin executable.
|
2024-04-25 17:30:30 +00:00
|
|
|
func execPlugin(ctx *Context, bin, prefix string, kind apitype.PluginKind,
|
2023-03-03 16:36:39 +00:00
|
|
|
pluginArgs []string, pwd string, env []string,
|
|
|
|
) (*plugin, error) {
|
2021-07-27 16:42:52 +00:00
|
|
|
args := buildPluginArguments(pluginArgumentOptions{
|
|
|
|
pluginArgs: pluginArgs,
|
|
|
|
tracingEndpoint: cmdutil.TracingEndpoint,
|
|
|
|
logFlow: logging.LogFlow,
|
|
|
|
logToStderr: logging.LogToStderr,
|
|
|
|
verbose: logging.Verbose,
|
|
|
|
})
|
2022-10-04 08:58:01 +00:00
|
|
|
|
|
|
|
// Check to see if we have a binary we can invoke directly
|
|
|
|
if _, err := os.Stat(bin); os.IsNotExist(err) {
|
|
|
|
// If we don't have the expected binary, see if we have a "PulumiPlugin.yaml" or "PulumiPolicy.yaml"
|
|
|
|
pluginDir := filepath.Dir(bin)
|
|
|
|
|
|
|
|
var runtimeInfo workspace.ProjectRuntimeInfo
|
2024-04-25 17:30:30 +00:00
|
|
|
if kind == apitype.ResourcePlugin || kind == apitype.ConverterPlugin {
|
2022-10-04 08:58:01 +00:00
|
|
|
proj, err := workspace.LoadPluginProject(filepath.Join(pluginDir, "PulumiPlugin.yaml"))
|
|
|
|
if err != nil {
|
|
|
|
return nil, fmt.Errorf("loading PulumiPlugin.yaml: %w", err)
|
|
|
|
}
|
|
|
|
runtimeInfo = proj.Runtime
|
2024-04-25 17:30:30 +00:00
|
|
|
} else if kind == apitype.AnalyzerPlugin {
|
2022-10-04 08:58:01 +00:00
|
|
|
proj, err := workspace.LoadPluginProject(filepath.Join(pluginDir, "PulumiPolicy.yaml"))
|
|
|
|
if err != nil {
|
|
|
|
return nil, fmt.Errorf("loading PulumiPolicy.yaml: %w", err)
|
|
|
|
}
|
|
|
|
runtimeInfo = proj.Runtime
|
|
|
|
} else {
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, errors.New("language plugins must be executable binaries")
|
2022-10-04 08:58:01 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
logging.V(9).Infof("Launching plugin '%v' from '%v' via runtime '%s'", prefix, pluginDir, runtimeInfo.Name())
|
|
|
|
|
2024-07-16 10:55:38 +00:00
|
|
|
// ProgramInfo needs pluginDir to be an absolute path
|
|
|
|
pluginDir, err = filepath.Abs(pluginDir)
|
|
|
|
if err != nil {
|
|
|
|
return nil, fmt.Errorf("getting absolute path for plugin directory: %w", err)
|
|
|
|
}
|
|
|
|
|
2024-01-25 23:28:58 +00:00
|
|
|
info := NewProgramInfo(pluginDir, pluginDir, ".", runtimeInfo.Options())
|
|
|
|
runtime, err := ctx.Host.LanguageRuntime(runtimeInfo.Name(), info)
|
2022-10-04 08:58:01 +00:00
|
|
|
if err != nil {
|
2023-12-12 12:19:42 +00:00
|
|
|
return nil, fmt.Errorf("loading runtime: %w", err)
|
2022-10-04 08:58:01 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
stdout, stderr, kill, err := runtime.RunPlugin(RunPluginInfo{
|
2024-01-25 23:28:58 +00:00
|
|
|
Info: info,
|
|
|
|
WorkingDirectory: ctx.Pwd,
|
|
|
|
Args: pluginArgs,
|
|
|
|
Env: env,
|
2022-10-04 08:58:01 +00:00
|
|
|
})
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
return &plugin{
|
|
|
|
Bin: bin,
|
|
|
|
Args: args,
|
|
|
|
Env: env,
|
|
|
|
Kill: func() error { kill(); return nil },
|
|
|
|
Stdout: io.NopCloser(stdout),
|
|
|
|
Stderr: io.NopCloser(stderr),
|
|
|
|
}, nil
|
|
|
|
}
|
|
|
|
|
2017-05-30 17:19:33 +00:00
|
|
|
cmd := exec.Command(bin, args...)
|
2018-02-14 21:56:07 +00:00
|
|
|
cmdutil.RegisterProcessGroup(cmd)
|
2017-12-12 20:31:09 +00:00
|
|
|
cmd.Dir = pwd
|
2019-12-16 22:51:02 +00:00
|
|
|
if len(env) > 0 {
|
|
|
|
cmd.Env = env
|
|
|
|
}
|
2023-08-31 16:29:55 +00:00
|
|
|
in, _ := cmd.StdinPipe()
|
|
|
|
out, _ := cmd.StdoutPipe()
|
|
|
|
err, _ := cmd.StderrPipe()
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
if err := cmd.Start(); err != nil {
|
2020-02-28 20:48:53 +00:00
|
|
|
// If we try to run a plugin that isn't found, intercept the error
|
|
|
|
// and instead return a custom one so we can more easily check for
|
|
|
|
// it upstream
|
|
|
|
//
|
|
|
|
// In the case of PAC, note that the plugin usually _does_ exist.
|
|
|
|
// It is a shell script like "pulumi-analyzer-policy". But during
|
|
|
|
// the execution of that script, it fails with the ENOENT error.
|
|
|
|
if pathErr, ok := err.(*os.PathError); ok {
|
|
|
|
syscallErr, ok := pathErr.Err.(syscall.Errno)
|
|
|
|
if ok && syscallErr == syscall.ENOENT {
|
|
|
|
return nil, errPluginNotFound
|
|
|
|
}
|
|
|
|
|
|
|
|
}
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
return nil, err
|
|
|
|
}
|
2017-10-30 22:55:40 +00:00
|
|
|
|
2022-04-12 14:32:54 +00:00
|
|
|
kill := func() error {
|
2023-08-31 16:29:55 +00:00
|
|
|
var result *multierror.Error
|
|
|
|
|
|
|
|
// On each platform, plugins are not loaded directly, instead a shell launches each plugin as a child process, so
|
|
|
|
// instead we need to kill all the children of the PID we have recorded, as well. Otherwise we will block waiting
|
|
|
|
// for the child processes to close.
|
|
|
|
if err := cmdutil.KillChildren(cmd.Process.Pid); err != nil {
|
|
|
|
result = multierror.Append(result, err)
|
|
|
|
}
|
|
|
|
|
|
|
|
// IDEA: consider a more graceful termination than just SIGKILL.
|
|
|
|
if err := cmd.Process.Kill(); err != nil {
|
|
|
|
result = multierror.Append(result, err)
|
|
|
|
}
|
|
|
|
|
|
|
|
return result.ErrorOrNil()
|
2022-04-12 14:32:54 +00:00
|
|
|
}
|
|
|
|
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
return &plugin{
|
2017-12-01 21:50:32 +00:00
|
|
|
Bin: bin,
|
|
|
|
Args: args,
|
2019-12-16 22:51:02 +00:00
|
|
|
Env: env,
|
2022-04-12 14:32:54 +00:00
|
|
|
Kill: kill,
|
2023-08-31 16:29:55 +00:00
|
|
|
Stdin: in,
|
|
|
|
Stdout: out,
|
|
|
|
Stderr: err,
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
}, nil
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|
|
|
|
|
2021-07-27 16:42:52 +00:00
|
|
|
type pluginArgumentOptions struct {
|
2023-01-13 21:24:48 +00:00
|
|
|
pluginArgs []string
|
|
|
|
tracingEndpoint string
|
|
|
|
logFlow, logToStderr bool
|
|
|
|
verbose int
|
2021-07-27 16:42:52 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
func buildPluginArguments(opts pluginArgumentOptions) []string {
|
|
|
|
var args []string
|
|
|
|
// Flow the logging information if set.
|
|
|
|
if opts.logFlow {
|
|
|
|
if opts.logToStderr {
|
|
|
|
args = append(args, "--logtostderr")
|
|
|
|
}
|
|
|
|
if opts.verbose > 0 {
|
|
|
|
args = append(args, "-v="+strconv.Itoa(opts.verbose))
|
|
|
|
}
|
|
|
|
}
|
2022-07-01 20:58:00 +00:00
|
|
|
if opts.tracingEndpoint != "" {
|
2021-07-27 16:42:52 +00:00
|
|
|
args = append(args, "--tracing", opts.tracingEndpoint)
|
|
|
|
}
|
|
|
|
args = append(args, opts.pluginArgs...)
|
|
|
|
return args
|
|
|
|
}
|
|
|
|
|
Add basic analyzer support
This change introduces the basic requirements for analyzers, as per
pulumi/coconut#119. In particular, an analyzer can implement either,
or both, of the RPC methods, Analyze and AnalyzeResource. The former
is meant to check an overall deployment (e.g., to ensure it has been
signed off on) and the latter is to check individual resources (e.g.,
to ensure properties of them are correct, such as checking style,
security, etc. rules). These run simultaneous to overall checking.
Analyzers are loaded as plugins just like providers are. The difference
is mainly in their naming ("analyzer-" prefix, rather than "resource-"),
and the RPC methods that they support.
This isn't 100% functional since we need a way to specify at the CLI
that a particular analyzer should be run, in addition to a way of
recording which analyzers certain projects should use in their manifests.
2017-03-11 07:49:17 +00:00
|
|
|
func (p *plugin) Close() error {
|
2017-10-17 03:51:18 +00:00
|
|
|
if p.Conn != nil {
|
2019-10-01 19:12:06 +00:00
|
|
|
contract.IgnoreClose(p.Conn)
|
2017-10-17 03:51:18 +00:00
|
|
|
}
|
2017-10-16 20:44:37 +00:00
|
|
|
|
2022-04-12 14:32:54 +00:00
|
|
|
result := p.Kill()
|
2017-10-16 20:44:37 +00:00
|
|
|
|
2022-04-19 11:41:18 +00:00
|
|
|
// Wait for stdout and stderr to drain if we attached to the plugin we won't have a stdout/err
|
2018-02-19 22:06:15 +00:00
|
|
|
if p.stdoutDone != nil {
|
|
|
|
<-p.stdoutDone
|
|
|
|
}
|
|
|
|
if p.stderrDone != nil {
|
|
|
|
<-p.stderrDone
|
|
|
|
}
|
2017-10-16 20:44:37 +00:00
|
|
|
|
2017-10-31 00:35:36 +00:00
|
|
|
return result
|
Implement resource provider plugins
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 19:08:06 +00:00
|
|
|
}
|