2018-05-22 19:43:36 +00:00
|
|
|
// Copyright 2016-2018, Pulumi Corporation.
|
|
|
|
//
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
// you may not use this file except in compliance with the License.
|
|
|
|
// You may obtain a copy of the License at
|
|
|
|
//
|
|
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
//
|
|
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
// See the License for the specific language governing permissions and
|
|
|
|
// limitations under the License.
|
2016-11-16 21:11:58 +00:00
|
|
|
|
Begin overhauling semantic phases
This change further merges the new AST and MuPack/MuIL formats and
abstractions into the core of the compiler. A good amount of the old
code is gone now; I decided against ripping it all out in one fell
swoop so that I can methodically check that we are preserving all
relevant decisions and/or functionality we had in the old model.
The changes are too numerous to outline in this commit message,
however, here are the noteworthy ones:
* Split up the notion of symbols and tokens, resulting in:
- pkg/symbols for true compiler symbols (bound nodes)
- pkg/tokens for name-based tokens, identifiers, constants
* Several packages move underneath pkg/compiler:
- pkg/ast becomes pkg/compiler/ast
- pkg/errors becomes pkg/compiler/errors
- pkg/symbols becomes pkg/compiler/symbols
* pkg/ast/... becomes pkg/compiler/legacy/ast/...
* pkg/pack/ast becomes pkg/compiler/ast.
* pkg/options goes away, merged back into pkg/compiler.
* All binding functionality moves underneath a dedicated
package, pkg/compiler/binder. The legacy.go file contains
cruft that will eventually go away, while the other files
represent a halfway point between new and old, but are
expected to stay roughly in the current shape.
* All parsing functionality is moved underneath a new
pkg/compiler/metadata namespace, and we adopt new terminology
"metadata reading" since real parsing happens in the MetaMu
compilers. Hence, Parser has become metadata.Reader.
* In general phases of the compiler no longer share access to
the actual compiler.Compiler object. Instead, shared state is
moved to the core.Context object underneath pkg/compiler/core.
* Dependency resolution during binding has been rewritten to
the new model, including stashing bound package symbols in the
context object, and detecting import cycles.
* Compiler construction does not take a workspace object. Instead,
creation of a workspace is entirely hidden inside of the compiler's
constructor logic.
* There are three Compile* functions on the Compiler interface, to
support different styles of invoking compilation: Compile() auto-
detects a Mu package, based on the workspace; CompilePath(string)
loads the target as a Mu package and compiles it, regardless of
the workspace settings; and, CompilePackage(*pack.Package) will
compile a pre-loaded package AST, again regardless of workspace.
* Delete the _fe, _sema, and parsetree phases. They are no longer
relevant and the functionality is largely subsumed by the above.
...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 20:18:37 +00:00
|
|
|
package tokens
|
2016-11-16 21:11:58 +00:00
|
|
|
|
|
|
|
import (
|
2016-11-21 20:06:32 +00:00
|
|
|
"regexp"
|
2016-11-16 21:11:58 +00:00
|
|
|
"strings"
|
2016-11-21 20:06:32 +00:00
|
|
|
|
2021-03-17 13:20:05 +00:00
|
|
|
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
|
2016-11-16 21:11:58 +00:00
|
|
|
)
|
|
|
|
|
2022-01-21 14:02:33 +00:00
|
|
|
// Name is an identifier. It conforms to NameRegexpPattern.
|
2017-01-18 01:41:28 +00:00
|
|
|
type Name string
|
|
|
|
|
2017-01-28 18:51:30 +00:00
|
|
|
func (nm Name) String() string { return string(nm) }
|
2016-11-21 20:06:32 +00:00
|
|
|
|
Clean up package URL logic
This change rearranges the old way we dealt with URLs. In the old system,
virtually every reference to an element, including types, was fully qualified
with a possible URL-like reference. (The old pkg/tokens/Ref type.) In the
new model, only dependency references are URL-like. All maps and references
within the MuPack/MuIL format are token and name based, using the new
pkg/tokens/Token and pkg/tokens/Name family of related types.
As such, this change renames Ref to PackageURLString, and RefParts to
PackageURL. (The convenient name is given to the thing with "more" structure,
since we prefer to deal with structured types and not strings.) It moves
out of the pkg/tokens package and into pkg/pack, since it is exclusively
there to support package resolution. Similarly, the Version, VersionSpec,
and related types move out of pkg/tokens and into pkg/pack.
This change cleans up the various binder, package, and workspace logic.
Most of these changes are a natural fallout of this overall restructuring,
although in a few places we remained sloppy about the difference between
Token, Name, and URL. Now the type system supports these distinctions and
forces us to be more methodical about any conversions that take place.
2017-01-20 19:46:36 +00:00
|
|
|
// Q turns a Name into a qualified name; this is legal, since Name's is a proper subset of QName's grammar.
|
|
|
|
func (nm Name) Q() QName { return QName(nm) }
|
|
|
|
|
2023-03-03 16:36:39 +00:00
|
|
|
var (
|
|
|
|
NameRegexp = regexp.MustCompile(NameRegexpPattern)
|
|
|
|
nameFirstCharRegexp = regexp.MustCompile("^" + nameFirstCharRegexpPattern + "$")
|
|
|
|
nameRestCharRegexp = regexp.MustCompile("^" + nameRestCharRegexpPattern + "$")
|
|
|
|
)
|
2021-12-17 22:52:01 +00:00
|
|
|
|
|
|
|
var NameRegexpPattern = nameFirstCharRegexpPattern + nameRestCharRegexpPattern
|
|
|
|
|
2023-03-03 16:36:39 +00:00
|
|
|
const (
|
|
|
|
nameFirstCharRegexpPattern = "[A-Za-z0-9_.-]"
|
|
|
|
nameRestCharRegexpPattern = "[A-Za-z0-9_.-]*"
|
|
|
|
)
|
2017-01-28 18:51:30 +00:00
|
|
|
|
2016-11-21 20:06:32 +00:00
|
|
|
// IsName checks whether a string is a legal Name.
|
|
|
|
func IsName(s string) bool {
|
Revive some compiler tests
This change revives some compiler tests that are still lingering around
from the old architecture, before our latest round of ship burning.
It also fixes up some bugs uncovered during this:
* Don't claim that a symbol's kind is incorrect in the binder error
message when it wasn't found. Instead, say that it was missing.
* Do not attempt to compile if an error was issued during workspace
resolution and/or loading of the Mufile. This leads to trying to
load an empty path and badness quickly ensues (crash).
* Issue an error if the Mufile wasn't found (this got lost apparently).
* Rename the ErrorMissingPackageName message to ErrorInvalidPackageName,
since missing names are now caught by our new fancy decoder that
understands required versus optional fields. We still need to guard
against illegal characters in the name, including the empty string "".
* During decoding, reject !src.IsValid elements. This represents the
zero value and should be treated equivalently to a missing field.
* Do not permit empty strings "" as Names or QNames. The old logic
accidentally permitted them because regexp.FindString("") == "", no
matter the regex!
* Move the TestDiagSink abstraction to a new pkg/util/testutil package,
allowing us to share this common code across multiple package tests.
* Fix up a few messages that needed tidying or to use Infof vs. Info.
The binder tests -- deleted in this -- are about to come back, however,
I am splitting up the changes, since this represents a passing fixed point.
2017-01-26 23:30:08 +00:00
|
|
|
return s != "" && NameRegexp.FindString(s) == s
|
2016-11-21 20:06:32 +00:00
|
|
|
}
|
|
|
|
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
// QName is a qualified identifier. The "/" character optionally delimits different pieces of the name. Each element
|
2022-01-21 14:02:33 +00:00
|
|
|
// conforms to NameRegexpPattern. For example, "pulumi/pulumi/stack".
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
type QName string
|
|
|
|
|
2017-01-28 18:51:30 +00:00
|
|
|
func (nm QName) String() string { return string(nm) }
|
|
|
|
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
// QNameDelimiter is what delimits Namespace and Name parts.
|
|
|
|
const QNameDelimiter = "/"
|
|
|
|
|
2023-03-03 16:36:39 +00:00
|
|
|
var (
|
|
|
|
QNameRegexp = regexp.MustCompile(QNameRegexpPattern)
|
|
|
|
QNameRegexpPattern = "(" + NameRegexpPattern + "\\" + QNameDelimiter + ")*" + NameRegexpPattern
|
|
|
|
)
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
|
2022-03-11 17:06:18 +00:00
|
|
|
// IsQName checks whether a string is a legal QName.
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
func IsQName(s string) bool {
|
Revive some compiler tests
This change revives some compiler tests that are still lingering around
from the old architecture, before our latest round of ship burning.
It also fixes up some bugs uncovered during this:
* Don't claim that a symbol's kind is incorrect in the binder error
message when it wasn't found. Instead, say that it was missing.
* Do not attempt to compile if an error was issued during workspace
resolution and/or loading of the Mufile. This leads to trying to
load an empty path and badness quickly ensues (crash).
* Issue an error if the Mufile wasn't found (this got lost apparently).
* Rename the ErrorMissingPackageName message to ErrorInvalidPackageName,
since missing names are now caught by our new fancy decoder that
understands required versus optional fields. We still need to guard
against illegal characters in the name, including the empty string "".
* During decoding, reject !src.IsValid elements. This represents the
zero value and should be treated equivalently to a missing field.
* Do not permit empty strings "" as Names or QNames. The old logic
accidentally permitted them because regexp.FindString("") == "", no
matter the regex!
* Move the TestDiagSink abstraction to a new pkg/util/testutil package,
allowing us to share this common code across multiple package tests.
* Fix up a few messages that needed tidying or to use Infof vs. Info.
The binder tests -- deleted in this -- are about to come back, however,
I am splitting up the changes, since this represents a passing fixed point.
2017-01-26 23:30:08 +00:00
|
|
|
return s != "" && QNameRegexp.FindString(s) == s
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
}
|
|
|
|
|
2021-12-17 22:52:01 +00:00
|
|
|
// IntoQName converts an arbitrary string into a QName, converting the string to a valid QName if
|
|
|
|
// necessary. The conversion is deterministic, but also lossy.
|
|
|
|
func IntoQName(s string) QName {
|
|
|
|
output := []string{}
|
|
|
|
for _, s := range strings.Split(s, QNameDelimiter) {
|
|
|
|
if s == "" {
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
segment := []byte(s)
|
|
|
|
if !nameFirstCharRegexp.Match([]byte{segment[0]}) {
|
|
|
|
segment[0] = '_'
|
|
|
|
}
|
|
|
|
for i := 1; i < len(s); i++ {
|
|
|
|
if !nameRestCharRegexp.Match([]byte{segment[i]}) {
|
|
|
|
segment[i] = '_'
|
|
|
|
}
|
|
|
|
}
|
|
|
|
output = append(output, string(segment))
|
|
|
|
}
|
|
|
|
result := strings.Join(output, QNameDelimiter)
|
|
|
|
if result == "" {
|
|
|
|
result = "_"
|
|
|
|
}
|
2022-07-15 13:56:54 +00:00
|
|
|
return QName(result)
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Name extracts the Name portion of a QName (dropping any namespace).
|
|
|
|
func (nm QName) Name() Name {
|
|
|
|
ix := strings.LastIndex(string(nm), QNameDelimiter)
|
|
|
|
var nmn string
|
2016-11-16 21:11:58 +00:00
|
|
|
if ix == -1 {
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
nmn = string(nm)
|
|
|
|
} else {
|
|
|
|
nmn = string(nm[ix+1:])
|
2016-11-16 21:11:58 +00:00
|
|
|
}
|
2023-02-15 01:06:56 +00:00
|
|
|
contract.Assertf(IsName(nmn), "QName %q has invalid name %q", nm, nmn)
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
return Name(nmn)
|
2016-11-16 21:11:58 +00:00
|
|
|
}
|
|
|
|
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
// Namespace extracts the namespace portion of a QName (dropping the name); this may be empty.
|
|
|
|
func (nm QName) Namespace() QName {
|
|
|
|
ix := strings.LastIndex(string(nm), QNameDelimiter)
|
|
|
|
var qn string
|
2016-11-16 21:11:58 +00:00
|
|
|
if ix == -1 {
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
qn = ""
|
|
|
|
} else {
|
|
|
|
qn = string(nm[:ix])
|
2016-11-16 21:11:58 +00:00
|
|
|
}
|
2023-02-15 01:06:56 +00:00
|
|
|
contract.Assertf(IsQName(qn), "QName %q has invalid namespace %q", nm, qn)
|
Overhaul names versus tokens
I was sloppy in my use of names versus tokens in the original AST.
Now that we're actually binding things to concrete symbols, etc., we
need to be more precise. In particular, names are just identifiers
that must be "interpreted" in a given lexical context for them to
make any sense; whereas, tokens stand alone and can be resolved without
context other than the set of imported packages, modules, and overall
module structure. As such, names are much simpler than tokens.
As explained in the comments, tokens.Names are simple identifiers:
Name = [A-Za-z_][A-Za-z0-9_]*
and tokens.QNames are fully qualified identifiers delimited by "/":
QName = [ <Name> "/" ]* <Name>
The legal grammar for a token depends on the subset of symbols that
token is meant to represent. However, the most general case, that
accepts all specializations of tokens, is roughly as follows:
Token = <Name> |
<PackageName>
[ ":" <ModuleName>
[ "/" <ModuleMemberName>
[ "." <Class MemberName> ]
]
]
where:
PackageName = <QName>
ModuleName = <QName>
ModuleMemberName = <Name>
ClassMemberName = <Name>
Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 01:57:20 +00:00
|
|
|
return QName(qn)
|
2016-11-16 21:11:58 +00:00
|
|
|
}
|
2017-04-19 17:53:14 +00:00
|
|
|
|
2022-01-21 19:23:14 +00:00
|
|
|
// PackageName is a qualified name referring to an imported package.
|
|
|
|
type PackageName QName
|
2017-04-19 17:53:14 +00:00
|
|
|
|
|
|
|
func (nm PackageName) String() string { return string(nm) }
|
|
|
|
|
|
|
|
// ModuleName is a qualified name referring to an imported module from a package.
|
|
|
|
type ModuleName QName
|
|
|
|
|
|
|
|
func (nm ModuleName) String() string { return string(nm) }
|
|
|
|
|
|
|
|
// ModuleMemberName is a simple name representing the module member's identifier.
|
|
|
|
type ModuleMemberName Name
|
|
|
|
|
|
|
|
func (nm ModuleMemberName) String() string { return string(nm) }
|
|
|
|
|
|
|
|
// ClassMemberName is a simple name representing the class member's identifier.
|
|
|
|
type ClassMemberName Name
|
|
|
|
|
|
|
|
func (nm ClassMemberName) Name() Name { return Name(nm) }
|
|
|
|
func (nm ClassMemberName) String() string { return string(nm) }
|
|
|
|
|
|
|
|
// TypeName is a simple name representing the type's name, without any package/module qualifiers.
|
|
|
|
type TypeName Name
|
|
|
|
|
|
|
|
func (nm TypeName) String() string { return string(nm) }
|