Documentation ¶
Overview ¶
Package pbparser is a library for parsing protocol buffer (".proto") files.
It exposes two apis for parsing protocol buffer files. Both the apis return a ProtoFile datastructure and a non-nil Error if there is an issue.
After the parsing operation, this library also validates any references to imported constructs i.e. any references to imported enums, messages etc in the file match the definitions in the imported modules.
API ¶
Clients should invoke the following apis :-
func Parse(r io.Reader, p ImportModuleProvider) (ProtoFile, error)
The Parse() function expects the client code to provide a reader for the protobuf content and also a ImportModuleProvider which can be used to callback the client code for any imports in the protobuf content. If there are no imports, the client can choose to pass this as nil.
func ParseFile(file string) (ProtoFile, error)
The ParseFile() function is a utility function which expects the client code to provide only the path of the protobuf file. If there are any imports in the protobuf file, the parser will look for them in the same directory where the protobuf file resides.
Choosing an API ¶
Clients should use the Parse() function if they are not comfortable with letting the pbparser library access the disk directly. This function should also be preferred if the imports in the protobuf file are accessible to the client code but the client code does not want to give pbparser direct access to them. In such cases, the client code has to construct a ImportModuleProvider instance and pass it to the library. This instance must know how to resolve a given "import" and provide a reader for it.
On the other hand, Clients should use the ParseFile() function if all the imported files as well as the protobuf file are on disk relative to the directory in which the protobuf file resides and they are comfortable with letting the pbparser library access the disk directly.
ProtoFile datastructure ¶
This datastructure represents parsed model of the given protobuf file. It includes the following information :-
type ProtoFile struct { PackageName string // name of the package Syntax string // the protocol buffer syntax Dependencies []string // names of any imports PublicDependencies []string // names of any public imports Options []OptionElement // any package level options Enums []EnumElement // any defined enums Messages []MessageElement // any defined messages Services []ServiceElement // any defined services ExtendDeclarations []ExtendElement // any extends directives }
Each attribute in turn has a defined structure, which is explained in the godoc of the corresponding elements.
Design Considerations ¶
This library consciously chooses to log no information on it's own. Any failures are communicated back to client code via the returned Error.
In case of a parsing error, it returns an Error back to the client with a line and column number in the file on which the parsing error was encountered.
In case of a post-parsing validation error, it returns an Error with enough information to identify the erroneous protobuf construct.
Example (Parse) ¶
Example code for the Parse() API
package main import ( "fmt" "io" "io/ioutil" "os" "path/filepath" "strings" "github.com/tallstoat/pbparser" ) // Example code for the Parse() API func main() { // read the proto file contents from disk & create a reader file := "./examples/mathservice.proto" raw, err := ioutil.ReadFile(file) if err != nil { fmt.Printf("Unable to read proto file: %v \n", err) os.Exit(-1) } r := strings.NewReader(string(raw[:])) // implement a dir based import module provider which reads // import modules from the same dir as the original proto file dir := filepath.Dir(file) pr := DirBasedImportModuleProvider{dir: dir} // invoke Parse() API to parse the file pf, err := pbparser.Parse(r, &pr) if err != nil { fmt.Printf("Unable to parse proto file: %v \n", err) os.Exit(-1) } // print attributes of the returned datastructure fmt.Printf("PackageName: %v, Syntax: %v\n", pf.PackageName, pf.Syntax) } // DirBasedImportModuleProvider is a import module provider which looks for import // modules in the dir that it was initialized with. type DirBasedImportModuleProvider struct { dir string } func (pi *DirBasedImportModuleProvider) Provide(module string) (io.Reader, error) { modulePath := pi.dir + string(filepath.Separator) + module // read the module file contents from dir & create a reader... raw, err := ioutil.ReadFile(modulePath) if err != nil { return nil, err } return strings.NewReader(string(raw[:])), nil }
Output:
Example (ParseFile) ¶
Example code for the ParseFile() API
package main import ( "fmt" "os" "github.com/tallstoat/pbparser" ) func main() { file := "./examples/mathservice.proto" // invoke ParseFile() API to parse the file pf, err := pbparser.ParseFile(file) if err != nil { fmt.Printf("Unable to parse proto file: %v \n", err) os.Exit(-1) } // print attributes of the returned datastructure fmt.Printf("PackageName: %v, Syntax: %v\n", pf.PackageName, pf.Syntax) }
Output:
Index ¶
- type DataType
- type DataTypeCategory
- type EnumConstantElement
- type EnumElement
- type ExtendElement
- type ExtensionsElement
- type FieldElement
- type ImportModuleProvider
- type MapDataType
- type MessageElement
- type NamedDataType
- type OneOfElement
- type OptionElement
- type ProtoFile
- type RPCElement
- type ReservedRangeElement
- type ScalarDataType
- type ScalarType
- type ServiceElement
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DataType ¶
type DataType interface { Name() string Category() DataTypeCategory }
DataType is the interface which must be implemented by the field datatypes. Name() returns the name of the datatype and Category() returns the category of the datatype.
type DataTypeCategory ¶
type DataTypeCategory int
DataTypeCategory is an enumeration which represents the possible kinds of field datatypes in message, oneof and extend declaration constructs.
const ( ScalarDataTypeCategory DataTypeCategory = iota MapDataTypeCategory NamedDataTypeCategory )
type EnumConstantElement ¶
type EnumConstantElement struct { Name string Documentation string Options []OptionElement Tag int }
EnumConstantElement is a datastructure which models the fields within an enum construct. Enum constants can also have inline options specified.
type EnumElement ¶
type EnumElement struct { Name string QualifiedName string Documentation string Options []OptionElement EnumConstants []EnumConstantElement }
EnumElement is a datastructure which models the enum construct in a protobuf file. Enums are defined standalone or as nested entities within messages.
type ExtendElement ¶
type ExtendElement struct { Name string QualifiedName string Documentation string Fields []FieldElement }
ExtendElement is a datastructure which models the extend construct in a protobuf file which is used to add new fields to a previously declared message type.
type ExtensionsElement ¶
ExtensionsElement is a datastructure which models an extensions construct in a protobuf file. An extension is a placeholder for a field whose type is not defined by the original .proto file. This allows other .proto files to add to the original message definition by defining field ranges which can be used for extensions.
type FieldElement ¶
type FieldElement struct { Name string Documentation string Options []OptionElement Label string /* optional, required, repeated, oneof */ Type DataType Tag int }
FieldElement is a datastructure which models a field of a message, a field of a oneof element or an entry in the extend declaration in a protobuf file.
type ImportModuleProvider ¶
ImportModuleProvider is the interface which given a protobuf import module returns a reader for it.
The import module could be on disk or elsewhere. In order for the pbparser library to not be tied in to a specific method of reading the import modules, it exposes this interface to the clients. The clients must provide a implementation of this interface which knows how to interpret the module string & returns a reader for the module. This is needed if the client is calling the Parse() function of the pbparser library.
If the client knows the import modules are on disk, they can instead call the ParseFile() function which internally creates a default import module reader which performs disk access to load the contents of the dependency modules.
type MapDataType ¶
type MapDataType struct {
// contains filtered or unexported fields
}
MapDataType is a construct which represents a protobuf map datatype.
func (MapDataType) Category ¶
func (mdt MapDataType) Category() DataTypeCategory
Category function implementation of interface DataType for MapDataType
func (MapDataType) Name ¶
func (mdt MapDataType) Name() string
Name function implementation of interface DataType for MapDataType
type MessageElement ¶
type MessageElement struct { Name string QualifiedName string Documentation string Options []OptionElement Fields []FieldElement Enums []EnumElement Messages []MessageElement OneOfs []OneOfElement ExtendDeclarations []ExtendElement Extensions []ExtensionsElement ReservedRanges []ReservedRangeElement ReservedNames []string }
MessageElement is a datastructure which models the message construct in a protobuf file.
type NamedDataType ¶
type NamedDataType struct {
// contains filtered or unexported fields
}
NamedDataType is a construct which represents a message datatype as a RPC request or response and a message/enum datatype as a field in message, oneof or extend declarations.
func (NamedDataType) Category ¶
func (ndt NamedDataType) Category() DataTypeCategory
Category function implementation of interface DataType for NamedDataType
func (NamedDataType) IsStream ¶
func (ndt NamedDataType) IsStream() bool
IsStream returns true if the NamedDataType is being used in a rpc as a request or response and is preceded by a Stream keyword.
func (NamedDataType) Name ¶
func (ndt NamedDataType) Name() string
Name function implementation of interface DataType for NamedDataType
type OneOfElement ¶
type OneOfElement struct { Name string Documentation string Options []OptionElement Fields []FieldElement }
OneOfElement is a datastructure which models a oneoff construct in a protobuf file. All the fields in a oneof construct share memory, and at most one field can be set at any time.
type OptionElement ¶
OptionElement is a datastructure which models the option construct in a protobuf file. Option constructs exist at various levels/contexts like file, message etc.
type ProtoFile ¶
type ProtoFile struct { PackageName string Syntax string Dependencies []string PublicDependencies []string Options []OptionElement Enums []EnumElement Messages []MessageElement Services []ServiceElement ExtendDeclarations []ExtendElement }
ProtoFile is a datastructure which represents the parsed model of the given protobuf file.
It includes the package name, the syntax, the import dependencies, any public import dependencies, any options, enums, messages, services, extension declarations etc.
This is populated by the parser & post-validation returned to the client code.
func Parse ¶
func Parse(r io.Reader, p ImportModuleProvider) (ProtoFile, error)
Parse function parses the protobuf content passed to it by the the client code via the reader. It also uses the passed-in ImportModuleProvider to callback the client code for any imports in the protobuf content. If there are no imports, the client can choose to pass this as nil.
This function returns populated ProtoFile struct if parsing is successful. If the parsing or validation fails, it returns an Error.
func ParseFile ¶
ParseFile function reads and parses the content of the protobuf file whose path is provided as sole argument to the function. If there are any imports in the protobuf file, the parser will look for them in the same directory where the protobuf file resides.
This function returns populated ProtoFile struct if parsing is successful. If the parsing or validation fails, it returns an Error.
type RPCElement ¶
type RPCElement struct { Name string Documentation string Options []OptionElement RequestType NamedDataType ResponseType NamedDataType }
RPCElement is a datastructure which models the rpc construct in a protobuf file. RPCs are defined nested within ServiceElements.
type ReservedRangeElement ¶
ReservedRangeElement is a datastructure which models a reserved construct in a protobuf message.
type ScalarDataType ¶
type ScalarDataType struct {
// contains filtered or unexported fields
}
ScalarDataType is a construct which represents all supported protobuf scalar datatypes.
func NewScalarDataType ¶
func NewScalarDataType(s string) (ScalarDataType, error)
NewScalarDataType creates and returns a new ScalarDataType for the given string. If a scalar data type mapping does not exist for the given string, an Error is returned.
func (ScalarDataType) Category ¶
func (sdt ScalarDataType) Category() DataTypeCategory
Category function implementation of interface DataType for ScalarDataType
func (ScalarDataType) Name ¶
func (sdt ScalarDataType) Name() string
Name function implementation of interface DataType for ScalarDataType
type ScalarType ¶
type ScalarType int
ScalarType is an enumeration which represents all known supported scalar field datatypes.
const ( AnyScalar ScalarType = iota + 1 BoolScalar BytesScalar DoubleScalar FloatScalar Fixed32Scalar Fixed64Scalar Int32Scalar Int64Scalar Sfixed32Scalar Sfixed64Scalar Sint32Scalar Sint64Scalar StringScalar Uint32Scalar Uint64Scalar )
type ServiceElement ¶
type ServiceElement struct { Name string QualifiedName string Documentation string Options []OptionElement RPCs []RPCElement }
ServiceElement is a datastructure which models the service construct in a protobuf file. Service construct defines the rpcs (apis) for the service.