README
¶
Claw
Claw's provides an easy alternative to Protocol Buffers that is free, performant and easy to use.
The whys for using it:
- More performant for applications needing low allocations
- Language generation is simple and easy to use, unlike protoc
- You can decode messages without the IDL (which only means no field names)
The why nots:
- Proto has better language support
- Proto is supported by Google (though they break it all the time)
- It is likely bigger on the wire on the average case
Goals:
- Be friendly to the machine at the cost of wire size
- Zero heap allocations for data conversion to native types (for Go, Rust and C++)
- Close to zero heap allocations for decode
- One pass encode/decode
- Supportable in Go, Rust, C++, Javascript
- Import statements that make sense instead of that protoc nonsense
- Allow JSON serialization on top of our binary serialization
- The binary format should be fairly easy to understand if you understand these types of binary file formats
- Offer a file level option, NoZeroValueCompression, that will not cause any value set to the zero value to have its header encoded. This allows using an IsSet() to determine if the value is set or is just the zero value.
Non-goals:
- Human readable output
- Small wire size (we want speed for the cost of size)
The wire size choice is probably the one to be the most contentious. It is my belief that variable length encoding buys little in terms of wire savings. I believe that even simple compression like zippy will eliminate any file size issue and will be substantially faster than variable length encoding (especially since you are likely to do both for protocol buffers). In addition, with something like gzip, I imagine the numbers will almost be indistiguishable.
But overall this format will be more in size. All entries are a minimum for 8 bytes (64 bits). In some things like lists of bools, we bit pack with great efficiency. But other things like 64 bit numbers, we use 16 bytes. Lists of numbers are more efficient. This pushes our size closer to JSON sizes, but we compress better and don't have costly field names to decode. We also always know the size of everything ahead of our reads, so conversion is greatly simplified.
Why not Protocol Buffers:
- Protoc - really you don't need much more than that
- Proto2/Proto3 both have features I want and don't want, not a combination of both
- Proto2 - detect nil vs zero values, but I get pointers everywhere
- Proto3 - have zero values only except for Message types
- Allocations - proto has to allocate more often, which eventually bogs down GC languages
You might say, why not use buf.build tooling for your protoc problems. I think buf is great, but using it in any substantial way requires me to pay. I don't want to figure out how to get my company to pay for their service (it is really easier to roll my own than to navigate what I'm sure would be a bunch of nonsense). You can hack on buf cli tools to let you get around the pay model, but I feel icky about that.
If you need rock solid support for every language and based on an IDL that has been around for years + can figure out how to get your company to pay for it, buf is the way to go.
SERIOUSLY, if you need support of every language and need some rock solid support, go check out buf.build. Combine their tools with their connect.build and you've got yourself some good stuff there.
Finally, I wanted zero allocations for reads and proto isn't going to provide me that (though Google's internal versions are much better in this regards). Proto is just built on too much legacy stuff and is kind of a mess between proto versions 1 - 3.
Why not Capt'n Proto:
Look, that guy is a genius. But the format is hard to understand (I don't know how many times I read his format document before I "got" it). The RPC mechanism is nuts. Only he has been able to write an implementation and just for C++ that supports I think more than level 2 support. Every other language that does simply is loading the C++. And the Go maintainer has left the building the last time I checked.
I don't want to write every implementation of this, so the format has to be easy to understand (at least for people who know how to write binary encoding formats) and the eventual RPC system has to be fairly easy to implement and work with tooling.
Why not Flat Buffers:
If you make a Message 3 levels deep with some lists, you will instantly know why. Its just hard to use. Also the Go version doesn't do bounds checks and the code panics on any error.
Why not Thrift:
I just didn't like the syntax and how it layed out. I've also never met anyone going to Thrift or pushing Thrift. That doesn't say much for it.
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
binary
Package binary replaces the encoding/binary package in the standard library for little endian encoding using generics.
|
Package binary replaces the encoding/binary package in the standard library for little endian encoding using generics. |
|
conversions
Package conversions is a set of unsafe conversions from one type to another.
|
Package conversions is a set of unsafe conversions from one type to another. |
|
imports
Package imports provides handling of claw.mod, local.replace and global.replace files.
|
Package imports provides handling of claw.mod, local.replace and global.replace files. |
|
render
Package render sets up the interface for rendering a .claw file to a language native representation.
|
Package render sets up the interface for rendering a .claw file to a language native representation. |
|
render/golang
Package golang implements the Go language renderer.
|
Package golang implements the Go language renderer. |
|
writer
Package writer contains interfaces that can be implemented to render files for a language implementation and a type that can be used to call those implementations and write out those files for all languages that were asked to be rendered.
|
Package writer contains interfaces that can be implemented to render files for a language implementation and a type that can be used to call those implementations and write out those files for all languages that were asked to be rendered. |
|
languages
|
|
|
go/field
Package field details field types used by the Claw format.
|
Package field details field types used by the Claw format. |
|
go/internal/pragma
Package pragma provides types that can be embedded into a struct to statically enforce or prevent certain language properties.
|
Package pragma provides types that can be embedded into a struct to statically enforce or prevent certain language properties. |
|
go/json
Package JSON provides for encoding and decoding Claw Structs into JSON.
|
Package JSON provides for encoding and decoding Claw Structs into JSON. |
|
go/mapping
Package mapping holds metadata mapping information used to map Struct field numbers to descriptions of the fields so that they can be encoded/decoded properly.
|
Package mapping holds metadata mapping information used to map Struct field numbers to descriptions of the fields so that they can be encoded/decoded properly. |
|
go/reflect/enums
Package enums is for internal use only.
|
Package enums is for internal use only. |
|
go/reflect/runtime
Package runtime provides runtime helpers for the Claw file.
|
Package runtime provides runtime helpers for the Claw file. |
|
go/structs
Package structs contains objects, functions and methods that are related to reading and writing Claw struct types from wire encoding.
|
Package structs contains objects, functions and methods that are related to reading and writing Claw struct types from wire encoding. |
|
go/types/list
Package list contains types that implement lists of scalar values that can be store in Claw Struct fields.
|
Package list contains types that implement lists of scalar values that can be store in Claw Struct fields. |
|
testing
|
|
|
imports/vehicles/claw
Package vehicles
|
Package vehicles |
|
imports/vehicles/claw/manufacturers
Package manufacturers
|
Package manufacturers |