FlatBuffers.jl Documentation
Overview
FlatBuffers.jl provides native Julia support for reading and writing binary structures following the google flatbuffer schema (see here for a more in-depth review of the binary format).
The typical language support for flatbuffers involves utilizing the flatc
compiler to translate a flatbuffer schema file (.fbs) into a langugage-specific set of types/classes and methods. See here for the official guide on writing schemas.
This Julia package provides the serialization primitives used by code that has been generated by flatc
. Since it was originally built without flatc
support, it can also be used as a minimal set of macros to provide flatbuffer-compatible serialization of existing Julia types. This has led to the Julia code generated by flatc
appearing somewhat more readable than for other languages.
For example, for this schema:
namespace example;
table SimpleType {
x: int = 1;
}
root_type SimpleType;
the code generated by flatc
looks like this:
module Example
using FlatBuffers
@with_kw mutable struct SimpleType
x::Int32 = 1
end
# ... other generated stuff
end
If you don't want to write a schema, you can pepper your existing Julia types with these macros and then call the functions below to produce flatbuffer-compatible binaries.
Usage
FlatBuffers
provides the following functions for reading and writing flatbuffers:
FlatBuffers.serialize(stream::IO, value::T)
FlatBuffers.deserialize(stream::IO, ::Type{T})
These methods are not exported to avoid naming clashes with the Serialization
module. For convenience, there are also two additional constructors defined for each generated type:
T(buf::AbstractVector{UInt8}, pos::Integer=0)
T(io::IO)
Here is an example showing how to use them to serialize the example type above.
import FlatBuffers, Example
# create an instance of our type
val = Example.SimpleType(2)
# serialize it to example.bin
open("example.bin", "w") do f FlatBuffers.serialize(f, val) end
# read the value back again from file
val2 = open("example.bin", "r") do f Example.SimpleType(f) end
In addition, this package provides the following types and methods, which are useful when inspecting and constructing flatbuffers:
FlatBuffers.Table{T}
- type for deserializing a Julia typeT
from a flatbufferFlatBuffers.Builder{T}
- type for serializing a Julia typeT
to a flatbufferFlatBuffers.read
- performs the actual deserializing on aFlatBuffer.Table
FlatBuffers.build!
- performs the actual serializing on aFlatBuffer.Builder
Methods for Generated Types
For a generated type T
, in addition to the constructors mentioned above:
- if
T
has default values, constructors will be defined as per the@with_kw
macro in Parameters.jl FlatBuffers.file_extension(T)
- returns thefile_extension
specified in the schema (if any)FlatBuffers.file_identifier(T)
- returns thefile_identifier
specified in the schema (if any)FlatBuffers.has_identifier(T, bytes)
- returns whether the given bytes contain the identifier forT
at the offset designated by the flatbuffers specificationFlatBuffers.slot_offsets(T)
- an array containing the positions of the slots in the vtable for typeT
, accounting for gaps caused by deprecated fieldsFlatBuffers.root_type(T)
- returns whether the type is designated as the root type by the schema. Also note however that noroot_type
definition is necessary in Julia; any of the generatedmutable struct
s can be a valid root table type.
Circular References
It's a bit unfortunate that the flatbuffers example uses mutually referential types, something which Julia doesn't have support for yet. However, there is a workaround - by modifying the code generated by flatc
slightly to add a type parameter, we can refer to a type that hasn't yet been defined.
FlatBuffers.@with_kw mutable struct Monster{T}
# ...
test::T = nothing
# ...
end
In general though, try to avoid schemas which introduce these kinds of circular references. For the full Monster
example see the test suite here.
Internal Utilities
These functions are used by the code generated by flatc
. Documentation is also included for many internal methods and may be queried using ?
at the REPL.
@ALIGN T size_in_bytes
- convenience macro for forcing a flatbuffer alignment on the Julia typeT
tosize_in_bytes
@with_kw mutable struct T fields...
- convenience macro for defining default field values for Julia typeT
@UNION T Union{T1,T2,...}
- convenience macro for defining a flatbuffer union typeT
@STRUCT struct T fields... end
- convenience macro for defining flatbuffer struct types, ensuring any necessary padding gets added to the type definition