FlatBuffers.jl Documentation
Overview
FlatBuffers.jl provides native Julia support for reading and writing binary structures following the google flatbuffer schema (see here for a more in-depth review of the binary format).
The typical language support for flatbuffers involves utilizing the flatc
compiler to translate a flatbuffer schema file (.fbs) into a langugage-specific set of types/classes and methods. See here for the official guide on writing schemas.
This Julia package contains serialization primitives and a minimal set of macros which provide flatbuffer-compatible serialization of existing Julia types. This has led to the Julia code appearing somewhat more readable than for other languages.
For example, for this schema:
namespace example;
table SimpleType {
x: int = 1;
}
root_type SimpleType;
the corresponding Julia looks like this:
module Example
using FlatBuffers
@with_kw mutable struct SimpleType
x::Int32 = 1
end
# ... other generated stuff
end
You can pepper your existing Julia types with these macros and then call the functions below to produce flatbuffer-compatible binaries.
Usage
FlatBuffers
provides the following functions for reading and writing flatbuffers:
FlatBuffers.serialize(stream::IO, value::T)
FlatBuffers.deserialize(stream::IO, ::Type{T})
These methods are not exported to avoid naming clashes with the Serialization
module. For convenience, there are also two additional constructors defined for each generated type:
T(buf::AbstractVector{UInt8}, pos::Integer=0)
T(io::IO)
Here is an example showing how to use them to serialize the example type above.
import FlatBuffers, Example
# create an instance of our type
val = Example.SimpleType(2)
# serialize it to example.bin
open("example.bin", "w") do f FlatBuffers.serialize(f, val) end
# read the value back again from file
val2 = open("example.bin", "r") do f
FlatBuffers.deserialize(f, Example.SimpleType)
end
In addition, this package provides the following types and methods, which are useful when inspecting and constructing flatbuffers:
FlatBuffers.Table{T}
- type for deserializing a Julia typeT
from a flatbufferFlatBuffers.Builder{T}
- type for serializing a Julia typeT
to a flatbufferFlatBuffers.read
- performs the actual deserializing on aFlatBuffer.Table
FlatBuffers.build!
- performs the actual serializing on aFlatBuffer.Builder
Additional Methods
For a type T
it is also possible to define:
FlatBuffers.file_extension(T)
- returns thefile_extension
specified in the schema (if any)FlatBuffers.file_identifier(T)
- returns thefile_identifier
specified in the schema (if any)FlatBuffers.has_identifier(T, bytes)
- returns whether the given bytes contain the identifier forT
at the offset designated by the flatbuffers specificationFlatBuffers.slot_offsets(T)
- an array containing the positions of the slots in the vtable for typeT
, accounting for gaps caused by deprecated fieldsFlatBuffers.root_type(T)
- returns whether the type is designated as the root type by the schema. Also note however that noroot_type
definition is necessary in Julia; any of the generatedmutable struct
s can be a valid root table type.
State of flatc integration
It was hoped that support for Julia could be added to flatc
, however this is difficult with the current approach due to https://github.com/JuliaLang/julia/issues/269. This package will require a rethink, or potentially a completely separate package may be created. The most up-to-date fork of flatc
with support for Julia may be found here: https://github.com/rjkat/flatbuffers-julia
Advanced utilities
Documentation is also included for many internal methods and may be queried using ?
at the REPL.
@ALIGN T size_in_bytes
- convenience macro for forcing a flatbuffer alignment on the Julia typeT
tosize_in_bytes
@with_kw mutable struct T fields...
- convenience macro for defining default field values for Julia typeT
@UNION T Union{T1,T2,...}
- convenience macro for defining a flatbuffer union typeT
@STRUCT struct T fields... end
- convenience macro for defining flatbuffer struct types, ensuring any necessary padding gets added to the type definition