GitHub

Parquet

Parquet support is provided by a package extension. It activates automatically once Parquet2 is loaded:

import Pkg; Pkg.add("Parquet2")
import Parquet2    # activates SerdeParquetExt

Parsing

Serde.SerdeParquet.parse_parquet — Function
parse_parquet

Decode Parquet bytes, an IO stream, or an on-disk file into a row-oriented Vector of dictionaries without mapping to a target type.

Each element corresponds to one row; keys are the Parquet column names (as Strings) and values are the column's decoded Julia values (including missing for nulls).

Arguments

  • x: a Vector{UInt8} holding raw Parquet bytes, or
  • io: an IO stream positioned at the start of a Parquet file, or
  • path: a filesystem path to a .parquet file.

Keyword arguments

  • dict_type::Type{<:AbstractDict}: concrete dict type used for each row (default Dict{String,Any}).
  • Additional keyword arguments are forwarded to Parquet2.Dataset.

Returns

A Vector of dict_type - one entry per data row.

Throws

  • ParseError: if the input is not a valid Parquet file.

Examples

julia> struct Row; name::String; age::Int; end

julia> bytes = to_parquet([Row("Alice", 30), Row("Bob", 25)]);

julia> parse_parquet(bytes)
2-element Vector{Dict{String, Any}}:
 Dict("name" => "Alice", "age" => 30)
 Dict("name" => "Bob",   "age" => 25)

See also: from_parquet, try_from_parquet.

source

Deserialization

Serde.SerdeParquet.from_parquet — Function
from_parquet

Parse Parquet input and deserialize each row into type T, returning a Vector{T}.

Because Parquet is an inherently tabular (columnar) format, the target type must be a Vector{T}. Passing a non-vector type raises ArgumentError. x may be a Vector{UInt8}, an IO, or a filesystem path, exactly as accepted by parse_parquet.

Arguments

  • ::Type{Vector{T}}: the target row-vector type. Vector{Any} and the bare Vector shortcut both return the raw dictionary rows produced by parse_parquet.
  • x: Parquet bytes, IO, or path.
  • strategy: optional context object.

Keyword arguments

  • dict_type::Type{<:AbstractDict}: dict type used for intermediate rows (default Dict{String,Any}).
  • Other keyword arguments are forwarded to Parquet2.Dataset.

Returns

Vector{T} - one element per data row.

Throws

  • ArgumentError: if the target type is not a Vector{T}.
  • ParseError: if x is not valid Parquet.
  • MissingFieldError: if a required struct field is absent.
  • TypeMismatchError: if a column value cannot be coerced to the field type.

Examples

julia> struct Row; name::String; age::Int; end

julia> bytes = to_parquet([Row("Alice", 30), Row("Bob", 25)]);

julia> from_parquet(Vector{Row}, bytes)
2-element Vector{Row}:
 Row("Alice", 30)
 Row("Bob", 25)

See also: try_from_parquet, to_parquet, parse_parquet.

source

Serialization

Serde.SerdeParquet.to_parquet — Function
to_parquet

Serialize a vector of structs into Parquet bytes (or write them to io).

data is interpreted row-wise: every element becomes one row, and the struct's field names — passed through Serde.ser_name — become column names. Field values are converted via Serde.ser_value / Serde.ser_type; fields skipped by Serde.ser_skip are omitted from the schema. Each column is automatically narrowed to the tightest element type (with missing added when any value is nothing or missing).

Arguments

  • data::AbstractVector: rows to serialize. Must be non-empty — Parquet has no zero-row encoding.
  • io::IO: optional sink to write the bytes to.
  • strategy: optional context object.

Keyword arguments

  • Forwarded to Parquet2.writefile (e.g. compression_codec).

Returns

  • The 1-arg / 2-arg forms return Vector{UInt8} containing the Parquet file.
  • The IO-sink forms write to io and return nothing.

Throws

  • ArgumentError: if data is empty, if its row type has no fields, or if every field is excluded by Serde.ser_skip.

Examples

julia> struct Row; name::String; age::Int; end

julia> bytes = to_parquet([Row("Alice", 30), Row("Bob", 25)]);

julia> from_parquet(Vector{Row}, bytes)
2-element Vector{Row}:
 Row("Alice", 30)
 Row("Bob", 25)

See also: from_parquet.

source