Plank
Plank is a high-performance, managed .NET implementation for reading and writing Apache Parquet files.
dotnet add package Plank
Quick start
Declare a schema with [ParquetSchema]:
[ParquetSchema]
public sealed partial class Measurement
{
public int SensorId { get; init; }
public double Value { get; init; }
public DateTimeOffset RecordedAt { get; init; }
}The source generator adds strongly typed APIs to Measurement.
Write
using var output = File.Create("measurements.parquet");
using var writer = Measurement.CreateRowWriter(output);
for (var sensorId = 1; sensorId <= 3; sensorId++)
{
var row = writer.GetRow();
row.SensorId = sensorId;
row.Value = 20.0 + sensorId;
row.RecordedAt = DateTimeOffset.UtcNow;
}
writer.Complete();Read
using var input = File.OpenRead("measurements.parquet");
using var reader = Measurement.CreateRowReader(input);
foreach (var row in reader)
Console.WriteLine(
$"{row.SensorId}: {row.Value} " +
$"at {row.RecordedAt:O}");- Fast by designPreallocated, reusable buffers keep steady-state reads and writes allocation-free. Encoding, decoding, compression, and decompression are parallelized across columns and row groups.
- Source-generated APIsDeclare your Parquet schema as a C# model and get matching row readers and writers, dataset writers, projections, and column accessors at compile time.
- APIs for every workloadUse rows for application-shaped data, columns for columnar workloads, datasets for partitioned outputs, or the physical API for low-level control.
- Fine-grained configurationConfigure encodings and compression globally or per column, with control over row groups, pages, indexes, statistics, Bloom filters, and CRCs. Limit parallelism or use worker hooks to pin threads to specific CPUs.
- Built for evolving dataRead selected columns, handle compatible schema changes, and define schemas dynamically at runtime.
API guides
Most applications can stay with these generated row APIs. The dataset API routes rows into partitioned files, while the column and physical APIs provide progressively more control.