Class RowWriterBase<TSlot>

Namespace

Plank.Writing

Assembly

Plank.dll

Provides the parallel serialization pipeline used by generated row writers.

public abstract class RowWriterBase<TSlot> : IDisposable where TSlot : class

Type Parameters

TSlot

The generated row-buffer slot type.

Inheritance

object

RowWriterBase<TSlot>

Implements

IDisposable

Derived

PipelineRowWriterBase<TSlot>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This unstable API supports Plank-generated code and is not intended for direct use by applications. Row submission is single-producer; it must not overlap completion, reset, or disposal.

Constructors

RowWriterBase(IParquetWriteSource, ParquetFilePath, ParquetSchema, uint, ParquetWriterOptions)

Initializes a rolling pipeline used by a generated row writer.

protected RowWriterBase(IParquetWriteSource file, ParquetFilePath filePath, ParquetSchema schema, uint maxParallelism, ParquetWriterOptions options)

Parameters

file IParquetWriteSource

The reusable destination used for each produced file.

filePath ParquetFilePath

Selects the path of each produced file.

schema ParquetSchema

The generated Parquet schema.

maxParallelism uint

The maximum number of serialization workers.

options ParquetWriterOptions

The Parquet writer options.

RowWriterBase(Stream, ParquetSchema, uint, ParquetWriterOptions)

Initializes the pipeline used by a generated row writer.

protected RowWriterBase(Stream stream, ParquetSchema schema, uint maxParallelism, ParquetWriterOptions options)

Parameters

stream Stream

The destination stream.

schema ParquetSchema

The generated Parquet schema.

maxParallelism uint

The maximum number of serialization workers.

options ParquetWriterOptions

The Parquet writer options.

Properties

WorkerThreadNamePrefix

Gets the name prefix used for serialization worker threads.

protected virtual string WorkerThreadNamePrefix { get; }

Property Value

string

Methods

Complete(TSlot, bool)

Completes the current serialization cycle and closes the Parquet file.

protected void Complete(TSlot activeSlot, bool hasRows)

Parameters

activeSlot TSlot

The generated writer's active slot.

hasRows bool

Whether the active slot contains rows to write.

CreateSlot(ParquetWriter)

Creates a generated row-buffer slot.

protected abstract TSlot CreateSlot(ParquetWriter writer)

Parameters

writer ParquetWriter

The destination Parquet writer.

Returns

TSlot

A new buffer slot.

Dispose()

Aborts the pipeline and releases its persistent workers, pooled buffers, and destination.

public void Dispose()

Remarks

Disposing does not complete the Parquet file. Call the generated writer's Complete() method before disposal to commit a valid file. Once disposed, the writer cannot be reused.

EnqueueAndTakeFree(TSlot)

Queues a filled slot and obtains the next writable slot.

protected TSlot EnqueueAndTakeFree(TSlot slot)

Parameters

slot TSlot

The filled slot to queue.

Returns

TSlot

The next writable slot.

InitializeSlots()

Creates the generated slots and starts serialization workers.

protected void InitializeSlots()

OnSlotWritten(TSlot)

Handles successful writing of a generated slot.

protected virtual void OnSlotWritten(TSlot slot)

Parameters

slot TSlot

The slot that was written.

ResetPipeline(Stream)

Resets a completed generated row-writer pipeline to a new destination stream.

protected void ResetPipeline(Stream stream)

Parameters

stream Stream

The new destination stream.

ResetSlotForReuse(TSlot)

Resets a generated slot before it is reused.

protected abstract void ResetSlotForReuse(TSlot slot)

Parameters

slot TSlot

The slot to reset.

SerializeSlot(TSlot)

Serializes a generated row-buffer slot.

protected abstract void SerializeSlot(TSlot slot)

Parameters

slot TSlot

The slot to serialize.

TakeInitialSlot()

Takes the first slot used by a generated writer.

protected TSlot TakeInitialSlot()

Returns

TSlot

The first writable slot.

ThrowIfDisposed()

Throws if this writer has been disposed.

protected void ThrowIfDisposed()

ThrowIfFaulted()

Rethrows a failure reported by a serialization worker.

protected void ThrowIfFaulted()

WriteSerializedSlot(TSlot, RowGroupWriter)

Writes a serialized slot to a row group.

protected abstract void WriteSerializedSlot(TSlot slot, RowGroupWriter rowGroupWriter)

Parameters

slot TSlot

The serialized slot.

rowGroupWriter RowGroupWriter

The destination row-group writer.