org.apache.spark.sql.parquet

Class CatalystPrimitiveStringConverter

Object

parquet.io.api.Converter

parquet.io.api.PrimitiveConverter

org.apache.spark.sql.parquet.CatalystPrimitiveConverter

org.apache.spark.sql.parquet.CatalystPrimitiveStringConverter

public class CatalystPrimitiveStringConverter
extends CatalystPrimitiveConverter

A parquet.io.api.PrimitiveConverter that converts Parquet Binary to Catalyst String. Supports dictionaries to reduce Binary to String conversion overhead.

Follows pattern in Parquet of using dictionaries, where supported, for String conversion.

Constructor Summary

Constructors

Constructor and Description
CatalystPrimitiveStringConverter(CatalystConverter parent, int fieldIndex)

Method Summary

Methods

Modifier and Type	Method and Description
void	addBinary(parquet.io.api.Binary value)
void	addValueFromDictionary(int dictionaryId)
boolean	hasDictionarySupport()
void	setDictionary(parquet.column.Dictionary dictionary)

Methods inherited from class org.apache.spark.sql.parquet.CatalystPrimitiveConverter

addBoolean, addDouble, addFloat, addInt, addLong

Methods inherited from class parquet.io.api.PrimitiveConverter

asPrimitiveConverter, isPrimitive

Methods inherited from class parquet.io.api.Converter

asGroupConverter

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CatalystPrimitiveStringConverter

public CatalystPrimitiveStringConverter(CatalystConverter parent,
                                int fieldIndex)

Method Detail

hasDictionarySupport

public boolean hasDictionarySupport()

Overrides:

hasDictionarySupport in class parquet.io.api.PrimitiveConverter

setDictionary

public void setDictionary(parquet.column.Dictionary dictionary)

Overrides:

setDictionary in class parquet.io.api.PrimitiveConverter

addValueFromDictionary

public void addValueFromDictionary(int dictionaryId)

Overrides:

addValueFromDictionary in class parquet.io.api.PrimitiveConverter

addBinary

public void addBinary(parquet.io.api.Binary value)

Overrides:

addBinary in class CatalystPrimitiveConverter