parquet.hadoop.codec

Class CodecConfig

java.lang.Object

parquet.hadoop.codec.CodecConfig

public abstract class CodecConfig
extends Object

Template class and factory for accessing codec related configurations in different APIs(mapreduce or mapred), use from(org.apache.hadoop.mapred.JobConf) for mapred API, use from(org.apache.hadoop.mapreduce.TaskAttemptContext) for mapreduce API

Author:

Tianshuo Deng

Constructor Summary

Constructors

Constructor and Description
CodecConfig()

Method Summary

Methods

Modifier and Type	Method and Description
static CodecConfig	from(org.apache.hadoop.mapred.JobConf jobConf) use mapred api to read codec config
static CodecConfig	from(org.apache.hadoop.mapreduce.TaskAttemptContext context) use mapreduce api to read codec config
CompressionCodecName	getCodec()
abstract org.apache.hadoop.conf.Configuration	getConfiguration()
abstract Class	getHadoopOutputCompressorClass(Class defaultCodec)
static CompressionCodecName	getParquetCompressionCodec(org.apache.hadoop.conf.Configuration configuration)
abstract boolean	isHadoopCompressionSet()
static boolean	isParquetCompressionSet(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CodecConfig

public CodecConfig()

Method Detail

isHadoopCompressionSet

public abstract boolean isHadoopCompressionSet()

Returns:

if a compress flag is set from hadoop

getHadoopOutputCompressorClass

public abstract Class getHadoopOutputCompressorClass(Class defaultCodec)

Parameters:

defaultCodec - the codec to use when codec is not set in conf

Returns:

codec specified in hadoop config

getConfiguration

public abstract org.apache.hadoop.conf.Configuration getConfiguration()

Returns:

configuration of the job

from

public static CodecConfig from(org.apache.hadoop.mapred.JobConf jobConf)

use mapred api to read codec config

Returns:

MapredCodecConfig

from

public static CodecConfig from(org.apache.hadoop.mapreduce.TaskAttemptContext context)

use mapreduce api to read codec config

Returns:

MapreduceCodecConfig

isParquetCompressionSet

public static boolean isParquetCompressionSet(org.apache.hadoop.conf.Configuration conf)

getParquetCompressionCodec

public static CompressionCodecName getParquetCompressionCodec(org.apache.hadoop.conf.Configuration configuration)

getCodec

public CompressionCodecName getCodec()