parquet.hadoop.codec

Class SnappyCodec

java.lang.Object

parquet.hadoop.codec.SnappyCodec

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.compress.CompressionCodec

public class SnappyCodec
extends Object
implements org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.compress.CompressionCodec

Snappy compression codec for Parquet. We do not use the default hadoop one since that codec adds a blocking structure around the base snappy compression algorithm. This is useful for hadoop to minimize the size of compression blocks for their file formats (e.g. SequenceFile) but is undesirable for Parquet since we already have the data page which provides that.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.io.compress.CompressionCodec

org.apache.hadoop.io.compress.CompressionCodec.Util

Constructor Summary

Constructors

Constructor and Description
SnappyCodec()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.io.compress.Compressor	createCompressor()
org.apache.hadoop.io.compress.Decompressor	createDecompressor()
org.apache.hadoop.io.compress.CompressionInputStream	createInputStream(InputStream stream)
org.apache.hadoop.io.compress.CompressionInputStream	createInputStream(InputStream stream, org.apache.hadoop.io.compress.Decompressor decompressor)
org.apache.hadoop.io.compress.CompressionOutputStream	createOutputStream(OutputStream stream)
org.apache.hadoop.io.compress.CompressionOutputStream	createOutputStream(OutputStream stream, org.apache.hadoop.io.compress.Compressor compressor)
Class<? extends org.apache.hadoop.io.compress.Compressor>	getCompressorType()
org.apache.hadoop.conf.Configuration	getConf()
Class<? extends org.apache.hadoop.io.compress.Decompressor>	getDecompressorType()
String	getDefaultExtension()
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnappyCodec

public SnappyCodec()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

createCompressor

public org.apache.hadoop.io.compress.Compressor createCompressor()

Specified by:

createCompressor in interface org.apache.hadoop.io.compress.CompressionCodec

createDecompressor

public org.apache.hadoop.io.compress.Decompressor createDecompressor()

Specified by:

createDecompressor in interface org.apache.hadoop.io.compress.CompressionCodec

createInputStream

public org.apache.hadoop.io.compress.CompressionInputStream createInputStream(InputStream stream)
                                                                       throws IOException

Specified by:

createInputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createInputStream

public org.apache.hadoop.io.compress.CompressionInputStream createInputStream(InputStream stream,
                                                                     org.apache.hadoop.io.compress.Decompressor decompressor)
                                                                       throws IOException

Specified by:

createInputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createOutputStream

public org.apache.hadoop.io.compress.CompressionOutputStream createOutputStream(OutputStream stream)
                                                                         throws IOException

Specified by:

createOutputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createOutputStream

public org.apache.hadoop.io.compress.CompressionOutputStream createOutputStream(OutputStream stream,
                                                                       org.apache.hadoop.io.compress.Compressor compressor)
                                                                         throws IOException

Specified by:

createOutputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

getCompressorType

public Class<? extends org.apache.hadoop.io.compress.Compressor> getCompressorType()

Specified by:

getCompressorType in interface org.apache.hadoop.io.compress.CompressionCodec

getDecompressorType

public Class<? extends org.apache.hadoop.io.compress.Decompressor> getDecompressorType()

Specified by:

getDecompressorType in interface org.apache.hadoop.io.compress.CompressionCodec

getDefaultExtension

public String getDefaultExtension()

Specified by:

getDefaultExtension in interface org.apache.hadoop.io.compress.CompressionCodec