org.apache.drill.exec.store.parquet

Class ParquetSimpleTestFileGenerator

java.lang.Object

org.apache.drill.exec.store.parquet.ParquetSimpleTestFileGenerator

public class ParquetSimpleTestFileGenerator
extends Object

Use the Parquet examples to build a simple to use Parquet test file generator. Files currently generated by the main program were renamed to be used in the unit tests for DRILL-5971 and cover all logical types defined in Parquet Logical Types that are supported by Drill. Embedded types specified in the Parquet specification are not covered by the examples but can be added. To create a new parquet file, define a schema, create a GroupWriter based on the schema, then add values for individual records to the GroupWriter.
TODO: DRILL-7904. To run this tool please use 28.2-jre instead of 19.0 in main POM file

See Also:

TestFileGenerator, GroupWriteSupport, Dremel Example

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParquetSimpleTestFileGenerator.EnumType

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.parquet.schema.MessageType	complexNullableSchema
static String	complexNullableSchemaMsg
static org.apache.parquet.schema.MessageType	complexSchema
static String	complexSchemaMsg
static org.apache.hadoop.conf.Configuration	conf
static org.apache.hadoop.fs.Path	root
static org.apache.parquet.schema.MessageType	simpleNullableSchema
static String	simpleNullableSchemaMsg
static org.apache.parquet.schema.MessageType	simpleSchema
static String	simpleSchemaMsg

Constructor Summary

Constructors

Constructor and Description
ParquetSimpleTestFileGenerator()

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	initFile(String fileName)
static org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group>	initWriter(org.apache.parquet.schema.MessageType schema, String fileName, boolean dictEncoding)
static void	main(String[] args)
static void	writeComplexValues(org.apache.parquet.example.data.GroupFactory gf, org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group> complexWriter, boolean writeNulls)
static void	writeSimpleValues(org.apache.parquet.example.data.simple.SimpleGroupFactory sgf, org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group> simpleWriter, boolean writeNulls)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

root

public static org.apache.hadoop.fs.Path root

conf

public static org.apache.hadoop.conf.Configuration conf

simpleSchemaMsg

public static String simpleSchemaMsg

simpleNullableSchemaMsg

public static String simpleNullableSchemaMsg

complexSchemaMsg

public static String complexSchemaMsg

complexNullableSchemaMsg

public static String complexNullableSchemaMsg

simpleSchema

public static org.apache.parquet.schema.MessageType simpleSchema

complexSchema

public static org.apache.parquet.schema.MessageType complexSchema

simpleNullableSchema

public static org.apache.parquet.schema.MessageType simpleNullableSchema

complexNullableSchema

public static org.apache.parquet.schema.MessageType complexNullableSchema

Constructor Detail

ParquetSimpleTestFileGenerator

public ParquetSimpleTestFileGenerator()

Method Detail

initFile

public static org.apache.hadoop.fs.Path initFile(String fileName)

initWriter

public static org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group> initWriter(org.apache.parquet.schema.MessageType schema,
                                                                                                        String fileName,
                                                                                                        boolean dictEncoding)
                                                                                                 throws IOException

Throws:

IOException

writeComplexValues

public static void writeComplexValues(org.apache.parquet.example.data.GroupFactory gf,
                                      org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group> complexWriter,
                                      boolean writeNulls)
                               throws IOException

Throws:

IOException

writeSimpleValues

public static void writeSimpleValues(org.apache.parquet.example.data.simple.SimpleGroupFactory sgf,
                                     org.apache.parquet.hadoop.ParquetWriter<org.apache.parquet.example.data.Group> simpleWriter,
                                     boolean writeNulls)
                              throws IOException

Throws:

IOException

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException