| Modifier and Type | Method and Description |
|---|---|
T |
ParquetMetadataProviderBuilder.withReaderConfig(ParquetReaderConfig readerConfig) |
MetastoreParquetTableMetadataProvider.Builder |
MetastoreParquetTableMetadataProvider.Builder.withReaderConfig(ParquetReaderConfig readerConfig) |
| Modifier and Type | Field and Description |
|---|---|
protected ParquetReaderConfig |
BaseParquetMetadataProvider.readerConfig |
protected ParquetReaderConfig |
AbstractParquetRowGroupScan.readerConfig |
protected ParquetReaderConfig |
AbstractParquetGroupScan.readerConfig |
| Modifier and Type | Method and Description |
|---|---|
ParquetReaderConfig |
ParquetReaderConfig.Builder.build() |
static ParquetReaderConfig |
ParquetReaderConfig.getDefaultInstance() |
ParquetReaderConfig |
AbstractParquetRowGroupScan.getReaderConfig() |
ParquetReaderConfig |
AbstractParquetGroupScan.getReaderConfig() |
ParquetReaderConfig |
AbstractParquetRowGroupScan.getReaderConfigForSerialization() |
ParquetReaderConfig |
AbstractParquetGroupScan.getReaderConfigForSerialization() |
| Modifier and Type | Method and Description |
|---|---|
static void |
ParquetReaderUtility.transformBinaryInMetadataCache(MetadataBase.ParquetTableMetadataBase parquetTableMetadata,
ParquetReaderConfig readerConfig)
Transforms values for min / max binary statistics to byte array.
|
T |
BaseParquetMetadataProvider.Builder.withReaderConfig(ParquetReaderConfig readerConfig) |
| Modifier and Type | Method and Description |
|---|---|
static void |
Metadata.createMeta(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
ParquetReaderConfig readerConfig,
boolean allColumnsInteresting,
Set<SchemaPath> columnSet)
Create the parquet metadata file for the directory at the given path, and for any subdirectories.
|
static Metadata_V4.ParquetFileAndRowCountMetadata |
Metadata.getParquetFileMetadata_v4(Metadata_V4.ParquetTableMetadata_v4 parquetTableMetadata,
org.apache.parquet.hadoop.metadata.ParquetMetadata footer,
org.apache.hadoop.fs.FileStatus file,
org.apache.hadoop.fs.FileSystem fs,
boolean allColumnsInteresting,
boolean skipNonInteresting,
Set<SchemaPath> columnSet,
ParquetReaderConfig readerConfig)
Get the file metadata for a single file
|
static Metadata_V4.ParquetTableMetadata_v4 |
Metadata.getParquetTableMetadata(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
ParquetReaderConfig readerConfig)
Get the parquet metadata for the parquet files in the given directory, including those in subdirectories.
|
static Metadata_V4.ParquetTableMetadata_v4 |
Metadata.getParquetTableMetadata(Map<org.apache.hadoop.fs.FileStatus,org.apache.hadoop.fs.FileSystem> fileStatusMap,
ParquetReaderConfig readerConfig)
Get the parquet metadata for a list of parquet files.
|
static Metadata_V4.MetadataSummary |
Metadata.getSummary(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path metadataParentDir,
boolean autoRefreshTriggered,
ParquetReaderConfig readerConfig)
Reads the summary from the metadata cache file, if the cache file is stale recreates the metadata
|
static MetadataBase.ParquetTableMetadataBase |
Metadata.readBlockMeta(org.apache.hadoop.fs.FileSystem fs,
List<org.apache.hadoop.fs.Path> paths,
MetadataContext metaContext,
ParquetReaderConfig readerConfig)
Get the parquet metadata for the table by reading the metadata file
|
static ParquetTableMetadataDirs |
Metadata.readMetadataDirs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
MetadataContext metaContext,
ParquetReaderConfig readerConfig)
Get the parquet metadata for all subdirectories by reading the metadata file
|
| Constructor and Description |
|---|
FileMetadataCollector(org.apache.parquet.hadoop.metadata.ParquetMetadata metadata,
org.apache.hadoop.fs.FileStatus file,
org.apache.hadoop.fs.FileSystem fs,
boolean allColumnsInteresting,
boolean skipNonInteresting,
Set<SchemaPath> columnSet,
ParquetReaderConfig readerConfig) |
Copyright © 2021 The Apache Software Foundation. All rights reserved.