org.apache.drill.exec.physical.impl.join

Class BatchSizePredictorImpl

java.lang.Object

org.apache.drill.exec.physical.impl.join.BatchSizePredictorImpl

All Implemented Interfaces:

BatchSizePredictor

public class BatchSizePredictorImpl
extends Object
implements BatchSizePredictor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BatchSizePredictorImpl.Factory

Constructor Summary

Constructors

Constructor and Description
BatchSizePredictorImpl(RecordBatch batch, double fragmentationFactor, double safetyFactor)

Method Summary

Modifier and Type	Method and Description
static long	computeMaxBatchSize(long incomingBatchSize, int incomingNumRecords, int desiredNumRecords, double fragmentationFactor, double safetyFactor, boolean reserveHash)
static long	computeMaxBatchSizeNoHash(long incomingBatchSize, int incomingNumRecords, int desiredNumRecords, double fragmentationFactor, double safetyFactor)
static long	computePartitionBatchSize(long incomingBatchSize, int incomingNumRecords, int desiredNumRecords)
static long	computeValueVectorSize(long numRecords, long byteSize)
static long	computeValueVectorSize(long numRecords, long byteSize, double safetyFactor)
long	getBatchSize() Gets the batchSize computed in the call to BatchSizePredictor.updateStats().
static long	getBatchSizeEstimate(RecordBatch recordBatch)
int	getNumRecords() Gets the number of records computed in the call to BatchSizePredictor.updateStats().
boolean	hadDataLastTime() True if the input batch had records in the last call to BatchSizePredictor.updateStats().
long	predictBatchSize(int desiredNumRecords, boolean reserveHash) Predicts the size of a batch using the current collected stats.
static long	roundUpToPowerOf2(long num)
void	updateStats() This method can be called multiple times to collect stats about the latest data in the provided record batch.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchSizePredictorImpl

public BatchSizePredictorImpl(RecordBatch batch,
                              double fragmentationFactor,
                              double safetyFactor)

Method Detail

getBatchSize

public long getBatchSize()

Description copied from interface: BatchSizePredictor

Gets the batchSize computed in the call to BatchSizePredictor.updateStats(). Returns 0 if BatchSizePredictor.hadDataLastTime() is false.

Specified by:

getBatchSize in interface BatchSizePredictor

Returns:

Gets the batchSize computed in the call to BatchSizePredictor.updateStats(). Returns 0 if BatchSizePredictor.hadDataLastTime() is false.

getNumRecords

public int getNumRecords()

Description copied from interface: BatchSizePredictor

Gets the number of records computed in the call to BatchSizePredictor.updateStats(). Returns 0 if BatchSizePredictor.hadDataLastTime() is false.

Specified by:

getNumRecords in interface BatchSizePredictor

Returns:

Gets the number of records computed in the call to BatchSizePredictor.updateStats(). Returns 0 if BatchSizePredictor.hadDataLastTime() is false.

hadDataLastTime

public boolean hadDataLastTime()

Description copied from interface: BatchSizePredictor

True if the input batch had records in the last call to BatchSizePredictor.updateStats(). False otherwise.

Specified by:

hadDataLastTime in interface BatchSizePredictor

Returns:

True if the input batch had records in the last call to BatchSizePredictor.updateStats(). False otherwise.

updateStats

public void updateStats()

Description copied from interface: BatchSizePredictor

This method can be called multiple times to collect stats about the latest data in the provided record batch. These stats are used to predict batch sizes. If the batch currently has no data, this method is a noop. This method must be called at least once before BatchSizePredictor.predictBatchSize(int, boolean).

Specified by:

updateStats in interface BatchSizePredictor

predictBatchSize

public long predictBatchSize(int desiredNumRecords,
                             boolean reserveHash)

Description copied from interface: BatchSizePredictor

Predicts the size of a batch using the current collected stats.

Specified by:

predictBatchSize in interface BatchSizePredictor

Parameters:

desiredNumRecords - The number of records contained in the batch whose size we want to predict.

reserveHash - Whether or not to include a column containing hash values.

Returns:

The size of the predicted batch.

computeValueVectorSize

public static long computeValueVectorSize(long numRecords,
                                          long byteSize)

computeValueVectorSize

public static long computeValueVectorSize(long numRecords,
                                          long byteSize,
                                          double safetyFactor)

roundUpToPowerOf2

public static long roundUpToPowerOf2(long num)

computeMaxBatchSizeNoHash

public static long computeMaxBatchSizeNoHash(long incomingBatchSize,
                                             int incomingNumRecords,
                                             int desiredNumRecords,
                                             double fragmentationFactor,
                                             double safetyFactor)

computeMaxBatchSize

public static long computeMaxBatchSize(long incomingBatchSize,
                                       int incomingNumRecords,
                                       int desiredNumRecords,
                                       double fragmentationFactor,
                                       double safetyFactor,
                                       boolean reserveHash)

computePartitionBatchSize

public static long computePartitionBatchSize(long incomingBatchSize,
                                             int incomingNumRecords,
                                             int desiredNumRecords)

getBatchSizeEstimate

public static long getBatchSizeEstimate(RecordBatch recordBatch)