Class HashJoinMemoryCalculatorImpl.PostBuildCalculationsImpl
java.lang.Object
org.apache.drill.exec.physical.impl.join.HashJoinMemoryCalculatorImpl.PostBuildCalculationsImpl
- All Implemented Interfaces:
HashJoinMemoryCalculator.PostBuildCalculations
,HashJoinStateCalculator<HashJoinMemoryCalculator>
- Enclosing class:
- HashJoinMemoryCalculatorImpl
public static class HashJoinMemoryCalculatorImpl.PostBuildCalculationsImpl
extends Object
implements HashJoinMemoryCalculator.PostBuildCalculations
In this state, we need to make sure there is enough room to spill probe side batches, if spilling is necessary. If there is not enough room, we have to evict build side partitions. If we don't have to evict build side partitions in this state, then we are done. If we do have to evict build side partitions then we have to recursively repeat the process.
Lifecycle
- Step 1: Call
initialize(boolean)
. This gives theHashJoinStateCalculator
additional information it needs to compute memory requirements. - Step 2: Call
shouldSpill()
. This tells you which build side partitions need to be spilled in order to make room for probing. - Step 3: Call
next()
. After you are done probing and partitioning the probe side, get the next calculator.
-
Field Summary
-
Constructor Summary
ConstructorDescriptionPostBuildCalculationsImpl
(boolean firstCycle, BatchSizePredictor probeSizePredictor, long memoryAvailable, long maxOutputBatchSize, int maxBatchNumRecordsProbe, int recordsPerPartitionBatchProbe, HashJoinMemoryCalculator.PartitionStatSet buildPartitionStatSet, Map<String, Long> keySizes, HashTableSizeCalculator hashTableSizeCalculator, HashJoinHelperSizeCalculator hashJoinHelperSizeCalculator, double fragmentationFactor, double safetyFactor, double loadFactor, boolean reserveHash, boolean semiJoin) -
Method Summary
Modifier and TypeMethodDescriptionstatic long
calculateReservedMemory
(int numSpilledPartitions, long maxProbeBatchSize, long maxOutputBatchSize, long partitionProbeBatchSize) static int
computeProbeRecordsPerBatch
(long memoryAvailable, int numPartitions, int defaultProbeRecordsPerBatch, int minProbeRecordsPerBatch, long maxProbeBatchSize, long maxOutputBatchSize, long defaultPartitionProbeBatchSize) long
long
long
int
getState()
The currentHashJoinState
corresponding to this calculator.void
initialize
(boolean probeEmpty) Initializes the calculator with additional information needed.next()
Signifies that the current state is complete and returns the nextHashJoinStateCalculator
.boolean
-
Field Details
-
MIN_RECORDS_PER_PARTITION_BATCH_PROBE
public static final int MIN_RECORDS_PER_PARTITION_BATCH_PROBE- See Also:
-
-
Constructor Details
-
PostBuildCalculationsImpl
public PostBuildCalculationsImpl(boolean firstCycle, BatchSizePredictor probeSizePredictor, long memoryAvailable, long maxOutputBatchSize, int maxBatchNumRecordsProbe, int recordsPerPartitionBatchProbe, HashJoinMemoryCalculator.PartitionStatSet buildPartitionStatSet, Map<String, Long> keySizes, HashTableSizeCalculator hashTableSizeCalculator, HashJoinHelperSizeCalculator hashJoinHelperSizeCalculator, double fragmentationFactor, double safetyFactor, double loadFactor, boolean reserveHash, boolean semiJoin)
-
-
Method Details
-
initialize
public void initialize(boolean probeEmpty) Description copied from interface:HashJoinMemoryCalculator.PostBuildCalculations
Initializes the calculator with additional information needed.- Specified by:
initialize
in interfaceHashJoinMemoryCalculator.PostBuildCalculations
- Parameters:
probeEmpty
- True if the probe is empty. False otherwise.
-
getProbeRecordsPerBatch
public int getProbeRecordsPerBatch()- Specified by:
getProbeRecordsPerBatch
in interfaceHashJoinMemoryCalculator.PostBuildCalculations
-
getIncomingProbeBatchReservedSpace
public long getIncomingProbeBatchReservedSpace() -
getPartitionProbeBatchSize
public long getPartitionProbeBatchSize() -
getConsumedMemory
public long getConsumedMemory() -
computeProbeRecordsPerBatch
public static int computeProbeRecordsPerBatch(long memoryAvailable, int numPartitions, int defaultProbeRecordsPerBatch, int minProbeRecordsPerBatch, long maxProbeBatchSize, long maxOutputBatchSize, long defaultPartitionProbeBatchSize) -
calculateReservedMemory
public static long calculateReservedMemory(int numSpilledPartitions, long maxProbeBatchSize, long maxOutputBatchSize, long partitionProbeBatchSize) -
shouldSpill
public boolean shouldSpill()- Specified by:
shouldSpill
in interfaceHashJoinMemoryCalculator.PostBuildCalculations
-
next
Description copied from interface:HashJoinStateCalculator
Signifies that the current state is complete and returns the nextHashJoinStateCalculator
. Returns null in the case where there is no next state.- Specified by:
next
in interfaceHashJoinStateCalculator<HashJoinMemoryCalculator>
- Returns:
- The next
HashJoinStateCalculator
or null if this was the last state.
-
getState
Description copied from interface:HashJoinStateCalculator
The currentHashJoinState
corresponding to this calculator.- Specified by:
getState
in interfaceHashJoinStateCalculator<HashJoinMemoryCalculator>
- Returns:
-
makeDebugString
- Specified by:
makeDebugString
in interfaceHashJoinMemoryCalculator.PostBuildCalculations
-