Class QueueQueryParallelizer
java.lang.Object
org.apache.drill.exec.planner.fragment.SimpleParallelizer
org.apache.drill.exec.planner.fragment.QueueQueryParallelizer
- All Implemented Interfaces:
ParallelizationParameters
,QueryParallelizer
Paralellizer specialized for managing resources for a query based on Queues. This parallelizer
does not deal with increase/decrease of the parallelization of a query plan based on the current
cluster state. However, the memory assignment for each operator, minor fragment and major
fragment is based on the cluster state and provided queue configuration.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.drill.exec.planner.fragment.SimpleParallelizer
SimpleParallelizer.CountRequiredFragments
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
adjustMemory
(PlanningSet planningSet, Set<Wrapper> roots, Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints) Function called by the SimpleParallelizer to adjust the memory post parallelization.Methods inherited from class org.apache.drill.exec.planner.fragment.SimpleParallelizer
collectStatsAndParallelizeFragments, generateWorkUnit, generateWorkUnit, getAffinityFactor, getMaxGlobalWidth, getMaxWidthPerNode, getRootFragments, getSliceTarget, getSplitFragments, initFragmentWrappers, prepareFragmentTree, traverse
-
Constructor Details
-
QueueQueryParallelizer
-
-
Method Details
-
getMemory
- Specified by:
getMemory
in classSimpleParallelizer
-
adjustMemory
public void adjustMemory(PlanningSet planningSet, Set<Wrapper> roots, Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints) throws PhysicalOperatorSetupException Function called by the SimpleParallelizer to adjust the memory post parallelization. The overall logic is to traverse the fragment tree and call the MemoryCalculator on each major fragment. Once the memory is computed, resource requirement are accumulated per drillbit. The total resource requirements are used to select a queue. If the selected queue's resource limit is more/less than the query's requirement than the memory will be re-adjusted.- Specified by:
adjustMemory
in classSimpleParallelizer
- Parameters:
planningSet
- context of the fragments.roots
- root fragments.activeEndpoints
- currently active endpoints.- Throws:
PhysicalOperatorSetupException
-