org.apache.drill.exec.ops

Class FragmentContextImpl

java.lang.Object

org.apache.drill.exec.ops.BaseFragmentContext

org.apache.drill.exec.ops.FragmentContextImpl

All Implemented Interfaces:

AutoCloseable, ExchangeFragmentContext, ExecutorFragmentContext, FragmentContext, RootFragmentContext, UdfUtilities

public class FragmentContextImpl
extends BaseFragmentContext
implements ExecutorFragmentContext

This is the core Context which implements all the Context interfaces:

• FragmentContext: A context provided to non-exchange operators.
• ExchangeFragmentContext: A context provided to exchange operators.
• RootFragmentContext: A context provided to fragment roots.
• ExecutorFragmentContext: A context used by the Drillbit.

The interfaces above expose resources to varying degrees. They are ordered from most restrictive (FragmentContext) to least restrictive (ExecutorFragmentContext).

Since FragmentContextImpl implements all of the interfaces listed above, the facade pattern is used in order to cast a FragmentContextImpl object to the desired interface where-ever it is needed. The facade pattern is powerful since it allows us to easily create minimal context objects to be used in unit tests. Without the use of interfaces and the facade pattern we would have to create a complete FragmentContextImpl object to unit test any part of the code that depends on a context.

General guideline: Use the most narrow interface for the task. For example, "internal" operators don't need visibility to the networking functionality. Using the narrow interface allows unit testing without using mocking libraries. Often, the surrounding structure already has exposed the most narrow interface. If there are opportunities to clean up older code, we can do so as needed to make testing easier.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.drill.exec.ops.FragmentContext

FragmentContext.ExecutorState

Field Summary

Fields inherited from interface org.apache.drill.exec.ops.UdfUtilities

INJECTABLE_GETTER_METHODS

Constructor Summary

Constructors

Constructor and Description
FragmentContextImpl(DrillbitContext dbContext, BitControl.PlanFragment fragment, FunctionImplementationRegistry funcRegistry) Create a FragmentContext instance for non-root fragment.
FragmentContextImpl(DrillbitContext dbContext, BitControl.PlanFragment fragment, QueryContext queryContext, UserClientConnection connection, FunctionImplementationRegistry funcRegistry) Create a FragmentContext instance for root fragment.
FragmentContextImpl(DrillbitContext dbContext, BitControl.PlanFragment fragment, UserClientConnection connection, FunctionImplementationRegistry funcRegistry) TODO: Remove this constructor when removing the SimpleRootExec (DRILL-2097).

Method Summary

Modifier and Type	Method and Description
void	addRuntimeFilter(RuntimeFilterWritable runtimeFilter) Add a RuntimeFilter when the RuntimeFilter receiver belongs to the same MinorFragment.
void	close()
void	fail(Throwable cause)
BufferAllocator	getAllocator() Deprecated.
Collection<CoordinationProtos.DrillbitEndpoint>	getBits()
protected BufferManager	getBufferManager()
IncomingBuffers	getBuffers()
ClusterCoordinator	getClusterCoordinator()
CodeCompiler	getCompiler()
DrillConfig	getConfig() Returns the Drill configuration for this run.
ValueHolder	getConstantValueHolder(String value, TypeProtos.MinorType type, org.apache.drill.shaded.guava.com.google.common.base.Function<DrillBuf,ValueHolder> holderInitializer) Works with value holders cache which holds constant value and its wrapper by type.
ContextInformation	getContextInformation() Get the context information such as: - query start time - root fragment timezone - query userName - system userName - default schema name in current session at the time of query.
Controller	getController()
AccountingDataTunnel	getDataTunnel(CoordinationProtos.DrillbitEndpoint endpoint)
CoordinationProtos.DrillbitEndpoint	getEndpoint()
ExecutionControls	getExecutionControls() Return the set of execution controls used to inject faults into running code for testing.
ExecutorService	getExecutor()
FragmentContext.ExecutorState	getExecutorState()
CoordinationProtos.DrillbitEndpoint	getForemanEndpoint()
String	getFragIdString()
org.apache.calcite.schema.SchemaPlus	getFullRootSchema()
ExecProtos.FragmentHandle	getHandle() The FragmentHandle for this Fragment
MetastoreRegistry	getMetastoreRegistry() Get instance of Metastore registry to obtain Metastore instance if needed.
BufferAllocator	getNewChildAllocator(String operatorName, int operatorId, long initialReservation, long maximumReservation)
OperatorCreatorRegistry	getOperatorCreatorRegistry()
OptionManager	getOptions() Returns a read-only version of the session options.
PartitionExplorer	getPartitionExplorer() A partition explorer allows UDFs to view the sub-partitions below a particular partition.
PhysicalPlanReader	getPlanReader()
QueryProfileStoreContext	getProfileStoreContext()
UserBitShared.QueryId	getQueryId()
String	getQueryIdString()
String	getQueryUserName()
BufferAllocator	getRootAllocator() Returns the root allocator for the Drillbit.
RuntimeFilterWritable	getRuntimeFilter(long rfIdentifier)
RuntimeFilterWritable	getRuntimeFilter(long rfIdentifier, long maxWaitTime, TimeUnit timeUnit) Get the RuntimeFilter with a blocking wait, if the waiting option is enabled.
ExecutorService	getScanDecodeExecutor()
ExecutorService	getScanExecutor()
QueryContext.SqlStatementType	getSQLStatementType() Returns the statement type (e.g.
FragmentStats	getStats()
Set<Map.Entry<UserServer.BitToUserConnection,UserServer.BitToUserConnectionConfig>>	getUserConnections()
AccountingUserConnection	getUserDataTunnel()
WorkEventBus	getWorkEventBus()
boolean	isBuffersDone()
boolean	isImpersonationEnabled()
boolean	isOverMemoryLimit()
boolean	isUserAuthenticationEnabled()
OperatorContext	newOperatorContext(PhysicalOperator popConfig)
OperatorContext	newOperatorContext(PhysicalOperator popConfig, OperatorStats stats)
void	requestMemory(RecordBatch requestor) An operator is experiencing memory pressure.
void	setBuffers(IncomingBuffers buffers)
void	setExecutorState(FragmentContext.ExecutorState executorState)
void	waitForSendComplete()

Methods inherited from class org.apache.drill.exec.ops.BaseFragmentContext

getFunctionRegistry, getImplementationClass, getImplementationClass, getImplementationClass, getImplementationClass, getManagedBuffer, getManagedBuffer, getManagedBufferManager, replace

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.ops.FragmentContext

getFunctionRegistry, getImplementationClass, getImplementationClass, getImplementationClass, getImplementationClass, getManagedBuffer, getManagedBuffer, getManagedBufferManager, replace

Constructor Detail

FragmentContextImpl

public FragmentContextImpl(DrillbitContext dbContext,
                           BitControl.PlanFragment fragment,
                           FunctionImplementationRegistry funcRegistry)
                    throws ExecutionSetupException

Create a FragmentContext instance for non-root fragment.

Parameters:

dbContext - DrillbitContext

fragment - Fragment implementation

funcRegistry - FunctionImplementationRegistry

Throws:

ExecutionSetupException - when unable to init fragment context

FragmentContextImpl

public FragmentContextImpl(DrillbitContext dbContext,
                           BitControl.PlanFragment fragment,
                           QueryContext queryContext,
                           UserClientConnection connection,
                           FunctionImplementationRegistry funcRegistry)
                    throws ExecutionSetupException

Create a FragmentContext instance for root fragment.

Parameters:

dbContext - DrillbitContext

fragment - Fragment implementation

queryContext - QueryContext

connection - UserClientConnection

funcRegistry - FunctionImplementationRegistry

Throws:

ExecutionSetupException - when unable to init fragment context

FragmentContextImpl

public FragmentContextImpl(DrillbitContext dbContext,
                           BitControl.PlanFragment fragment,
                           UserClientConnection connection,
                           FunctionImplementationRegistry funcRegistry)
                    throws ExecutionSetupException

TODO: Remove this constructor when removing the SimpleRootExec (DRILL-2097). This is kept only to avoid modifying the long list of test files.

Throws:

ExecutionSetupException

Method Detail

getOptions

public OptionManager getOptions()

Description copied from interface: FragmentContext

Returns a read-only version of the session options.

Specified by:

getOptions in interface FragmentContext

Returns:

the session options

getPlanReader

public PhysicalPlanReader getPlanReader()

Specified by:

getPlanReader in interface ExecutorFragmentContext

getClusterCoordinator

public ClusterCoordinator getClusterCoordinator()

Specified by:

getClusterCoordinator in interface ExecutorFragmentContext

setBuffers

public void setBuffers(IncomingBuffers buffers)

Specified by:

setBuffers in interface ExecutorFragmentContext

getProfileStoreContext

public QueryProfileStoreContext getProfileStoreContext()

Specified by:

getProfileStoreContext in interface ExecutorFragmentContext

getUserConnections

public Set<Map.Entry<UserServer.BitToUserConnection,UserServer.BitToUserConnectionConfig>> getUserConnections()

Specified by:

getUserConnections in interface ExecutorFragmentContext

setExecutorState

public void setExecutorState(FragmentContext.ExecutorState executorState)

Specified by:

setExecutorState in interface RootFragmentContext

fail

public void fail(Throwable cause)

getFullRootSchema

public org.apache.calcite.schema.SchemaPlus getFullRootSchema()

Specified by:

getFullRootSchema in interface FragmentContext

getStats

public FragmentStats getStats()

Specified by:

getStats in interface RootFragmentContext

getBits

public Collection<CoordinationProtos.DrillbitEndpoint> getBits()

Specified by:

getBits in interface ExecutorFragmentContext

getContextInformation

public ContextInformation getContextInformation()

Description copied from interface: UdfUtilities

Get the context information such as: - query start time - root fragment timezone - query userName - system userName - default schema name in current session at the time of query.

Specified by:

getContextInformation in interface UdfUtilities

Returns:

- ContextInformation

getForemanEndpoint

public CoordinationProtos.DrillbitEndpoint getForemanEndpoint()

Specified by:

getForemanEndpoint in interface ExecutorFragmentContext

getEndpoint

public CoordinationProtos.DrillbitEndpoint getEndpoint()

Specified by:

getEndpoint in interface ExecutorFragmentContext

getController

public Controller getController()

Specified by:

getController in interface ExchangeFragmentContext

getOperatorCreatorRegistry

public OperatorCreatorRegistry getOperatorCreatorRegistry()

Specified by:

getOperatorCreatorRegistry in interface ExecutorFragmentContext

getScanDecodeExecutor

public ExecutorService getScanDecodeExecutor()

Specified by:

getScanDecodeExecutor in interface FragmentContext

getScanExecutor

public ExecutorService getScanExecutor()

Specified by:

getScanExecutor in interface FragmentContext

getHandle

public ExecProtos.FragmentHandle getHandle()

The FragmentHandle for this Fragment

Specified by:

getHandle in interface FragmentContext

Returns:

FragmentHandle

getFragIdString

public String getFragIdString()

Specified by:

getFragIdString in interface FragmentContext

isUserAuthenticationEnabled

public boolean isUserAuthenticationEnabled()

Specified by:

isUserAuthenticationEnabled in interface ExecutorFragmentContext

addRuntimeFilter

public void addRuntimeFilter(RuntimeFilterWritable runtimeFilter)

Description copied from interface: FragmentContext

Add a RuntimeFilter when the RuntimeFilter receiver belongs to the same MinorFragment.

Specified by:

addRuntimeFilter in interface FragmentContext

Parameters:

runtimeFilter - runtime filter

getRuntimeFilter

public RuntimeFilterWritable getRuntimeFilter(long rfIdentifier)

Specified by:

getRuntimeFilter in interface FragmentContext

getRuntimeFilter

public RuntimeFilterWritable getRuntimeFilter(long rfIdentifier,
                                              long maxWaitTime,
                                              TimeUnit timeUnit)

Description copied from interface: FragmentContext

Get the RuntimeFilter with a blocking wait, if the waiting option is enabled.

Specified by:

getRuntimeFilter in interface FragmentContext

Parameters:

rfIdentifier - runtime filter identifier

maxWaitTime - max wait time

timeUnit - time unit

Returns:

the RFW or null

getAllocator

@Deprecated
public BufferAllocator getAllocator()

Deprecated.

Get this fragment's allocator.

Specified by:

getAllocator in interface FragmentContext

Returns:

the allocator

getRootAllocator

public BufferAllocator getRootAllocator()

Description copied from interface: ExecutorFragmentContext

Returns the root allocator for the Drillbit.

Specified by:

getRootAllocator in interface ExecutorFragmentContext

getNewChildAllocator

public BufferAllocator getNewChildAllocator(String operatorName,
                                            int operatorId,
                                            long initialReservation,
                                            long maximumReservation)
                                     throws OutOfMemoryException

Specified by:

getNewChildAllocator in interface FragmentContext

Throws:

OutOfMemoryException

isOverMemoryLimit

public boolean isOverMemoryLimit()

getCompiler

public CodeCompiler getCompiler()

Specified by:

getCompiler in interface FragmentContext

getUserDataTunnel

public AccountingUserConnection getUserDataTunnel()

Specified by:

getUserDataTunnel in interface ExchangeFragmentContext

getDataTunnel

public AccountingDataTunnel getDataTunnel(CoordinationProtos.DrillbitEndpoint endpoint)

Specified by:

getDataTunnel in interface ExchangeFragmentContext

getBuffers

public IncomingBuffers getBuffers()

Specified by:

getBuffers in interface ExchangeFragmentContext

newOperatorContext

public OperatorContext newOperatorContext(PhysicalOperator popConfig,
                                          OperatorStats stats)
                                   throws OutOfMemoryException

Specified by:

newOperatorContext in interface FragmentContext

Throws:

OutOfMemoryException

newOperatorContext

public OperatorContext newOperatorContext(PhysicalOperator popConfig)
                                   throws OutOfMemoryException

Specified by:

newOperatorContext in interface FragmentContext

Throws:

OutOfMemoryException

getConfig

public DrillConfig getConfig()

Description copied from interface: FragmentContext

Returns the Drill configuration for this run. Note that the config is global and immutable.

Specified by:

getConfig in interface FragmentContext

Returns:

the Drill configuration

getExecutorState

public FragmentContext.ExecutorState getExecutorState()

Specified by:

getExecutorState in interface FragmentContext

getExecutionControls

public ExecutionControls getExecutionControls()

Description copied from interface: FragmentContext

Return the set of execution controls used to inject faults into running code for testing.

Specified by:

getExecutionControls in interface FragmentContext

Returns:

the execution controls

getQueryUserName

public String getQueryUserName()

Specified by:

getQueryUserName in interface FragmentContext

Overrides:

getQueryUserName in class BaseFragmentContext

getQueryId

public UserBitShared.QueryId getQueryId()

Specified by:

getQueryId in interface FragmentContext

Overrides:

getQueryId in class BaseFragmentContext

Returns:

ID UUID of the current query

getQueryIdString

public String getQueryIdString()

Specified by:

getQueryIdString in interface FragmentContext

Overrides:

getQueryIdString in class BaseFragmentContext

Returns:

The string representation of the ID UUID of the current query

isImpersonationEnabled

public boolean isImpersonationEnabled()

Specified by:

isImpersonationEnabled in interface FragmentContext

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface FragmentContext

getPartitionExplorer

public PartitionExplorer getPartitionExplorer()

Description copied from interface: UdfUtilities

A partition explorer allows UDFs to view the sub-partitions below a particular partition. This allows for the implementation of UDFs to query against the partition information, without having to read the actual data contained in the partition. This interface is designed for UDFs that take only constant inputs, as this interface will only be useful if we can evaluate the constant UDF at planning time. Any function defined to use this interface that is not evaluated at planning time by the constant folding rule will be querying the storage plugin for meta-data for each record processed. Be sure to check the query plans to see that this expression has already been evaluated during planning if you write UDFs against this interface. See DirectoryExplorers for example usages of this interface.

Specified by:

getPartitionExplorer in interface UdfUtilities

Returns:

- an object for exploring partitions of all available schemas

getConstantValueHolder

public ValueHolder getConstantValueHolder(String value,
                                          TypeProtos.MinorType type,
                                          org.apache.drill.shaded.guava.com.google.common.base.Function<DrillBuf,ValueHolder> holderInitializer)

Description copied from interface: UdfUtilities

Works with value holders cache which holds constant value and its wrapper by type. If value is absent uses holderInitializer to create holder and adds it to cache.

Specified by:

getConstantValueHolder in interface UdfUtilities

Returns:

- a wrapper object for an constant value.

getExecutor

public ExecutorService getExecutor()

Specified by:

getExecutor in interface FragmentContext

waitForSendComplete

public void waitForSendComplete()

Specified by:

waitForSendComplete in interface ExchangeFragmentContext

getWorkEventBus

public WorkEventBus getWorkEventBus()

Specified by:

getWorkEventBus in interface ExecutorFragmentContext

isBuffersDone

public boolean isBuffersDone()

getBufferManager

protected BufferManager getBufferManager()

Specified by:

getBufferManager in class BaseFragmentContext

getSQLStatementType

public QueryContext.SqlStatementType getSQLStatementType()

Description copied from interface: FragmentContext

Returns the statement type (e.g. SELECT, CTAS, ANALYZE) from the query context.

Specified by:

getSQLStatementType in interface FragmentContext

Overrides:

getSQLStatementType in class BaseFragmentContext

Returns:

query statement type QueryContext.SqlStatementType, if known.

getMetastoreRegistry

public MetastoreRegistry getMetastoreRegistry()

Description copied from interface: FragmentContext

Get instance of Metastore registry to obtain Metastore instance if needed.

Specified by:

getMetastoreRegistry in interface FragmentContext

Returns:

Metastore registry

requestMemory

public void requestMemory(RecordBatch requestor)

Description copied from interface: FragmentContext

An operator is experiencing memory pressure. Asks the fragment executor to poll all operators to release an optional memory (such as by spilling.) The request is advisory. The caller should again try to allocate memory, and if the second request fails, throw an OutOfMemoryException.

Specified by:

requestMemory in interface FragmentContext